[==========] Running 56 tests from 4 test suites.
[----------] Global test environment set-up.
[----------] 30 tests from SecurityITest
[ RUN ] SecurityITest.TestAuthorizationOnListTablets
Loading random data
Initializing database '/tmp/dist-test-taskRiQe_a/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790201539513534-1567-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Sep 23 22:12:19 dist-test-slave-s2fj krb5kdc[1574](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Sep 23 22:12:19 dist-test-slave-s2fj krb5kdc[1574](info): set up 2 sockets
Sep 23 22:12:19 dist-test-slave-s2fj krb5kdc[1574](info): commencing operation
krb5kdc: starting...
WARNING: Logging before InitGoogleLogging() is written to STDERR
W20260923 22:12:20.580806 1567 mini_kdc.cc:121] Time spent starting KDC: real 1.049s user 0.000s sys 0.005s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskRiQe_a/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790201539513534-1567-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskRiQe_a/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790201539513534-1567-0/krb5kdc/test-user.keytab.
Sep 23 22:12:20 dist-test-slave-s2fj krb5kdc[1574](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790201540, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-09-23T22:12:20Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-09-23T22:12:20Z Disabled control of system clock
WARNING: no policy specified for kudu/127.1.135.254@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.1.135.254@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.1.135.254 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskRiQe_a/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790201539513534-1567-0/krb5kdc/kudu_127.1.135.254.keytab.
Entry for principal kudu/127.1.135.254 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskRiQe_a/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790201539513534-1567-0/krb5kdc/kudu_127.1.135.254.keytab.
I20260923 22:12:20.691892 1567 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskRiQe_a/build/release/bin/kudu
/tmp/dist-test-taskRiQe_a/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskRiQe_a/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790201539513534-1567-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskRiQe_a/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790201539513534-1567-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskRiQe_a/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790201539513534-1567-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskRiQe_a/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790201539513534-1567-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.1.135.254:33987
--webserver_interface=127.1.135.254
--webserver_port=0
--keytab_file=/tmp/dist-test-taskRiQe_a/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790201539513534-1567-0/krb5kdc/kudu_127.1.135.254.keytab
--principal=kudu/127.1.135.254
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.1.135.212:36061
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.1.135.254:33987
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskRiQe_a/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790201539513534-1567-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskRiQe_a/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790201539513534-1567-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskRiQe_a/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790201539513534-1567-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260923 22:12:20.769033 1587 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260923 22:12:20.769243 1587 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260923 22:12:20.769276 1587 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260923 22:12:20.770361 1587 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260923 22:12:20.770426 1587 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260923 22:12:20.770448 1587 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260923 22:12:20.770471 1587 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260923 22:12:20.770499 1587 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260923 22:12:20.771893 1587 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.1.135.212:36061
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskRiQe_a/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790201539513534-1567-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskRiQe_a/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790201539513534-1567-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.1.135.254:33987
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskRiQe_a/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790201539513534-1567-0/krb5kdc/kudu_127.1.135.254.keytab
--principal=kudu/127.1.135.254
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.1.135.254:33987
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskRiQe_a/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790201539513534-1567-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.1.135.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.1587
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskRiQe_a/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790201539513534-1567-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 23 Sep 2026 20:53:37 UTC on ccycloud.quasar-rkviyx.root.comops.site
build id 4668083
I20260923 22:12:20.772208 1587 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260923 22:12:20.772475 1587 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260923 22:12:20.777170 1593 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260923 22:12:20.777266 1592 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260923 22:12:20.777181 1595 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260923 22:12:20.777388 1587 server_base.cc:1034] running on GCE node
I20260923 22:12:20.777657 1587 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260923 22:12:20.777921 1587 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260923 22:12:20.779079 1587 hybrid_clock.cc:648] HybridClock initialized: now 1790201540779055 us; error 45 us; skew 500 ppm
Sep 23 22:12:20 dist-test-slave-s2fj krb5kdc[1574](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790201540, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.135.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260923 22:12:20.781430 1587 init.cc:377] Logged in from keytab as kudu/127.1.135.254@KRBTEST.COM (short username kudu)
I20260923 22:12:20.781885 1587 webserver.cc:460] Webserver started at http://127.1.135.254:40939/ using document root <none> and password file <none>
I20260923 22:12:20.782128 1587 fs_manager.cc:359] Metadata directory not provided
I20260923 22:12:20.782195 1587 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260923 22:12:20.782343 1587 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260923 22:12:20.783325 1587 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskRiQe_a/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790201539513534-1567-0/minicluster-data/master-0/data/instance:
uuid: "89bcf0de4ba2423e851b551b9aaec928"
format_stamp: "Formatted at 2026-09-23 22:12:20 on dist-test-slave-s2fj"
server_key: "75925ab6f36b58dbd1d1ae611cdecb8a"
server_key_iv: "1492e1d230023c0f584dc0afaff0ef07"
server_key_version: "encryptionkey@0"
I20260923 22:12:20.783685 1587 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskRiQe_a/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790201539513534-1567-0/minicluster-data/master-0/wal/instance:
uuid: "89bcf0de4ba2423e851b551b9aaec928"
format_stamp: "Formatted at 2026-09-23 22:12:20 on dist-test-slave-s2fj"
server_key: "75925ab6f36b58dbd1d1ae611cdecb8a"
server_key_iv: "1492e1d230023c0f584dc0afaff0ef07"
server_key_version: "encryptionkey@0"
I20260923 22:12:20.785046 1587 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.003s
I20260923 22:12:20.785727 1602 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260923 22:12:20.785889 1587 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.001s
I20260923 22:12:20.785986 1587 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskRiQe_a/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790201539513534-1567-0/minicluster-data/master-0/data,/tmp/dist-test-taskRiQe_a/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790201539513534-1567-0/minicluster-data/master-0/wal
uuid: "89bcf0de4ba2423e851b551b9aaec928"
format_stamp: "Formatted at 2026-09-23 22:12:20 on dist-test-slave-s2fj"
server_key: "75925ab6f36b58dbd1d1ae611cdecb8a"
server_key_iv: "1492e1d230023c0f584dc0afaff0ef07"
server_key_version: "encryptionkey@0"
I20260923 22:12:20.786070 1587 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskRiQe_a/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790201539513534-1567-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskRiQe_a/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790201539513534-1567-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskRiQe_a/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790201539513534-1567-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260923 22:12:20.804340 1587 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260923 22:12:20.806072 1587 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260923 22:12:20.806250 1587 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260923 22:12:20.810055 1587 rpc_server.cc:307] RPC server started. Bound to: 127.1.135.254:33987
I20260923 22:12:20.810086 1654 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.1.135.254:33987 every 8 connection(s)
I20260923 22:12:20.810424 1587 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskRiQe_a/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790201539513534-1567-0/minicluster-data/master-0/data/info.pb
I20260923 22:12:20.811301 1655 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260923 22:12:20.813611 1655 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 89bcf0de4ba2423e851b551b9aaec928: Bootstrap starting.
I20260923 22:12:20.814175 1655 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 89bcf0de4ba2423e851b551b9aaec928: Neither blocks nor log segments found. Creating new log.
I20260923 22:12:20.814450 1655 log.cc:824] T 00000000000000000000000000000000 P 89bcf0de4ba2423e851b551b9aaec928: Log is configured to *not* fsync() on all Append() calls
I20260923 22:12:20.815085 1655 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 89bcf0de4ba2423e851b551b9aaec928: No bootstrap required, opened a new log
I20260923 22:12:20.815856 1567 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskRiQe_a/build/release/bin/kudu as pid 1587
I20260923 22:12:20.815961 1567 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskRiQe_a/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790201539513534-1567-0/minicluster-data/master-0/wal/instance
I20260923 22:12:20.816715 1567 external_mini_cluster.cc:1428] Setting key 5fb8709cd94172f1fbfb844b36f4e1a0
I20260923 22:12:20.817292 1655 raft_consensus.cc:348] T 00000000000000000000000000000000 P 89bcf0de4ba2423e851b551b9aaec928 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "89bcf0de4ba2423e851b551b9aaec928" member_type: VOTER last_known_addr { host: "127.1.135.254" port: 33987 } }
I20260923 22:12:20.817466 1655 raft_consensus.cc:374] T 00000000000000000000000000000000 P 89bcf0de4ba2423e851b551b9aaec928 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260923 22:12:20.817515 1655 raft_consensus.cc:729] T 00000000000000000000000000000000 P 89bcf0de4ba2423e851b551b9aaec928 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 89bcf0de4ba2423e851b551b9aaec928, State: Initialized, Role: FOLLOWER
I20260923 22:12:20.817620 1655 consensus_queue.cc:260] T 00000000000000000000000000000000 P 89bcf0de4ba2423e851b551b9aaec928 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "89bcf0de4ba2423e851b551b9aaec928" member_type: VOTER last_known_addr { host: "127.1.135.254" port: 33987 } }
I20260923 22:12:20.817731 1655 raft_consensus.cc:388] T 00000000000000000000000000000000 P 89bcf0de4ba2423e851b551b9aaec928 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260923 22:12:20.817771 1655 raft_consensus.cc:482] T 00000000000000000000000000000000 P 89bcf0de4ba2423e851b551b9aaec928 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260923 22:12:20.817816 1655 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 89bcf0de4ba2423e851b551b9aaec928 [term 0 FOLLOWER]: Advancing to term 1
I20260923 22:12:20.818840 1655 raft_consensus.cc:504] T 00000000000000000000000000000000 P 89bcf0de4ba2423e851b551b9aaec928 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "89bcf0de4ba2423e851b551b9aaec928" member_type: VOTER last_known_addr { host: "127.1.135.254" port: 33987 } }
I20260923 22:12:20.818946 1655 leader_election.cc:302] T 00000000000000000000000000000000 P 89bcf0de4ba2423e851b551b9aaec928 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 89bcf0de4ba2423e851b551b9aaec928; no voters:
I20260923 22:12:20.819092 1655 leader_election.cc:288] T 00000000000000000000000000000000 P 89bcf0de4ba2423e851b551b9aaec928 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260923 22:12:20.819118 1660 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 89bcf0de4ba2423e851b551b9aaec928 [term 1 FOLLOWER]: Leader election won for term 1
I20260923 22:12:20.819272 1660 raft_consensus.cc:686] T 00000000000000000000000000000000 P 89bcf0de4ba2423e851b551b9aaec928 [term 1 LEADER]: Becoming Leader. State: Replica: 89bcf0de4ba2423e851b551b9aaec928, State: Running, Role: LEADER
I20260923 22:12:20.819337 1655 sys_catalog.cc:564] T 00000000000000000000000000000000 P 89bcf0de4ba2423e851b551b9aaec928 [sys.catalog]: configured and running, proceeding with master startup.
I20260923 22:12:20.819351 1660 consensus_queue.cc:237] T 00000000000000000000000000000000 P 89bcf0de4ba2423e851b551b9aaec928 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "89bcf0de4ba2423e851b551b9aaec928" member_type: VOTER last_known_addr { host: "127.1.135.254" port: 33987 } }
I20260923 22:12:20.819751 1661 sys_catalog.cc:455] T 00000000000000000000000000000000 P 89bcf0de4ba2423e851b551b9aaec928 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "89bcf0de4ba2423e851b551b9aaec928" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "89bcf0de4ba2423e851b551b9aaec928" member_type: VOTER last_known_addr { host: "127.1.135.254" port: 33987 } } }
I20260923 22:12:20.819814 1661 sys_catalog.cc:458] T 00000000000000000000000000000000 P 89bcf0de4ba2423e851b551b9aaec928 [sys.catalog]: This master's current role is: LEADER
I20260923 22:12:20.819998 1662 sys_catalog.cc:455] T 00000000000000000000000000000000 P 89bcf0de4ba2423e851b551b9aaec928 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 89bcf0de4ba2423e851b551b9aaec928. Latest consensus state: current_term: 1 leader_uuid: "89bcf0de4ba2423e851b551b9aaec928" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "89bcf0de4ba2423e851b551b9aaec928" member_type: VOTER last_known_addr { host: "127.1.135.254" port: 33987 } } }
I20260923 22:12:20.820261 1666 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260923 22:12:20.820779 1666 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260923 22:12:20.821362 1662 sys_catalog.cc:458] T 00000000000000000000000000000000 P 89bcf0de4ba2423e851b551b9aaec928 [sys.catalog]: This master's current role is: LEADER
I20260923 22:12:20.822317 1666 catalog_manager.cc:1347] Generated new cluster ID: 6cf97fadf4d149e6886c7bd8f4cd4610
I20260923 22:12:20.822366 1666 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
Sep 23 22:12:20 dist-test-slave-s2fj krb5kdc[1574](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790201540, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.1.135.254@KRBTEST.COM
I20260923 22:12:20.832043 1659 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 22:12:20.817585 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49640 (local address 127.1.135.254:33987)
0923 22:12:20.817761 (+ 176us) server_negotiation.cc:206] Beginning negotiation
0923 22:12:20.817770 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0923 22:12:20.818103 (+ 333us) server_negotiation.cc:407] Connection header received
0923 22:12:20.820575 (+ 2472us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 22:12:20.820582 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 22:12:20.820645 (+ 63us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 22:12:20.820779 (+ 134us) server_negotiation.cc:226] Negotiated authn=SASL
0923 22:12:20.822864 (+ 2085us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 22:12:20.823771 (+ 907us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 22:12:20.824880 (+ 1109us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 22:12:20.825293 (+ 413us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 22:12:20.825409 (+ 116us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 22:12:20.828604 (+ 3195us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0923 22:12:20.828608 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0923 22:12:20.828608 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0923 22:12:20.828618 (+ 10us) server_negotiation.cc:922] calling sasl_server_start()
0923 22:12:20.830276 (+ 1658us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0923 22:12:20.830714 (+ 438us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0923 22:12:20.830719 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0923 22:12:20.830720 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0923 22:12:20.830810 (+ 90us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0923 22:12:20.831069 (+ 259us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0923 22:12:20.831070 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0923 22:12:20.831070 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0923 22:12:20.831294 (+ 224us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0923 22:12:20.831451 (+ 157us) server_negotiation.cc:1035] Waiting for connection context
0923 22:12:20.831710 (+ 259us) server_negotiation.cc:299] Negotiation successful
0923 22:12:20.831833 (+ 123us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":137,"thread_start_us":76,"threads_started":1}
I20260923 22:12:20.852669 1666 catalog_manager.cc:1370] Generated new certificate authority record
I20260923 22:12:20.853156 1666 catalog_manager.cc:1504] Loading token signing keys...
I20260923 22:12:20.862727 1666 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 89bcf0de4ba2423e851b551b9aaec928: Generated new TSK 0
I20260923 22:12:20.862901 1666 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.1.135.193@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.1.135.193@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.1.135.193 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskRiQe_a/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790201539513534-1567-0/krb5kdc/kudu_127.1.135.193.keytab.
Entry for principal kudu/127.1.135.193 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskRiQe_a/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790201539513534-1567-0/krb5kdc/kudu_127.1.135.193.keytab.
I20260923 22:12:20.889519 1567 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskRiQe_a/build/release/bin/kudu
/tmp/dist-test-taskRiQe_a/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskRiQe_a/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790201539513534-1567-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskRiQe_a/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790201539513534-1567-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskRiQe_a/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790201539513534-1567-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskRiQe_a/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790201539513534-1567-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.1.135.193:0
--local_ip_for_outbound_sockets=127.1.135.193
--webserver_interface=127.1.135.193
--webserver_port=0
--tserver_master_addrs=127.1.135.254:33987
--keytab_file=/tmp/dist-test-taskRiQe_a/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790201539513534-1567-0/krb5kdc/kudu_127.1.135.193.keytab
--principal=kudu/127.1.135.193
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.1.135.212:36061
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--tserver_enforce_access_control with env {KRB5CCNAME=/tmp/dist-test-taskRiQe_a/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790201539513534-1567-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskRiQe_a/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790201539513534-1567-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskRiQe_a/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790201539513534-1567-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260923 22:12:20.992616 1681 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260923 22:12:20.992813 1681 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260923 22:12:20.992835 1681 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260923 22:12:20.994165 1681 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260923 22:12:20.994220 1681 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260923 22:12:20.994247 1681 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.1.135.193
I20260923 22:12:20.995721 1681 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.1.135.212:36061
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskRiQe_a/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790201539513534-1567-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskRiQe_a/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790201539513534-1567-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskRiQe_a/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790201539513534-1567-0/krb5kdc/kudu_127.1.135.193.keytab
--principal=kudu/127.1.135.193
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.1.135.193:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskRiQe_a/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790201539513534-1567-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.1.135.193
--webserver_port=0
--tserver_master_addrs=127.1.135.254:33987
--tserver_enforce_access_control=true
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.1681
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.1.135.193
--log_dir=/tmp/dist-test-taskRiQe_a/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790201539513534-1567-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 23 Sep 2026 20:53:37 UTC on ccycloud.quasar-rkviyx.root.comops.site
build id 4668083
I20260923 22:12:20.995899 1681 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260923 22:12:20.996193 1681 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260923 22:12:21.001122 1687 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260923 22:12:21.001159 1686 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260923 22:12:21.001191 1689 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260923 22:12:21.001552 1681 server_base.cc:1034] running on GCE node
I20260923 22:12:21.001745 1681 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260923 22:12:21.001996 1681 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260923 22:12:21.003145 1681 hybrid_clock.cc:648] HybridClock initialized: now 1790201541003131 us; error 36 us; skew 500 ppm
Sep 23 22:12:21 dist-test-slave-s2fj krb5kdc[1574](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790201541, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.135.193@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260923 22:12:21.005650 1681 init.cc:377] Logged in from keytab as kudu/127.1.135.193@KRBTEST.COM (short username kudu)
I20260923 22:12:21.006054 1681 webserver.cc:460] Webserver started at http://127.1.135.193:39299/ using document root <none> and password file <none>
I20260923 22:12:21.006274 1681 fs_manager.cc:359] Metadata directory not provided
I20260923 22:12:21.006318 1681 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260923 22:12:21.006457 1681 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260923 22:12:21.007463 1681 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskRiQe_a/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790201539513534-1567-0/minicluster-data/ts-0/data/instance:
uuid: "5bfbcaaa024649bc8b1e14108200bcce"
format_stamp: "Formatted at 2026-09-23 22:12:21 on dist-test-slave-s2fj"
server_key: "48d41963517b53aa8e61ea7ce76542a2"
server_key_iv: "dc5b7ce4f2240d6cd204de9f6cce0421"
server_key_version: "encryptionkey@0"
I20260923 22:12:21.007819 1681 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskRiQe_a/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790201539513534-1567-0/minicluster-data/ts-0/wal/instance:
uuid: "5bfbcaaa024649bc8b1e14108200bcce"
format_stamp: "Formatted at 2026-09-23 22:12:21 on dist-test-slave-s2fj"
server_key: "48d41963517b53aa8e61ea7ce76542a2"
server_key_iv: "dc5b7ce4f2240d6cd204de9f6cce0421"
server_key_version: "encryptionkey@0"
I20260923 22:12:21.009138 1681 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.003s
I20260923 22:12:21.009826 1696 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260923 22:12:21.009974 1681 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.001s
I20260923 22:12:21.010061 1681 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskRiQe_a/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790201539513534-1567-0/minicluster-data/ts-0/data,/tmp/dist-test-taskRiQe_a/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790201539513534-1567-0/minicluster-data/ts-0/wal
uuid: "5bfbcaaa024649bc8b1e14108200bcce"
format_stamp: "Formatted at 2026-09-23 22:12:21 on dist-test-slave-s2fj"
server_key: "48d41963517b53aa8e61ea7ce76542a2"
server_key_iv: "dc5b7ce4f2240d6cd204de9f6cce0421"
server_key_version: "encryptionkey@0"
I20260923 22:12:21.010120 1681 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskRiQe_a/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790201539513534-1567-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskRiQe_a/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790201539513534-1567-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskRiQe_a/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790201539513534-1567-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260923 22:12:21.039224 1681 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260923 22:12:21.041018 1681 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260923 22:12:21.041127 1681 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260923 22:12:21.041313 1681 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260923 22:12:21.041564 1681 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260923 22:12:21.041611 1681 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260923 22:12:21.041632 1681 ts_tablet_manager.cc:610] Registered 0 tablets
I20260923 22:12:21.041646 1681 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260923 22:12:21.047226 1681 rpc_server.cc:307] RPC server started. Bound to: 127.1.135.193:36245
I20260923 22:12:21.047307 1809 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.1.135.193:36245 every 8 connection(s)
I20260923 22:12:21.047618 1681 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskRiQe_a/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790201539513534-1567-0/minicluster-data/ts-0/data/info.pb
Sep 23 22:12:21 dist-test-slave-s2fj krb5kdc[1574](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790201541, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.135.193@KRBTEST.COM for kudu/127.1.135.254@KRBTEST.COM
I20260923 22:12:21.054603 1567 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskRiQe_a/build/release/bin/kudu as pid 1681
I20260923 22:12:21.054702 1567 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskRiQe_a/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790201539513534-1567-0/minicluster-data/ts-0/wal/instance
I20260923 22:12:21.054865 1567 external_mini_cluster.cc:1428] Setting key 62fe33497b517980a44bc056cd4f6888
I20260923 22:12:21.056624 1812 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 22:12:21.048316 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.135.254:33987 (local address 127.1.135.193:59543)
0923 22:12:21.048466 (+ 150us) negotiation.cc:107] Waiting for socket to connect
0923 22:12:21.048515 (+ 49us) client_negotiation.cc:174] Beginning negotiation
0923 22:12:21.048966 (+ 451us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0923 22:12:21.049174 (+ 208us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0923 22:12:21.049177 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0923 22:12:21.049715 (+ 538us) client_negotiation.cc:189] Negotiated authn=SASL
0923 22:12:21.049894 (+ 179us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 22:12:21.049899 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 22:12:21.050532 (+ 633us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 22:12:21.050532 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 22:12:21.051291 (+ 759us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 22:12:21.051292 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 22:12:21.051627 (+ 335us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 22:12:21.051627 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 22:12:21.051719 (+ 92us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 22:12:21.052172 (+ 453us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0923 22:12:21.052179 (+ 7us) client_negotiation.cc:656] calling sasl_client_start()
0923 22:12:21.053453 (+ 1274us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0923 22:12:21.055221 (+ 1768us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0923 22:12:21.055229 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0923 22:12:21.055230 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0923 22:12:21.055526 (+ 296us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0923 22:12:21.055732 (+ 206us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0923 22:12:21.055732 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0923 22:12:21.055733 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0923 22:12:21.055824 (+ 91us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0923 22:12:21.056157 (+ 333us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0923 22:12:21.056161 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0923 22:12:21.056267 (+ 106us) client_negotiation.cc:769] Sending connection context
0923 22:12:21.056349 (+ 82us) client_negotiation.cc:240] Negotiation successful
0923 22:12:21.056458 (+ 109us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":112,"thread_start_us":86,"threads_started":1}
I20260923 22:12:21.056646 1659 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 22:12:21.048362 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.135.193:59543 (local address 127.1.135.254:33987)
0923 22:12:21.048405 (+ 43us) server_negotiation.cc:206] Beginning negotiation
0923 22:12:21.048407 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0923 22:12:21.048587 (+ 180us) server_negotiation.cc:407] Connection header received
0923 22:12:21.049052 (+ 465us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 22:12:21.049052 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 22:12:21.049063 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 22:12:21.049114 (+ 51us) server_negotiation.cc:226] Negotiated authn=SASL
0923 22:12:21.049972 (+ 858us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 22:12:21.050456 (+ 484us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 22:12:21.051367 (+ 911us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 22:12:21.051556 (+ 189us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 22:12:21.051604 (+ 48us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 22:12:21.053550 (+ 1946us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0923 22:12:21.053551 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0923 22:12:21.053551 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0923 22:12:21.053557 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0923 22:12:21.055098 (+ 1541us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0923 22:12:21.055606 (+ 508us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0923 22:12:21.055607 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0923 22:12:21.055608 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0923 22:12:21.055668 (+ 60us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0923 22:12:21.055890 (+ 222us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0923 22:12:21.055891 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0923 22:12:21.055891 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0923 22:12:21.056046 (+ 155us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0923 22:12:21.056159 (+ 113us) server_negotiation.cc:1035] Waiting for connection context
0923 22:12:21.056392 (+ 233us) server_negotiation.cc:299] Negotiation successful
0923 22:12:21.056500 (+ 108us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":21}
I20260923 22:12:21.057518 1810 heartbeater.cc:347] Connected to a master server at 127.1.135.254:33987
I20260923 22:12:21.057623 1810 heartbeater.cc:464] Registering TS with master...
I20260923 22:12:21.057804 1810 heartbeater.cc:511] Master 127.1.135.254:33987 requested a full tablet report, sending...
I20260923 22:12:21.058319 1619 ts_manager.cc:194] Registered new tserver with Master: 5bfbcaaa024649bc8b1e14108200bcce (127.1.135.193:36245)
I20260923 22:12:21.058904 1619 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.1.135.193@KRBTEST.COM'} at 127.1.135.193:59543
WARNING: no policy specified for kudu/127.1.135.194@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.1.135.194@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.1.135.194 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskRiQe_a/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790201539513534-1567-0/krb5kdc/kudu_127.1.135.194.keytab.
Entry for principal kudu/127.1.135.194 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskRiQe_a/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790201539513534-1567-0/krb5kdc/kudu_127.1.135.194.keytab.
I20260923 22:12:21.076963 1567 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskRiQe_a/build/release/bin/kudu
/tmp/dist-test-taskRiQe_a/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskRiQe_a/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790201539513534-1567-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskRiQe_a/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790201539513534-1567-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskRiQe_a/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790201539513534-1567-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskRiQe_a/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790201539513534-1567-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.1.135.194:0
--local_ip_for_outbound_sockets=127.1.135.194
--webserver_interface=127.1.135.194
--webserver_port=0
--tserver_master_addrs=127.1.135.254:33987
--keytab_file=/tmp/dist-test-taskRiQe_a/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790201539513534-1567-0/krb5kdc/kudu_127.1.135.194.keytab
--principal=kudu/127.1.135.194
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.1.135.212:36061
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--tserver_enforce_access_control with env {KRB5CCNAME=/tmp/dist-test-taskRiQe_a/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790201539513534-1567-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskRiQe_a/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790201539513534-1567-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskRiQe_a/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790201539513534-1567-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260923 22:12:21.155579 1815 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260923 22:12:21.155802 1815 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260923 22:12:21.155841 1815 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260923 22:12:21.157014 1815 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260923 22:12:21.157079 1815 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260923 22:12:21.157126 1815 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.1.135.194
I20260923 22:12:21.158569 1815 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.1.135.212:36061
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskRiQe_a/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790201539513534-1567-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskRiQe_a/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790201539513534-1567-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskRiQe_a/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790201539513534-1567-0/krb5kdc/kudu_127.1.135.194.keytab
--principal=kudu/127.1.135.194
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.1.135.194:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskRiQe_a/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790201539513534-1567-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.1.135.194
--webserver_port=0
--tserver_master_addrs=127.1.135.254:33987
--tserver_enforce_access_control=true
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.1815
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.1.135.194
--log_dir=/tmp/dist-test-taskRiQe_a/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790201539513534-1567-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 23 Sep 2026 20:53:37 UTC on ccycloud.quasar-rkviyx.root.comops.site
build id 4668083
I20260923 22:12:21.158823 1815 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260923 22:12:21.159068 1815 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260923 22:12:21.163738 1820 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260923 22:12:21.163796 1815 server_base.cc:1034] running on GCE node
W20260923 22:12:21.163733 1823 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260923 22:12:21.163714 1821 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260923 22:12:21.164244 1815 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260923 22:12:21.164490 1815 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260923 22:12:21.165701 1815 hybrid_clock.cc:648] HybridClock initialized: now 1790201541165687 us; error 27 us; skew 500 ppm
Sep 23 22:12:21 dist-test-slave-s2fj krb5kdc[1574](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790201541, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.135.194@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260923 22:12:21.174914 1815 init.cc:377] Logged in from keytab as kudu/127.1.135.194@KRBTEST.COM (short username kudu)
I20260923 22:12:21.175390 1815 webserver.cc:460] Webserver started at http://127.1.135.194:35043/ using document root <none> and password file <none>
I20260923 22:12:21.175633 1815 fs_manager.cc:359] Metadata directory not provided
I20260923 22:12:21.175698 1815 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260923 22:12:21.175841 1815 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260923 22:12:21.177425 1815 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskRiQe_a/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790201539513534-1567-0/minicluster-data/ts-1/data/instance:
uuid: "ef35bb854b0042b7a98cf666a58ead79"
format_stamp: "Formatted at 2026-09-23 22:12:21 on dist-test-slave-s2fj"
server_key: "f3fe1c0235d16b90a96f9905cdd2e8ae"
server_key_iv: "30094c0de6f097b6f6146a5adb9101fa"
server_key_version: "encryptionkey@0"
I20260923 22:12:21.177845 1815 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskRiQe_a/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790201539513534-1567-0/minicluster-data/ts-1/wal/instance:
uuid: "ef35bb854b0042b7a98cf666a58ead79"
format_stamp: "Formatted at 2026-09-23 22:12:21 on dist-test-slave-s2fj"
server_key: "f3fe1c0235d16b90a96f9905cdd2e8ae"
server_key_iv: "30094c0de6f097b6f6146a5adb9101fa"
server_key_version: "encryptionkey@0"
I20260923 22:12:21.179754 1815 fs_manager.cc:691] Time spent creating directory manager: real 0.002s user 0.003s sys 0.000s
I20260923 22:12:21.180696 1830 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260923 22:12:21.180894 1815 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.000s
I20260923 22:12:21.180994 1815 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskRiQe_a/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790201539513534-1567-0/minicluster-data/ts-1/data,/tmp/dist-test-taskRiQe_a/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790201539513534-1567-0/minicluster-data/ts-1/wal
uuid: "ef35bb854b0042b7a98cf666a58ead79"
format_stamp: "Formatted at 2026-09-23 22:12:21 on dist-test-slave-s2fj"
server_key: "f3fe1c0235d16b90a96f9905cdd2e8ae"
server_key_iv: "30094c0de6f097b6f6146a5adb9101fa"
server_key_version: "encryptionkey@0"
I20260923 22:12:21.181064 1815 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskRiQe_a/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790201539513534-1567-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskRiQe_a/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790201539513534-1567-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskRiQe_a/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790201539513534-1567-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260923 22:12:21.197917 1815 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260923 22:12:21.200584 1815 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260923 22:12:21.200719 1815 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260923 22:12:21.200937 1815 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260923 22:12:21.201292 1815 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260923 22:12:21.201345 1815 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260923 22:12:21.201378 1815 ts_tablet_manager.cc:610] Registered 0 tablets
I20260923 22:12:21.201406 1815 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260923 22:12:21.207271 1815 rpc_server.cc:307] RPC server started. Bound to: 127.1.135.194:42917
I20260923 22:12:21.207320 1943 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.1.135.194:42917 every 8 connection(s)
I20260923 22:12:21.207602 1815 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskRiQe_a/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790201539513534-1567-0/minicluster-data/ts-1/data/info.pb
I20260923 22:12:21.211333 1567 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskRiQe_a/build/release/bin/kudu as pid 1815
I20260923 22:12:21.211426 1567 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskRiQe_a/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790201539513534-1567-0/minicluster-data/ts-1/wal/instance
I20260923 22:12:21.211571 1567 external_mini_cluster.cc:1428] Setting key d9d436281ffb41ba8345b32fe7f8c284
Sep 23 22:12:21 dist-test-slave-s2fj krb5kdc[1574](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790201541, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.135.194@KRBTEST.COM for kudu/127.1.135.254@KRBTEST.COM
I20260923 22:12:21.216843 1659 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 22:12:21.208338 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.135.194:36747 (local address 127.1.135.254:33987)
0923 22:12:21.208379 (+ 41us) server_negotiation.cc:206] Beginning negotiation
0923 22:12:21.208381 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0923 22:12:21.208554 (+ 173us) server_negotiation.cc:407] Connection header received
0923 22:12:21.208990 (+ 436us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 22:12:21.208990 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 22:12:21.208999 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 22:12:21.209058 (+ 59us) server_negotiation.cc:226] Negotiated authn=SASL
0923 22:12:21.209902 (+ 844us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 22:12:21.210376 (+ 474us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 22:12:21.211347 (+ 971us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 22:12:21.211658 (+ 311us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 22:12:21.211731 (+ 73us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 22:12:21.213886 (+ 2155us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0923 22:12:21.213888 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0923 22:12:21.213888 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0923 22:12:21.213896 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0923 22:12:21.215370 (+ 1474us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0923 22:12:21.215879 (+ 509us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0923 22:12:21.215880 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0923 22:12:21.215880 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0923 22:12:21.215925 (+ 45us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0923 22:12:21.216250 (+ 325us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0923 22:12:21.216251 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0923 22:12:21.216251 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0923 22:12:21.216346 (+ 95us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0923 22:12:21.216396 (+ 50us) server_negotiation.cc:1035] Waiting for connection context
0923 22:12:21.216718 (+ 322us) server_negotiation.cc:299] Negotiation successful
0923 22:12:21.216782 (+ 64us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":17}
I20260923 22:12:21.217140 1946 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 22:12:21.208284 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.135.254:33987 (local address 127.1.135.194:36747)
0923 22:12:21.208433 (+ 149us) negotiation.cc:107] Waiting for socket to connect
0923 22:12:21.208471 (+ 38us) client_negotiation.cc:174] Beginning negotiation
0923 22:12:21.208912 (+ 441us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0923 22:12:21.209131 (+ 219us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0923 22:12:21.209134 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0923 22:12:21.209663 (+ 529us) client_negotiation.cc:189] Negotiated authn=SASL
0923 22:12:21.209831 (+ 168us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 22:12:21.209836 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 22:12:21.210452 (+ 616us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 22:12:21.210452 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 22:12:21.211267 (+ 815us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 22:12:21.211269 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 22:12:21.211738 (+ 469us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 22:12:21.211739 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 22:12:21.211880 (+ 141us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 22:12:21.212356 (+ 476us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0923 22:12:21.212366 (+ 10us) client_negotiation.cc:656] calling sasl_client_start()
0923 22:12:21.213795 (+ 1429us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0923 22:12:21.215456 (+ 1661us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0923 22:12:21.215462 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0923 22:12:21.215464 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0923 22:12:21.215805 (+ 341us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0923 22:12:21.215989 (+ 184us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0923 22:12:21.215991 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0923 22:12:21.215991 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0923 22:12:21.216181 (+ 190us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0923 22:12:21.216412 (+ 231us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0923 22:12:21.216417 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0923 22:12:21.216584 (+ 167us) client_negotiation.cc:769] Sending connection context
0923 22:12:21.216701 (+ 117us) client_negotiation.cc:240] Negotiation successful
0923 22:12:21.216869 (+ 168us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":113,"thread_start_us":89,"threads_started":1}
I20260923 22:12:21.217644 1944 heartbeater.cc:347] Connected to a master server at 127.1.135.254:33987
I20260923 22:12:21.217742 1944 heartbeater.cc:464] Registering TS with master...
I20260923 22:12:21.217937 1944 heartbeater.cc:511] Master 127.1.135.254:33987 requested a full tablet report, sending...
I20260923 22:12:21.218318 1619 ts_manager.cc:194] Registered new tserver with Master: ef35bb854b0042b7a98cf666a58ead79 (127.1.135.194:42917)
I20260923 22:12:21.218729 1619 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.1.135.194@KRBTEST.COM'} at 127.1.135.194:36747
WARNING: no policy specified for kudu/127.1.135.195@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.1.135.195@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.1.135.195 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskRiQe_a/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790201539513534-1567-0/krb5kdc/kudu_127.1.135.195.keytab.
Entry for principal kudu/127.1.135.195 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskRiQe_a/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790201539513534-1567-0/krb5kdc/kudu_127.1.135.195.keytab.
I20260923 22:12:21.236811 1567 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskRiQe_a/build/release/bin/kudu
/tmp/dist-test-taskRiQe_a/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskRiQe_a/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790201539513534-1567-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskRiQe_a/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790201539513534-1567-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskRiQe_a/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790201539513534-1567-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskRiQe_a/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790201539513534-1567-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.1.135.195:0
--local_ip_for_outbound_sockets=127.1.135.195
--webserver_interface=127.1.135.195
--webserver_port=0
--tserver_master_addrs=127.1.135.254:33987
--keytab_file=/tmp/dist-test-taskRiQe_a/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790201539513534-1567-0/krb5kdc/kudu_127.1.135.195.keytab
--principal=kudu/127.1.135.195
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.1.135.212:36061
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--tserver_enforce_access_control with env {KRB5CCNAME=/tmp/dist-test-taskRiQe_a/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790201539513534-1567-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskRiQe_a/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790201539513534-1567-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskRiQe_a/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790201539513534-1567-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260923 22:12:21.316546 1949 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260923 22:12:21.316753 1949 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260923 22:12:21.316787 1949 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260923 22:12:21.317915 1949 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260923 22:12:21.317983 1949 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260923 22:12:21.318029 1949 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.1.135.195
I20260923 22:12:21.319456 1949 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.1.135.212:36061
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskRiQe_a/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790201539513534-1567-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskRiQe_a/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790201539513534-1567-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskRiQe_a/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790201539513534-1567-0/krb5kdc/kudu_127.1.135.195.keytab
--principal=kudu/127.1.135.195
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.1.135.195:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskRiQe_a/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790201539513534-1567-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.1.135.195
--webserver_port=0
--tserver_master_addrs=127.1.135.254:33987
--tserver_enforce_access_control=true
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.1949
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.1.135.195
--log_dir=/tmp/dist-test-taskRiQe_a/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790201539513534-1567-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 23 Sep 2026 20:53:37 UTC on ccycloud.quasar-rkviyx.root.comops.site
build id 4668083
I20260923 22:12:21.319777 1949 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260923 22:12:21.319995 1949 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260923 22:12:21.324726 1954 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260923 22:12:21.324726 1955 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260923 22:12:21.324730 1957 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260923 22:12:21.325474 1949 server_base.cc:1034] running on GCE node
I20260923 22:12:21.325654 1949 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260923 22:12:21.325827 1949 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260923 22:12:21.326960 1949 hybrid_clock.cc:648] HybridClock initialized: now 1790201541326929 us; error 49 us; skew 500 ppm
Sep 23 22:12:21 dist-test-slave-s2fj krb5kdc[1574](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790201541, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.135.195@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260923 22:12:21.329502 1949 init.cc:377] Logged in from keytab as kudu/127.1.135.195@KRBTEST.COM (short username kudu)
I20260923 22:12:21.329921 1949 webserver.cc:460] Webserver started at http://127.1.135.195:44321/ using document root <none> and password file <none>
I20260923 22:12:21.330101 1949 fs_manager.cc:359] Metadata directory not provided
I20260923 22:12:21.330142 1949 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260923 22:12:21.330225 1949 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260923 22:12:21.331198 1949 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskRiQe_a/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790201539513534-1567-0/minicluster-data/ts-2/data/instance:
uuid: "54fd9a88c8174065bb8916a3914532db"
format_stamp: "Formatted at 2026-09-23 22:12:21 on dist-test-slave-s2fj"
server_key: "7da7daf46cc5c25f54250254c2c9bc66"
server_key_iv: "a7fca537769d1340ad517bfe7cb1ee4a"
server_key_version: "encryptionkey@0"
I20260923 22:12:21.331553 1949 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskRiQe_a/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790201539513534-1567-0/minicluster-data/ts-2/wal/instance:
uuid: "54fd9a88c8174065bb8916a3914532db"
format_stamp: "Formatted at 2026-09-23 22:12:21 on dist-test-slave-s2fj"
server_key: "7da7daf46cc5c25f54250254c2c9bc66"
server_key_iv: "a7fca537769d1340ad517bfe7cb1ee4a"
server_key_version: "encryptionkey@0"
I20260923 22:12:21.332857 1949 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.003s
I20260923 22:12:21.333487 1964 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260923 22:12:21.333659 1949 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.001s
I20260923 22:12:21.333745 1949 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskRiQe_a/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790201539513534-1567-0/minicluster-data/ts-2/data,/tmp/dist-test-taskRiQe_a/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790201539513534-1567-0/minicluster-data/ts-2/wal
uuid: "54fd9a88c8174065bb8916a3914532db"
format_stamp: "Formatted at 2026-09-23 22:12:21 on dist-test-slave-s2fj"
server_key: "7da7daf46cc5c25f54250254c2c9bc66"
server_key_iv: "a7fca537769d1340ad517bfe7cb1ee4a"
server_key_version: "encryptionkey@0"
I20260923 22:12:21.333815 1949 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskRiQe_a/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790201539513534-1567-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskRiQe_a/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790201539513534-1567-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskRiQe_a/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790201539513534-1567-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260923 22:12:21.356503 1949 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260923 22:12:21.358667 1949 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260923 22:12:21.358825 1949 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260923 22:12:21.359045 1949 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260923 22:12:21.359332 1949 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260923 22:12:21.359385 1949 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260923 22:12:21.359426 1949 ts_tablet_manager.cc:610] Registered 0 tablets
I20260923 22:12:21.359454 1949 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260923 22:12:21.366209 1949 rpc_server.cc:307] RPC server started. Bound to: 127.1.135.195:36605
I20260923 22:12:21.366257 2077 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.1.135.195:36605 every 8 connection(s)
I20260923 22:12:21.366566 1949 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskRiQe_a/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790201539513534-1567-0/minicluster-data/ts-2/data/info.pb
I20260923 22:12:21.371387 1567 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskRiQe_a/build/release/bin/kudu as pid 1949
I20260923 22:12:21.371490 1567 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskRiQe_a/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790201539513534-1567-0/minicluster-data/ts-2/wal/instance
I20260923 22:12:21.371666 1567 external_mini_cluster.cc:1428] Setting key 578df0de46efe8757e0f287ee8e3964c
Sep 23 22:12:21 dist-test-slave-s2fj krb5kdc[1574](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790201541, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.135.195@KRBTEST.COM for kudu/127.1.135.254@KRBTEST.COM
I20260923 22:12:21.375767 1659 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 22:12:21.367263 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.135.195:38965 (local address 127.1.135.254:33987)
0923 22:12:21.367329 (+ 66us) server_negotiation.cc:206] Beginning negotiation
0923 22:12:21.367331 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0923 22:12:21.367509 (+ 178us) server_negotiation.cc:407] Connection header received
0923 22:12:21.368449 (+ 940us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 22:12:21.368450 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 22:12:21.368459 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 22:12:21.368517 (+ 58us) server_negotiation.cc:226] Negotiated authn=SASL
0923 22:12:21.369378 (+ 861us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 22:12:21.369876 (+ 498us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 22:12:21.370783 (+ 907us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 22:12:21.370957 (+ 174us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 22:12:21.371008 (+ 51us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 22:12:21.373283 (+ 2275us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0923 22:12:21.373284 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0923 22:12:21.373284 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0923 22:12:21.373292 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0923 22:12:21.374411 (+ 1119us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0923 22:12:21.374820 (+ 409us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0923 22:12:21.374821 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0923 22:12:21.374822 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0923 22:12:21.374883 (+ 61us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0923 22:12:21.375146 (+ 263us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0923 22:12:21.375147 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0923 22:12:21.375147 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0923 22:12:21.375262 (+ 115us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0923 22:12:21.375354 (+ 92us) server_negotiation.cc:1035] Waiting for connection context
0923 22:12:21.375610 (+ 256us) server_negotiation.cc:299] Negotiation successful
0923 22:12:21.375682 (+ 72us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":39}
I20260923 22:12:21.375950 2080 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 22:12:21.367236 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.135.254:33987 (local address 127.1.135.195:38965)
0923 22:12:21.367389 (+ 153us) negotiation.cc:107] Waiting for socket to connect
0923 22:12:21.367428 (+ 39us) client_negotiation.cc:174] Beginning negotiation
0923 22:12:21.368363 (+ 935us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0923 22:12:21.368576 (+ 213us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0923 22:12:21.368579 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0923 22:12:21.369114 (+ 535us) client_negotiation.cc:189] Negotiated authn=SASL
0923 22:12:21.369306 (+ 192us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 22:12:21.369310 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 22:12:21.369950 (+ 640us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 22:12:21.369951 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 22:12:21.370713 (+ 762us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 22:12:21.370714 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 22:12:21.371024 (+ 310us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 22:12:21.371025 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 22:12:21.371116 (+ 91us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 22:12:21.371596 (+ 480us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0923 22:12:21.371607 (+ 11us) client_negotiation.cc:656] calling sasl_client_start()
0923 22:12:21.373187 (+ 1580us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0923 22:12:21.374515 (+ 1328us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0923 22:12:21.374521 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0923 22:12:21.374523 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0923 22:12:21.374734 (+ 211us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0923 22:12:21.374948 (+ 214us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0923 22:12:21.374949 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0923 22:12:21.374949 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0923 22:12:21.375069 (+ 120us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0923 22:12:21.375346 (+ 277us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0923 22:12:21.375352 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0923 22:12:21.375484 (+ 132us) client_negotiation.cc:769] Sending connection context
0923 22:12:21.375588 (+ 104us) client_negotiation.cc:240] Negotiation successful
0923 22:12:21.375732 (+ 144us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":115,"thread_start_us":79,"threads_started":1}
I20260923 22:12:21.376385 2078 heartbeater.cc:347] Connected to a master server at 127.1.135.254:33987
I20260923 22:12:21.376472 2078 heartbeater.cc:464] Registering TS with master...
I20260923 22:12:21.376601 2078 heartbeater.cc:511] Master 127.1.135.254:33987 requested a full tablet report, sending...
I20260923 22:12:21.376933 1619 ts_manager.cc:194] Registered new tserver with Master: 54fd9a88c8174065bb8916a3914532db (127.1.135.195:36605)
I20260923 22:12:21.377290 1619 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.1.135.195@KRBTEST.COM'} at 127.1.135.195:38965
I20260923 22:12:21.383535 1567 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
Sep 23 22:12:21 dist-test-slave-s2fj krb5kdc[1574](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790201541, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-user@KRBTEST.COM:
Sep 23 22:12:21 dist-test-slave-s2fj krb5kdc[1574](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790201541, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for kudu/127.1.135.193@KRBTEST.COM
I20260923 22:12:21.405347 2085 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 22:12:21.398343 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35544 (local address 127.1.135.193:36245)
0923 22:12:21.398541 (+ 198us) server_negotiation.cc:206] Beginning negotiation
0923 22:12:21.398545 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0923 22:12:21.398556 (+ 11us) server_negotiation.cc:407] Connection header received
0923 22:12:21.398579 (+ 23us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 22:12:21.398581 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 22:12:21.398593 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 22:12:21.398654 (+ 61us) server_negotiation.cc:226] Negotiated authn=SASL
0923 22:12:21.399147 (+ 493us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 22:12:21.399840 (+ 693us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 22:12:21.400569 (+ 729us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 22:12:21.400858 (+ 289us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 22:12:21.400911 (+ 53us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 22:12:21.402969 (+ 2058us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0923 22:12:21.402975 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0923 22:12:21.402976 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0923 22:12:21.402981 (+ 5us) server_negotiation.cc:922] calling sasl_server_start()
0923 22:12:21.404356 (+ 1375us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0923 22:12:21.404667 (+ 311us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0923 22:12:21.404669 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0923 22:12:21.404669 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0923 22:12:21.404718 (+ 49us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0923 22:12:21.404885 (+ 167us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0923 22:12:21.404886 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0923 22:12:21.404886 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0923 22:12:21.405003 (+ 117us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0923 22:12:21.405113 (+ 110us) server_negotiation.cc:1035] Waiting for connection context
0923 22:12:21.405166 (+ 53us) server_negotiation.cc:299] Negotiation successful
0923 22:12:21.405222 (+ 56us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":160,"thread_start_us":93,"threads_started":1}
W20260923 22:12:21.405683 1724 server_base.cc:1116] Unauthorized access attempt to method kudu.tserver.TabletServerService.ListTablets from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:35544
Sep 23 22:12:21 dist-test-slave-s2fj krb5kdc[1574](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790201541, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
Sep 23 22:12:21 dist-test-slave-s2fj krb5kdc[1574](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790201541, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.1.135.193@KRBTEST.COM
I20260923 22:12:21.426061 2085 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 22:12:21.419625 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35560 (local address 127.1.135.193:36245)
0923 22:12:21.419707 (+ 82us) server_negotiation.cc:206] Beginning negotiation
0923 22:12:21.419709 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0923 22:12:21.419812 (+ 103us) server_negotiation.cc:407] Connection header received
0923 22:12:21.419839 (+ 27us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 22:12:21.419840 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 22:12:21.419871 (+ 31us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 22:12:21.419939 (+ 68us) server_negotiation.cc:226] Negotiated authn=SASL
0923 22:12:21.420522 (+ 583us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 22:12:21.421040 (+ 518us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 22:12:21.421705 (+ 665us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 22:12:21.421880 (+ 175us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 22:12:21.421926 (+ 46us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 22:12:21.423853 (+ 1927us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0923 22:12:21.423854 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0923 22:12:21.423854 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0923 22:12:21.423860 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0923 22:12:21.425225 (+ 1365us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0923 22:12:21.425514 (+ 289us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0923 22:12:21.425514 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0923 22:12:21.425515 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0923 22:12:21.425565 (+ 50us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0923 22:12:21.425706 (+ 141us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0923 22:12:21.425707 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0923 22:12:21.425707 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0923 22:12:21.425778 (+ 71us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0923 22:12:21.425858 (+ 80us) server_negotiation.cc:1035] Waiting for connection context
0923 22:12:21.425921 (+ 63us) server_negotiation.cc:299] Negotiation successful
0923 22:12:21.425974 (+ 53us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":62}
I20260923 22:12:21.426584 1567 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskRiQe_a/build/release/bin/kudu with pid 1681
I20260923 22:12:21.430969 1567 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskRiQe_a/build/release/bin/kudu with pid 1815
I20260923 22:12:21.435806 1567 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskRiQe_a/build/release/bin/kudu with pid 1949
I20260923 22:12:21.439939 1567 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskRiQe_a/build/release/bin/kudu with pid 1587
2026-09-23T22:12:21Z chronyd exiting
[ OK ] SecurityITest.TestAuthorizationOnListTablets (1933 ms)
[ RUN ] SecurityITest.TestAuthorizationOnChecksum
Loading random data
Initializing database '/tmp/dist-test-taskRiQe_a/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790201539513534-1567-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Sep 23 22:12:21 dist-test-slave-s2fj krb5kdc[2092](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Sep 23 22:12:21 dist-test-slave-s2fj krb5kdc[2092](info): set up 2 sockets
Sep 23 22:12:21 dist-test-slave-s2fj krb5kdc[2092](info): commencing operation
krb5kdc: starting...
W20260923 22:12:22.467582 1567 mini_kdc.cc:121] Time spent starting KDC: real 1.008s user 0.000s sys 0.004s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskRiQe_a/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790201539513534-1567-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskRiQe_a/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790201539513534-1567-0/krb5kdc/test-user.keytab.
Sep 23 22:12:22 dist-test-slave-s2fj krb5kdc[2092](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790201542, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-09-23T22:12:22Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-09-23T22:12:22Z Disabled control of system clock
WARNING: no policy specified for kudu/127.1.135.254@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.1.135.254@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.1.135.254 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskRiQe_a/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790201539513534-1567-0/krb5kdc/kudu_127.1.135.254.keytab.
Entry for principal kudu/127.1.135.254 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskRiQe_a/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790201539513534-1567-0/krb5kdc/kudu_127.1.135.254.keytab.
I20260923 22:12:22.575894 1567 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskRiQe_a/build/release/bin/kudu
/tmp/dist-test-taskRiQe_a/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskRiQe_a/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790201539513534-1567-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskRiQe_a/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790201539513534-1567-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskRiQe_a/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790201539513534-1567-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskRiQe_a/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790201539513534-1567-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.1.135.254:32983
--webserver_interface=127.1.135.254
--webserver_port=0
--keytab_file=/tmp/dist-test-taskRiQe_a/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790201539513534-1567-0/krb5kdc/kudu_127.1.135.254.keytab
--principal=kudu/127.1.135.254
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.1.135.212:33503
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.1.135.254:32983
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskRiQe_a/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790201539513534-1567-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskRiQe_a/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790201539513534-1567-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskRiQe_a/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790201539513534-1567-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260923 22:12:22.652681 2105 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260923 22:12:22.652874 2105 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260923 22:12:22.652895 2105 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260923 22:12:22.653895 2105 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260923 22:12:22.653927 2105 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260923 22:12:22.653947 2105 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260923 22:12:22.653959 2105 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260923 22:12:22.653971 2105 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260923 22:12:22.655318 2105 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.1.135.212:33503
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskRiQe_a/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790201539513534-1567-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskRiQe_a/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790201539513534-1567-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.1.135.254:32983
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskRiQe_a/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790201539513534-1567-0/krb5kdc/kudu_127.1.135.254.keytab
--principal=kudu/127.1.135.254
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.1.135.254:32983
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskRiQe_a/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790201539513534-1567-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.1.135.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.2105
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskRiQe_a/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790201539513534-1567-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 23 Sep 2026 20:53:37 UTC on ccycloud.quasar-rkviyx.root.comops.site
build id 4668083
I20260923 22:12:22.655499 2105 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260923 22:12:22.655649 2105 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260923 22:12:22.660264 2110 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260923 22:12:22.660272 2111 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260923 22:12:22.660277 2113 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260923 22:12:22.660296 2105 server_base.cc:1034] running on GCE node
I20260923 22:12:22.660692 2105 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260923 22:12:22.661005 2105 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260923 22:12:22.662173 2105 hybrid_clock.cc:648] HybridClock initialized: now 1790201542662148 us; error 35 us; skew 500 ppm
Sep 23 22:12:22 dist-test-slave-s2fj krb5kdc[2092](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790201542, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.135.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260923 22:12:22.664623 2105 init.cc:377] Logged in from keytab as kudu/127.1.135.254@KRBTEST.COM (short username kudu)
I20260923 22:12:22.665076 2105 webserver.cc:460] Webserver started at http://127.1.135.254:36163/ using document root <none> and password file <none>
I20260923 22:12:22.665307 2105 fs_manager.cc:359] Metadata directory not provided
I20260923 22:12:22.665375 2105 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260923 22:12:22.665524 2105 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260923 22:12:22.666486 2105 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskRiQe_a/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790201539513534-1567-0/minicluster-data/master-0/data/instance:
uuid: "2bbbfabf95284391b13371bc33ac8ef3"
format_stamp: "Formatted at 2026-09-23 22:12:22 on dist-test-slave-s2fj"
server_key: "d7716d587bc21aea5db3b008deb3c4d6"
server_key_iv: "6032bb37a39c78023e652f56be40f6e3"
server_key_version: "encryptionkey@0"
I20260923 22:12:22.666827 2105 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskRiQe_a/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790201539513534-1567-0/minicluster-data/master-0/wal/instance:
uuid: "2bbbfabf95284391b13371bc33ac8ef3"
format_stamp: "Formatted at 2026-09-23 22:12:22 on dist-test-slave-s2fj"
server_key: "d7716d587bc21aea5db3b008deb3c4d6"
server_key_iv: "6032bb37a39c78023e652f56be40f6e3"
server_key_version: "encryptionkey@0"
I20260923 22:12:22.668051 2105 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260923 22:12:22.668778 2120 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260923 22:12:22.668934 2105 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.001s
I20260923 22:12:22.669034 2105 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskRiQe_a/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790201539513534-1567-0/minicluster-data/master-0/data,/tmp/dist-test-taskRiQe_a/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790201539513534-1567-0/minicluster-data/master-0/wal
uuid: "2bbbfabf95284391b13371bc33ac8ef3"
format_stamp: "Formatted at 2026-09-23 22:12:22 on dist-test-slave-s2fj"
server_key: "d7716d587bc21aea5db3b008deb3c4d6"
server_key_iv: "6032bb37a39c78023e652f56be40f6e3"
server_key_version: "encryptionkey@0"
I20260923 22:12:22.669116 2105 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskRiQe_a/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790201539513534-1567-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskRiQe_a/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790201539513534-1567-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskRiQe_a/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790201539513534-1567-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260923 22:12:22.677079 2105 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260923 22:12:22.678716 2105 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260923 22:12:22.678865 2105 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260923 22:12:22.682562 2105 rpc_server.cc:307] RPC server started. Bound to: 127.1.135.254:32983
I20260923 22:12:22.682593 2172 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.1.135.254:32983 every 8 connection(s)
I20260923 22:12:22.682902 2105 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskRiQe_a/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790201539513534-1567-0/minicluster-data/master-0/data/info.pb
I20260923 22:12:22.683674 2173 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260923 22:12:22.685829 2173 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 2bbbfabf95284391b13371bc33ac8ef3: Bootstrap starting.
I20260923 22:12:22.686345 2173 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 2bbbfabf95284391b13371bc33ac8ef3: Neither blocks nor log segments found. Creating new log.
I20260923 22:12:22.686563 2173 log.cc:824] T 00000000000000000000000000000000 P 2bbbfabf95284391b13371bc33ac8ef3: Log is configured to *not* fsync() on all Append() calls
I20260923 22:12:22.687175 2173 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 2bbbfabf95284391b13371bc33ac8ef3: No bootstrap required, opened a new log
I20260923 22:12:22.688679 2173 raft_consensus.cc:348] T 00000000000000000000000000000000 P 2bbbfabf95284391b13371bc33ac8ef3 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2bbbfabf95284391b13371bc33ac8ef3" member_type: VOTER last_known_addr { host: "127.1.135.254" port: 32983 } }
I20260923 22:12:22.688818 2173 raft_consensus.cc:374] T 00000000000000000000000000000000 P 2bbbfabf95284391b13371bc33ac8ef3 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260923 22:12:22.688858 2173 raft_consensus.cc:729] T 00000000000000000000000000000000 P 2bbbfabf95284391b13371bc33ac8ef3 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 2bbbfabf95284391b13371bc33ac8ef3, State: Initialized, Role: FOLLOWER
I20260923 22:12:22.688951 2173 consensus_queue.cc:260] T 00000000000000000000000000000000 P 2bbbfabf95284391b13371bc33ac8ef3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2bbbfabf95284391b13371bc33ac8ef3" member_type: VOTER last_known_addr { host: "127.1.135.254" port: 32983 } }
I20260923 22:12:22.689006 2173 raft_consensus.cc:388] T 00000000000000000000000000000000 P 2bbbfabf95284391b13371bc33ac8ef3 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260923 22:12:22.689026 2173 raft_consensus.cc:482] T 00000000000000000000000000000000 P 2bbbfabf95284391b13371bc33ac8ef3 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260923 22:12:22.689051 2173 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 2bbbfabf95284391b13371bc33ac8ef3 [term 0 FOLLOWER]: Advancing to term 1
I20260923 22:12:22.689478 2173 raft_consensus.cc:504] T 00000000000000000000000000000000 P 2bbbfabf95284391b13371bc33ac8ef3 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2bbbfabf95284391b13371bc33ac8ef3" member_type: VOTER last_known_addr { host: "127.1.135.254" port: 32983 } }
I20260923 22:12:22.689555 2173 leader_election.cc:302] T 00000000000000000000000000000000 P 2bbbfabf95284391b13371bc33ac8ef3 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 2bbbfabf95284391b13371bc33ac8ef3; no voters:
I20260923 22:12:22.689664 2173 leader_election.cc:288] T 00000000000000000000000000000000 P 2bbbfabf95284391b13371bc33ac8ef3 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260923 22:12:22.689728 2176 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 2bbbfabf95284391b13371bc33ac8ef3 [term 1 FOLLOWER]: Leader election won for term 1
I20260923 22:12:22.689875 2173 sys_catalog.cc:564] T 00000000000000000000000000000000 P 2bbbfabf95284391b13371bc33ac8ef3 [sys.catalog]: configured and running, proceeding with master startup.
I20260923 22:12:22.689883 2176 raft_consensus.cc:686] T 00000000000000000000000000000000 P 2bbbfabf95284391b13371bc33ac8ef3 [term 1 LEADER]: Becoming Leader. State: Replica: 2bbbfabf95284391b13371bc33ac8ef3, State: Running, Role: LEADER
I20260923 22:12:22.689996 1567 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskRiQe_a/build/release/bin/kudu as pid 2105
I20260923 22:12:22.690026 2176 consensus_queue.cc:237] T 00000000000000000000000000000000 P 2bbbfabf95284391b13371bc33ac8ef3 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2bbbfabf95284391b13371bc33ac8ef3" member_type: VOTER last_known_addr { host: "127.1.135.254" port: 32983 } }
I20260923 22:12:22.690076 1567 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskRiQe_a/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790201539513534-1567-0/minicluster-data/master-0/wal/instance
I20260923 22:12:22.690263 1567 external_mini_cluster.cc:1428] Setting key fd5b477251e830c077999a22f499eefc
I20260923 22:12:22.690351 2177 sys_catalog.cc:455] T 00000000000000000000000000000000 P 2bbbfabf95284391b13371bc33ac8ef3 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "2bbbfabf95284391b13371bc33ac8ef3" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2bbbfabf95284391b13371bc33ac8ef3" member_type: VOTER last_known_addr { host: "127.1.135.254" port: 32983 } } }
I20260923 22:12:22.690461 2177 sys_catalog.cc:458] T 00000000000000000000000000000000 P 2bbbfabf95284391b13371bc33ac8ef3 [sys.catalog]: This master's current role is: LEADER
I20260923 22:12:22.690361 2178 sys_catalog.cc:455] T 00000000000000000000000000000000 P 2bbbfabf95284391b13371bc33ac8ef3 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 2bbbfabf95284391b13371bc33ac8ef3. Latest consensus state: current_term: 1 leader_uuid: "2bbbfabf95284391b13371bc33ac8ef3" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2bbbfabf95284391b13371bc33ac8ef3" member_type: VOTER last_known_addr { host: "127.1.135.254" port: 32983 } } }
I20260923 22:12:22.690689 2178 sys_catalog.cc:458] T 00000000000000000000000000000000 P 2bbbfabf95284391b13371bc33ac8ef3 [sys.catalog]: This master's current role is: LEADER
I20260923 22:12:22.691391 2180 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260923 22:12:22.691876 2180 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260923 22:12:22.693380 2180 catalog_manager.cc:1347] Generated new cluster ID: d4461207dbcb4a099f6610025e765346
I20260923 22:12:22.693437 2180 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
Sep 23 22:12:22 dist-test-slave-s2fj krb5kdc[2092](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790201542, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.1.135.254@KRBTEST.COM
I20260923 22:12:22.702152 2187 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 22:12:22.690890 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52770 (local address 127.1.135.254:32983)
0923 22:12:22.691041 (+ 151us) server_negotiation.cc:206] Beginning negotiation
0923 22:12:22.691051 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0923 22:12:22.691141 (+ 90us) server_negotiation.cc:407] Connection header received
0923 22:12:22.691287 (+ 146us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 22:12:22.691293 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 22:12:22.691356 (+ 63us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 22:12:22.691488 (+ 132us) server_negotiation.cc:226] Negotiated authn=SASL
0923 22:12:22.692378 (+ 890us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 22:12:22.693587 (+ 1209us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 22:12:22.694628 (+ 1041us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 22:12:22.695212 (+ 584us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 22:12:22.695302 (+ 90us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 22:12:22.698112 (+ 2810us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0923 22:12:22.698115 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0923 22:12:22.698116 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0923 22:12:22.698125 (+ 9us) server_negotiation.cc:922] calling sasl_server_start()
0923 22:12:22.700606 (+ 2481us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0923 22:12:22.700984 (+ 378us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0923 22:12:22.700987 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0923 22:12:22.700987 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0923 22:12:22.701104 (+ 117us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0923 22:12:22.701314 (+ 210us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0923 22:12:22.701314 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0923 22:12:22.701315 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0923 22:12:22.701577 (+ 262us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0923 22:12:22.701726 (+ 149us) server_negotiation.cc:1035] Waiting for connection context
0923 22:12:22.701848 (+ 122us) server_negotiation.cc:299] Negotiation successful
0923 22:12:22.701969 (+ 121us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":102,"thread_start_us":84,"threads_started":1}
I20260923 22:12:22.724687 2180 catalog_manager.cc:1370] Generated new certificate authority record
I20260923 22:12:22.725158 2180 catalog_manager.cc:1504] Loading token signing keys...
I20260923 22:12:22.729866 2180 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 2bbbfabf95284391b13371bc33ac8ef3: Generated new TSK 0
I20260923 22:12:22.730029 2180 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.1.135.193@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.1.135.193@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.1.135.193 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskRiQe_a/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790201539513534-1567-0/krb5kdc/kudu_127.1.135.193.keytab.
Entry for principal kudu/127.1.135.193 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskRiQe_a/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790201539513534-1567-0/krb5kdc/kudu_127.1.135.193.keytab.
I20260923 22:12:22.762709 1567 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskRiQe_a/build/release/bin/kudu
/tmp/dist-test-taskRiQe_a/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskRiQe_a/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790201539513534-1567-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskRiQe_a/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790201539513534-1567-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskRiQe_a/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790201539513534-1567-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskRiQe_a/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790201539513534-1567-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.1.135.193:0
--local_ip_for_outbound_sockets=127.1.135.193
--webserver_interface=127.1.135.193
--webserver_port=0
--tserver_master_addrs=127.1.135.254:32983
--keytab_file=/tmp/dist-test-taskRiQe_a/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790201539513534-1567-0/krb5kdc/kudu_127.1.135.193.keytab
--principal=kudu/127.1.135.193
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.1.135.212:33503
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--tserver_enforce_access_control with env {KRB5CCNAME=/tmp/dist-test-taskRiQe_a/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790201539513534-1567-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskRiQe_a/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790201539513534-1567-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskRiQe_a/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790201539513534-1567-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260923 22:12:22.858846 2199 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260923 22:12:22.859017 2199 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260923 22:12:22.859040 2199 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260923 22:12:22.860198 2199 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260923 22:12:22.860234 2199 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260923 22:12:22.860261 2199 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.1.135.193
I20260923 22:12:22.861778 2199 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.1.135.212:33503
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskRiQe_a/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790201539513534-1567-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskRiQe_a/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790201539513534-1567-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskRiQe_a/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790201539513534-1567-0/krb5kdc/kudu_127.1.135.193.keytab
--principal=kudu/127.1.135.193
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.1.135.193:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskRiQe_a/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790201539513534-1567-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.1.135.193
--webserver_port=0
--tserver_master_addrs=127.1.135.254:32983
--tserver_enforce_access_control=true
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.2199
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.1.135.193
--log_dir=/tmp/dist-test-taskRiQe_a/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790201539513534-1567-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 23 Sep 2026 20:53:37 UTC on ccycloud.quasar-rkviyx.root.comops.site
build id 4668083
I20260923 22:12:22.862005 2199 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260923 22:12:22.862231 2199 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260923 22:12:22.866948 2207 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260923 22:12:22.866961 2204 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260923 22:12:22.866964 2205 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260923 22:12:22.867185 2199 server_base.cc:1034] running on GCE node
I20260923 22:12:22.867403 2199 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260923 22:12:22.867636 2199 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260923 22:12:22.868800 2199 hybrid_clock.cc:648] HybridClock initialized: now 1790201542868794 us; error 32 us; skew 500 ppm
Sep 23 22:12:22 dist-test-slave-s2fj krb5kdc[2092](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790201542, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.135.193@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260923 22:12:22.871166 2199 init.cc:377] Logged in from keytab as kudu/127.1.135.193@KRBTEST.COM (short username kudu)
I20260923 22:12:22.871690 2199 webserver.cc:460] Webserver started at http://127.1.135.193:38017/ using document root <none> and password file <none>
I20260923 22:12:22.871959 2199 fs_manager.cc:359] Metadata directory not provided
I20260923 22:12:22.872027 2199 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260923 22:12:22.872172 2199 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260923 22:12:22.873270 2199 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskRiQe_a/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790201539513534-1567-0/minicluster-data/ts-0/data/instance:
uuid: "0983183088f24c5bac2c87918556364c"
format_stamp: "Formatted at 2026-09-23 22:12:22 on dist-test-slave-s2fj"
server_key: "6f9e53862e959966dd014b8c494d6cf6"
server_key_iv: "96902617b856f959aec27785357bd036"
server_key_version: "encryptionkey@0"
I20260923 22:12:22.873626 2199 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskRiQe_a/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790201539513534-1567-0/minicluster-data/ts-0/wal/instance:
uuid: "0983183088f24c5bac2c87918556364c"
format_stamp: "Formatted at 2026-09-23 22:12:22 on dist-test-slave-s2fj"
server_key: "6f9e53862e959966dd014b8c494d6cf6"
server_key_iv: "96902617b856f959aec27785357bd036"
server_key_version: "encryptionkey@0"
I20260923 22:12:22.874884 2199 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.003s
I20260923 22:12:22.875545 2214 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260923 22:12:22.875716 2199 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.001s
I20260923 22:12:22.875801 2199 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskRiQe_a/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790201539513534-1567-0/minicluster-data/ts-0/data,/tmp/dist-test-taskRiQe_a/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790201539513534-1567-0/minicluster-data/ts-0/wal
uuid: "0983183088f24c5bac2c87918556364c"
format_stamp: "Formatted at 2026-09-23 22:12:22 on dist-test-slave-s2fj"
server_key: "6f9e53862e959966dd014b8c494d6cf6"
server_key_iv: "96902617b856f959aec27785357bd036"
server_key_version: "encryptionkey@0"
I20260923 22:12:22.875883 2199 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskRiQe_a/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790201539513534-1567-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskRiQe_a/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790201539513534-1567-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskRiQe_a/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790201539513534-1567-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260923 22:12:22.896788 2199 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260923 22:12:22.898453 2199 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260923 22:12:22.898615 2199 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260923 22:12:22.898838 2199 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260923 22:12:22.899143 2199 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260923 22:12:22.899196 2199 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260923 22:12:22.899227 2199 ts_tablet_manager.cc:610] Registered 0 tablets
I20260923 22:12:22.899256 2199 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260923 22:12:22.904649 2199 rpc_server.cc:307] RPC server started. Bound to: 127.1.135.193:44397
I20260923 22:12:22.904717 2327 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.1.135.193:44397 every 8 connection(s)
I20260923 22:12:22.905031 2199 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskRiQe_a/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790201539513534-1567-0/minicluster-data/ts-0/data/info.pb
I20260923 22:12:22.907644 1567 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskRiQe_a/build/release/bin/kudu as pid 2199
I20260923 22:12:22.907733 1567 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskRiQe_a/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790201539513534-1567-0/minicluster-data/ts-0/wal/instance
I20260923 22:12:22.907897 1567 external_mini_cluster.cc:1428] Setting key 45b479ac04bfb34cf72b61a6636746dc
Sep 23 22:12:22 dist-test-slave-s2fj krb5kdc[2092](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790201542, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.135.193@KRBTEST.COM for kudu/127.1.135.254@KRBTEST.COM
I20260923 22:12:22.914376 2187 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 22:12:22.905705 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.135.193:37545 (local address 127.1.135.254:32983)
0923 22:12:22.905762 (+ 57us) server_negotiation.cc:206] Beginning negotiation
0923 22:12:22.905765 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0923 22:12:22.905959 (+ 194us) server_negotiation.cc:407] Connection header received
0923 22:12:22.906864 (+ 905us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 22:12:22.906865 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 22:12:22.906873 (+ 8us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 22:12:22.906927 (+ 54us) server_negotiation.cc:226] Negotiated authn=SASL
0923 22:12:22.907462 (+ 535us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 22:12:22.908275 (+ 813us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 22:12:22.909288 (+ 1013us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 22:12:22.909502 (+ 214us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 22:12:22.909575 (+ 73us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 22:12:22.911602 (+ 2027us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0923 22:12:22.911604 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0923 22:12:22.911604 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0923 22:12:22.911612 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0923 22:12:22.913029 (+ 1417us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0923 22:12:22.913440 (+ 411us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0923 22:12:22.913442 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0923 22:12:22.913442 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0923 22:12:22.913500 (+ 58us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0923 22:12:22.913740 (+ 240us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0923 22:12:22.913741 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0923 22:12:22.913741 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0923 22:12:22.913848 (+ 107us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0923 22:12:22.913941 (+ 93us) server_negotiation.cc:1035] Waiting for connection context
0923 22:12:22.914205 (+ 264us) server_negotiation.cc:299] Negotiation successful
0923 22:12:22.914290 (+ 85us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":34}
I20260923 22:12:22.914566 2330 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 22:12:22.905684 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.135.254:32983 (local address 127.1.135.193:37545)
0923 22:12:22.905848 (+ 164us) negotiation.cc:107] Waiting for socket to connect
0923 22:12:22.905887 (+ 39us) client_negotiation.cc:174] Beginning negotiation
0923 22:12:22.906781 (+ 894us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0923 22:12:22.906983 (+ 202us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0923 22:12:22.906986 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0923 22:12:22.907214 (+ 228us) client_negotiation.cc:189] Negotiated authn=SASL
0923 22:12:22.907387 (+ 173us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 22:12:22.907391 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 22:12:22.908393 (+ 1002us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 22:12:22.908394 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 22:12:22.909197 (+ 803us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 22:12:22.909199 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 22:12:22.909592 (+ 393us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 22:12:22.909593 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 22:12:22.909710 (+ 117us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 22:12:22.910212 (+ 502us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0923 22:12:22.910221 (+ 9us) client_negotiation.cc:656] calling sasl_client_start()
0923 22:12:22.911507 (+ 1286us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0923 22:12:22.913128 (+ 1621us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0923 22:12:22.913135 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0923 22:12:22.913137 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0923 22:12:22.913356 (+ 219us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0923 22:12:22.913564 (+ 208us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0923 22:12:22.913564 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0923 22:12:22.913565 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0923 22:12:22.913671 (+ 106us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0923 22:12:22.913944 (+ 273us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0923 22:12:22.913950 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0923 22:12:22.914077 (+ 127us) client_negotiation.cc:769] Sending connection context
0923 22:12:22.914185 (+ 108us) client_negotiation.cc:240] Negotiation successful
0923 22:12:22.914330 (+ 145us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":123,"thread_start_us":89,"threads_started":1}
WARNING: no policy specified for kudu/127.1.135.194@KRBTEST.COM; defaulting to no policy
I20260923 22:12:22.915158 2328 heartbeater.cc:347] Connected to a master server at 127.1.135.254:32983
I20260923 22:12:22.915251 2328 heartbeater.cc:464] Registering TS with master...
I20260923 22:12:22.915416 2328 heartbeater.cc:511] Master 127.1.135.254:32983 requested a full tablet report, sending...
I20260923 22:12:22.915864 2137 ts_manager.cc:194] Registered new tserver with Master: 0983183088f24c5bac2c87918556364c (127.1.135.193:44397)
I20260923 22:12:22.916513 2137 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.1.135.193@KRBTEST.COM'} at 127.1.135.193:37545
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.1.135.194@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.1.135.194 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskRiQe_a/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790201539513534-1567-0/krb5kdc/kudu_127.1.135.194.keytab.
Entry for principal kudu/127.1.135.194 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskRiQe_a/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790201539513534-1567-0/krb5kdc/kudu_127.1.135.194.keytab.
I20260923 22:12:22.930866 1567 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskRiQe_a/build/release/bin/kudu
/tmp/dist-test-taskRiQe_a/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskRiQe_a/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790201539513534-1567-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskRiQe_a/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790201539513534-1567-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskRiQe_a/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790201539513534-1567-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskRiQe_a/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790201539513534-1567-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.1.135.194:0
--local_ip_for_outbound_sockets=127.1.135.194
--webserver_interface=127.1.135.194
--webserver_port=0
--tserver_master_addrs=127.1.135.254:32983
--keytab_file=/tmp/dist-test-taskRiQe_a/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790201539513534-1567-0/krb5kdc/kudu_127.1.135.194.keytab
--principal=kudu/127.1.135.194
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.1.135.212:33503
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--tserver_enforce_access_control with env {KRB5CCNAME=/tmp/dist-test-taskRiQe_a/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790201539513534-1567-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskRiQe_a/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790201539513534-1567-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskRiQe_a/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790201539513534-1567-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260923 22:12:23.009910 2333 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260923 22:12:23.010089 2333 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260923 22:12:23.010111 2333 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260923 22:12:23.011200 2333 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260923 22:12:23.011237 2333 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260923 22:12:23.011263 2333 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.1.135.194
I20260923 22:12:23.012657 2333 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.1.135.212:33503
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskRiQe_a/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790201539513534-1567-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskRiQe_a/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790201539513534-1567-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskRiQe_a/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790201539513534-1567-0/krb5kdc/kudu_127.1.135.194.keytab
--principal=kudu/127.1.135.194
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.1.135.194:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskRiQe_a/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790201539513534-1567-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.1.135.194
--webserver_port=0
--tserver_master_addrs=127.1.135.254:32983
--tserver_enforce_access_control=true
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.2333
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.1.135.194
--log_dir=/tmp/dist-test-taskRiQe_a/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790201539513534-1567-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 23 Sep 2026 20:53:37 UTC on ccycloud.quasar-rkviyx.root.comops.site
build id 4668083
I20260923 22:12:23.012823 2333 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260923 22:12:23.012982 2333 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260923 22:12:23.017755 2341 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260923 22:12:23.017884 2333 server_base.cc:1034] running on GCE node
W20260923 22:12:23.017784 2338 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260923 22:12:23.017757 2339 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260923 22:12:23.018172 2333 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260923 22:12:23.018384 2333 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260923 22:12:23.019547 2333 hybrid_clock.cc:648] HybridClock initialized: now 1790201543019527 us; error 37 us; skew 500 ppm
Sep 23 22:12:23 dist-test-slave-s2fj krb5kdc[2092](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790201543, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.135.194@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260923 22:12:23.022153 2333 init.cc:377] Logged in from keytab as kudu/127.1.135.194@KRBTEST.COM (short username kudu)
I20260923 22:12:23.022531 2333 webserver.cc:460] Webserver started at http://127.1.135.194:37931/ using document root <none> and password file <none>
I20260923 22:12:23.022697 2333 fs_manager.cc:359] Metadata directory not provided
I20260923 22:12:23.022737 2333 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260923 22:12:23.022819 2333 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260923 22:12:23.023729 2333 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskRiQe_a/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790201539513534-1567-0/minicluster-data/ts-1/data/instance:
uuid: "a578afa357444ef7bec6d019e2634cd8"
format_stamp: "Formatted at 2026-09-23 22:12:23 on dist-test-slave-s2fj"
server_key: "9ad42545fce81243307f09774c1d4415"
server_key_iv: "39f306a8c4259b4d069ea6f12b4dc2a7"
server_key_version: "encryptionkey@0"
I20260923 22:12:23.024017 2333 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskRiQe_a/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790201539513534-1567-0/minicluster-data/ts-1/wal/instance:
uuid: "a578afa357444ef7bec6d019e2634cd8"
format_stamp: "Formatted at 2026-09-23 22:12:23 on dist-test-slave-s2fj"
server_key: "9ad42545fce81243307f09774c1d4415"
server_key_iv: "39f306a8c4259b4d069ea6f12b4dc2a7"
server_key_version: "encryptionkey@0"
I20260923 22:12:23.025326 2333 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.001s sys 0.000s
I20260923 22:12:23.025954 2348 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260923 22:12:23.026140 2333 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260923 22:12:23.026208 2333 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskRiQe_a/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790201539513534-1567-0/minicluster-data/ts-1/data,/tmp/dist-test-taskRiQe_a/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790201539513534-1567-0/minicluster-data/ts-1/wal
uuid: "a578afa357444ef7bec6d019e2634cd8"
format_stamp: "Formatted at 2026-09-23 22:12:23 on dist-test-slave-s2fj"
server_key: "9ad42545fce81243307f09774c1d4415"
server_key_iv: "39f306a8c4259b4d069ea6f12b4dc2a7"
server_key_version: "encryptionkey@0"
I20260923 22:12:23.026296 2333 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskRiQe_a/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790201539513534-1567-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskRiQe_a/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790201539513534-1567-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskRiQe_a/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790201539513534-1567-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260923 22:12:23.048579 2333 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260923 22:12:23.050742 2333 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260923 22:12:23.050836 2333 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260923 22:12:23.051003 2333 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260923 22:12:23.051288 2333 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260923 22:12:23.051318 2333 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260923 22:12:23.051338 2333 ts_tablet_manager.cc:610] Registered 0 tablets
I20260923 22:12:23.051380 2333 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260923 22:12:23.056891 2333 rpc_server.cc:307] RPC server started. Bound to: 127.1.135.194:43183
I20260923 22:12:23.056936 2461 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.1.135.194:43183 every 8 connection(s)
I20260923 22:12:23.057212 2333 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskRiQe_a/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790201539513534-1567-0/minicluster-data/ts-1/data/info.pb
Sep 23 22:12:23 dist-test-slave-s2fj krb5kdc[2092](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790201543, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.135.194@KRBTEST.COM for kudu/127.1.135.254@KRBTEST.COM
I20260923 22:12:23.065011 1567 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskRiQe_a/build/release/bin/kudu as pid 2333
I20260923 22:12:23.065102 1567 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskRiQe_a/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790201539513534-1567-0/minicluster-data/ts-1/wal/instance
I20260923 22:12:23.065266 1567 external_mini_cluster.cc:1428] Setting key b0fe0f6fd6c238691a55235d66376e3f
I20260923 22:12:23.065953 2187 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 22:12:23.057899 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.135.194:50039 (local address 127.1.135.254:32983)
0923 22:12:23.057979 (+ 80us) server_negotiation.cc:206] Beginning negotiation
0923 22:12:23.057981 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0923 22:12:23.058104 (+ 123us) server_negotiation.cc:407] Connection header received
0923 22:12:23.058936 (+ 832us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 22:12:23.058938 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 22:12:23.058947 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 22:12:23.059001 (+ 54us) server_negotiation.cc:226] Negotiated authn=SASL
0923 22:12:23.059539 (+ 538us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 22:12:23.060031 (+ 492us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 22:12:23.061008 (+ 977us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 22:12:23.061186 (+ 178us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 22:12:23.061235 (+ 49us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 22:12:23.063100 (+ 1865us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0923 22:12:23.063101 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0923 22:12:23.063101 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0923 22:12:23.063108 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0923 22:12:23.064540 (+ 1432us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0923 22:12:23.064893 (+ 353us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0923 22:12:23.064895 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0923 22:12:23.064895 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0923 22:12:23.064955 (+ 60us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0923 22:12:23.065255 (+ 300us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0923 22:12:23.065256 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0923 22:12:23.065257 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0923 22:12:23.065385 (+ 128us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0923 22:12:23.065462 (+ 77us) server_negotiation.cc:1035] Waiting for connection context
0923 22:12:23.065744 (+ 282us) server_negotiation.cc:299] Negotiation successful
0923 22:12:23.065838 (+ 94us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":42}
I20260923 22:12:23.066080 2464 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 22:12:23.057854 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.135.254:32983 (local address 127.1.135.194:50039)
0923 22:12:23.057997 (+ 143us) negotiation.cc:107] Waiting for socket to connect
0923 22:12:23.058028 (+ 31us) client_negotiation.cc:174] Beginning negotiation
0923 22:12:23.058846 (+ 818us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0923 22:12:23.059051 (+ 205us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0923 22:12:23.059054 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0923 22:12:23.059267 (+ 213us) client_negotiation.cc:189] Negotiated authn=SASL
0923 22:12:23.059468 (+ 201us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 22:12:23.059471 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 22:12:23.060145 (+ 674us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 22:12:23.060145 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 22:12:23.060923 (+ 778us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 22:12:23.060925 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 22:12:23.061255 (+ 330us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 22:12:23.061256 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 22:12:23.061347 (+ 91us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 22:12:23.061730 (+ 383us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0923 22:12:23.061738 (+ 8us) client_negotiation.cc:656] calling sasl_client_start()
0923 22:12:23.063016 (+ 1278us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0923 22:12:23.064606 (+ 1590us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0923 22:12:23.064610 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0923 22:12:23.064612 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0923 22:12:23.064806 (+ 194us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0923 22:12:23.065055 (+ 249us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0923 22:12:23.065055 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0923 22:12:23.065055 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0923 22:12:23.065205 (+ 150us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0923 22:12:23.065468 (+ 263us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0923 22:12:23.065473 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0923 22:12:23.065625 (+ 152us) client_negotiation.cc:769] Sending connection context
0923 22:12:23.065706 (+ 81us) client_negotiation.cc:240] Negotiation successful
0923 22:12:23.065854 (+ 148us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":112,"thread_start_us":83,"threads_started":1}
I20260923 22:12:23.066493 2462 heartbeater.cc:347] Connected to a master server at 127.1.135.254:32983
I20260923 22:12:23.066587 2462 heartbeater.cc:464] Registering TS with master...
I20260923 22:12:23.066772 2462 heartbeater.cc:511] Master 127.1.135.254:32983 requested a full tablet report, sending...
I20260923 22:12:23.067137 2137 ts_manager.cc:194] Registered new tserver with Master: a578afa357444ef7bec6d019e2634cd8 (127.1.135.194:43183)
I20260923 22:12:23.067633 2137 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.1.135.194@KRBTEST.COM'} at 127.1.135.194:50039
WARNING: no policy specified for kudu/127.1.135.195@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.1.135.195@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.1.135.195 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskRiQe_a/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790201539513534-1567-0/krb5kdc/kudu_127.1.135.195.keytab.
Entry for principal kudu/127.1.135.195 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskRiQe_a/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790201539513534-1567-0/krb5kdc/kudu_127.1.135.195.keytab.
I20260923 22:12:23.088704 1567 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskRiQe_a/build/release/bin/kudu
/tmp/dist-test-taskRiQe_a/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskRiQe_a/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790201539513534-1567-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskRiQe_a/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790201539513534-1567-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskRiQe_a/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790201539513534-1567-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskRiQe_a/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790201539513534-1567-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.1.135.195:0
--local_ip_for_outbound_sockets=127.1.135.195
--webserver_interface=127.1.135.195
--webserver_port=0
--tserver_master_addrs=127.1.135.254:32983
--keytab_file=/tmp/dist-test-taskRiQe_a/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790201539513534-1567-0/krb5kdc/kudu_127.1.135.195.keytab
--principal=kudu/127.1.135.195
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.1.135.212:33503
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--tserver_enforce_access_control with env {KRB5CCNAME=/tmp/dist-test-taskRiQe_a/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790201539513534-1567-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskRiQe_a/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790201539513534-1567-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskRiQe_a/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790201539513534-1567-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260923 22:12:23.166185 2467 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260923 22:12:23.166360 2467 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260923 22:12:23.166384 2467 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260923 22:12:23.167448 2467 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260923 22:12:23.167482 2467 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260923 22:12:23.167507 2467 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.1.135.195
I20260923 22:12:23.168888 2467 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.1.135.212:33503
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskRiQe_a/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790201539513534-1567-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskRiQe_a/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790201539513534-1567-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskRiQe_a/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790201539513534-1567-0/krb5kdc/kudu_127.1.135.195.keytab
--principal=kudu/127.1.135.195
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.1.135.195:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskRiQe_a/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790201539513534-1567-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.1.135.195
--webserver_port=0
--tserver_master_addrs=127.1.135.254:32983
--tserver_enforce_access_control=true
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.2467
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.1.135.195
--log_dir=/tmp/dist-test-taskRiQe_a/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790201539513534-1567-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 23 Sep 2026 20:53:37 UTC on ccycloud.quasar-rkviyx.root.comops.site
build id 4668083
I20260923 22:12:23.169056 2467 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260923 22:12:23.169210 2467 file_cache.cc:493] Constructed file cache file cache with capacity 419430
I20260923 22:12:23.173868 2467 server_base.cc:1034] running on GCE node
W20260923 22:12:23.173892 2475 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260923 22:12:23.173980 2473 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260923 22:12:23.174142 2472 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260923 22:12:23.174371 2467 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260923 22:12:23.174608 2467 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260923 22:12:23.175763 2467 hybrid_clock.cc:648] HybridClock initialized: now 1790201543175759 us; error 35 us; skew 500 ppm
Sep 23 22:12:23 dist-test-slave-s2fj krb5kdc[2092](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790201543, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.135.195@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260923 22:12:23.179358 2467 init.cc:377] Logged in from keytab as kudu/127.1.135.195@KRBTEST.COM (short username kudu)
I20260923 22:12:23.179859 2467 webserver.cc:460] Webserver started at http://127.1.135.195:35655/ using document root <none> and password file <none>
I20260923 22:12:23.180131 2467 fs_manager.cc:359] Metadata directory not provided
I20260923 22:12:23.180220 2467 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260923 22:12:23.180359 2467 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260923 22:12:23.181536 2467 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskRiQe_a/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790201539513534-1567-0/minicluster-data/ts-2/data/instance:
uuid: "f53d83ad2d734dddb2a49178ea23d7d9"
format_stamp: "Formatted at 2026-09-23 22:12:23 on dist-test-slave-s2fj"
server_key: "5c49d334b1cc822b38b605b43efc6c42"
server_key_iv: "2bc70e3be429dcc79e9fad5ebce42251"
server_key_version: "encryptionkey@0"
I20260923 22:12:23.181916 2467 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskRiQe_a/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790201539513534-1567-0/minicluster-data/ts-2/wal/instance:
uuid: "f53d83ad2d734dddb2a49178ea23d7d9"
format_stamp: "Formatted at 2026-09-23 22:12:23 on dist-test-slave-s2fj"
server_key: "5c49d334b1cc822b38b605b43efc6c42"
server_key_iv: "2bc70e3be429dcc79e9fad5ebce42251"
server_key_version: "encryptionkey@0"
I20260923 22:12:23.183691 2467 fs_manager.cc:691] Time spent creating directory manager: real 0.002s user 0.003s sys 0.000s
I20260923 22:12:23.184589 2482 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260923 22:12:23.184762 2467 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.000s
I20260923 22:12:23.184866 2467 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskRiQe_a/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790201539513534-1567-0/minicluster-data/ts-2/data,/tmp/dist-test-taskRiQe_a/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790201539513534-1567-0/minicluster-data/ts-2/wal
uuid: "f53d83ad2d734dddb2a49178ea23d7d9"
format_stamp: "Formatted at 2026-09-23 22:12:23 on dist-test-slave-s2fj"
server_key: "5c49d334b1cc822b38b605b43efc6c42"
server_key_iv: "2bc70e3be429dcc79e9fad5ebce42251"
server_key_version: "encryptionkey@0"
I20260923 22:12:23.184937 2467 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskRiQe_a/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790201539513534-1567-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskRiQe_a/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790201539513534-1567-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskRiQe_a/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790201539513534-1567-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260923 22:12:23.196538 2467 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260923 22:12:23.198181 2467 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260923 22:12:23.198326 2467 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260923 22:12:23.198554 2467 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260923 22:12:23.198830 2467 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260923 22:12:23.198881 2467 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260923 22:12:23.198920 2467 ts_tablet_manager.cc:610] Registered 0 tablets
I20260923 22:12:23.198946 2467 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260923 22:12:23.204561 2467 rpc_server.cc:307] RPC server started. Bound to: 127.1.135.195:36101
I20260923 22:12:23.204604 2595 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.1.135.195:36101 every 8 connection(s)
I20260923 22:12:23.204878 2467 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskRiQe_a/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790201539513534-1567-0/minicluster-data/ts-2/data/info.pb
Sep 23 22:12:23 dist-test-slave-s2fj krb5kdc[2092](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790201543, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.135.195@KRBTEST.COM for kudu/127.1.135.254@KRBTEST.COM
I20260923 22:12:23.212788 1567 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskRiQe_a/build/release/bin/kudu as pid 2467
I20260923 22:12:23.212885 1567 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskRiQe_a/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790201539513534-1567-0/minicluster-data/ts-2/wal/instance
I20260923 22:12:23.213052 1567 external_mini_cluster.cc:1428] Setting key 7663f91e9be6a801129c2f9e14d64668
I20260923 22:12:23.214809 2187 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 22:12:23.205593 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.135.195:57667 (local address 127.1.135.254:32983)
0923 22:12:23.205641 (+ 48us) server_negotiation.cc:206] Beginning negotiation
0923 22:12:23.205660 (+ 19us) server_negotiation.cc:399] Waiting for connection header
0923 22:12:23.205911 (+ 251us) server_negotiation.cc:407] Connection header received
0923 22:12:23.207201 (+ 1290us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 22:12:23.207202 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 22:12:23.207216 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 22:12:23.207294 (+ 78us) server_negotiation.cc:226] Negotiated authn=SASL
0923 22:12:23.208251 (+ 957us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 22:12:23.208911 (+ 660us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 22:12:23.209897 (+ 986us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 22:12:23.210103 (+ 206us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 22:12:23.210185 (+ 82us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 22:12:23.212028 (+ 1843us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0923 22:12:23.212029 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0923 22:12:23.212030 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0923 22:12:23.212037 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0923 22:12:23.213602 (+ 1565us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0923 22:12:23.214061 (+ 459us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0923 22:12:23.214062 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0923 22:12:23.214062 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0923 22:12:23.214115 (+ 53us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0923 22:12:23.214356 (+ 241us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0923 22:12:23.214357 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0923 22:12:23.214357 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0923 22:12:23.214458 (+ 101us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0923 22:12:23.214520 (+ 62us) server_negotiation.cc:1035] Waiting for connection context
0923 22:12:23.214695 (+ 175us) server_negotiation.cc:299] Negotiation successful
0923 22:12:23.214752 (+ 57us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":20}
I20260923 22:12:23.215047 2598 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 22:12:23.205595 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.135.254:32983 (local address 127.1.135.195:57667)
0923 22:12:23.205777 (+ 182us) negotiation.cc:107] Waiting for socket to connect
0923 22:12:23.205823 (+ 46us) client_negotiation.cc:174] Beginning negotiation
0923 22:12:23.207083 (+ 1260us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0923 22:12:23.207395 (+ 312us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0923 22:12:23.207399 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0923 22:12:23.207817 (+ 418us) client_negotiation.cc:189] Negotiated authn=SASL
0923 22:12:23.208159 (+ 342us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 22:12:23.208165 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 22:12:23.209021 (+ 856us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 22:12:23.209022 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 22:12:23.209793 (+ 771us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 22:12:23.209795 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 22:12:23.210193 (+ 398us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 22:12:23.210193 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 22:12:23.210321 (+ 128us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 22:12:23.210701 (+ 380us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0923 22:12:23.210708 (+ 7us) client_negotiation.cc:656] calling sasl_client_start()
0923 22:12:23.211939 (+ 1231us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0923 22:12:23.213703 (+ 1764us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0923 22:12:23.213708 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0923 22:12:23.213710 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0923 22:12:23.213997 (+ 287us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0923 22:12:23.214203 (+ 206us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0923 22:12:23.214203 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0923 22:12:23.214203 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0923 22:12:23.214289 (+ 86us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0923 22:12:23.214511 (+ 222us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0923 22:12:23.214513 (+ 2us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0923 22:12:23.214609 (+ 96us) client_negotiation.cc:769] Sending connection context
0923 22:12:23.214680 (+ 71us) client_negotiation.cc:240] Negotiation successful
0923 22:12:23.214784 (+ 104us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":113,"thread_start_us":83,"threads_started":1}
I20260923 22:12:23.215452 2596 heartbeater.cc:347] Connected to a master server at 127.1.135.254:32983
I20260923 22:12:23.215538 2596 heartbeater.cc:464] Registering TS with master...
I20260923 22:12:23.215657 2596 heartbeater.cc:511] Master 127.1.135.254:32983 requested a full tablet report, sending...
I20260923 22:12:23.215950 2137 ts_manager.cc:194] Registered new tserver with Master: f53d83ad2d734dddb2a49178ea23d7d9 (127.1.135.195:36101)
I20260923 22:12:23.216336 2137 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.1.135.195@KRBTEST.COM'} at 127.1.135.195:57667
I20260923 22:12:23.224638 1567 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260923 22:12:23.231717 2187 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 22:12:23.226107 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52782 (local address 127.1.135.254:32983)
0923 22:12:23.226181 (+ 74us) server_negotiation.cc:206] Beginning negotiation
0923 22:12:23.226183 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0923 22:12:23.226271 (+ 88us) server_negotiation.cc:407] Connection header received
0923 22:12:23.226344 (+ 73us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 22:12:23.226345 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 22:12:23.226355 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 22:12:23.226409 (+ 54us) server_negotiation.cc:226] Negotiated authn=SASL
0923 22:12:23.226918 (+ 509us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 22:12:23.227370 (+ 452us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 22:12:23.228110 (+ 740us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 22:12:23.228317 (+ 207us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 22:12:23.228399 (+ 82us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 22:12:23.229538 (+ 1139us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0923 22:12:23.229540 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0923 22:12:23.229540 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0923 22:12:23.229547 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0923 22:12:23.230845 (+ 1298us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0923 22:12:23.231137 (+ 292us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0923 22:12:23.231138 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0923 22:12:23.231138 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0923 22:12:23.231182 (+ 44us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0923 22:12:23.231343 (+ 161us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0923 22:12:23.231343 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0923 22:12:23.231343 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0923 22:12:23.231462 (+ 119us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0923 22:12:23.231532 (+ 70us) server_negotiation.cc:1035] Waiting for connection context
0923 22:12:23.231613 (+ 81us) server_negotiation.cc:299] Negotiation successful
0923 22:12:23.231664 (+ 51us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":49}
I20260923 22:12:23.233565 2137 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:52782:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260923 22:12:23.235037 2137 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260923 22:12:23.241310 2607 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 22:12:23.237345 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.135.193:44397 (local address 127.0.0.1:43462)
0923 22:12:23.237615 (+ 270us) negotiation.cc:107] Waiting for socket to connect
0923 22:12:23.237626 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0923 22:12:23.237682 (+ 56us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0923 22:12:23.237940 (+ 258us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0923 22:12:23.237941 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0923 22:12:23.237944 (+ 3us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0923 22:12:23.238014 (+ 70us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 22:12:23.238016 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 22:12:23.239019 (+ 1003us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 22:12:23.239020 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 22:12:23.240289 (+ 1269us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 22:12:23.240292 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 22:12:23.241002 (+ 710us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 22:12:23.241003 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 22:12:23.241090 (+ 87us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 22:12:23.241102 (+ 12us) client_negotiation.cc:769] Sending connection context
0923 22:12:23.241193 (+ 91us) client_negotiation.cc:240] Negotiation successful
0923 22:12:23.241243 (+ 50us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":257,"thread_start_us":86,"threads_started":1}
I20260923 22:12:23.241319 2612 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 22:12:23.237609 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43462 (local address 127.1.135.193:44397)
0923 22:12:23.237781 (+ 172us) server_negotiation.cc:206] Beginning negotiation
0923 22:12:23.237787 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0923 22:12:23.237801 (+ 14us) server_negotiation.cc:407] Connection header received
0923 22:12:23.237831 (+ 30us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 22:12:23.237833 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 22:12:23.237844 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 22:12:23.237934 (+ 90us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0923 22:12:23.238122 (+ 188us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 22:12:23.238921 (+ 799us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 22:12:23.240375 (+ 1454us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 22:12:23.240898 (+ 523us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 22:12:23.240998 (+ 100us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 22:12:23.241067 (+ 69us) server_negotiation.cc:1035] Waiting for connection context
0923 22:12:23.241211 (+ 144us) server_negotiation.cc:299] Negotiation successful
0923 22:12:23.241252 (+ 41us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":146,"thread_start_us":78,"threads_started":1}
I20260923 22:12:23.241608 2609 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 22:12:23.237344 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.135.194:43183 (local address 127.0.0.1:40208)
0923 22:12:23.237532 (+ 188us) negotiation.cc:107] Waiting for socket to connect
0923 22:12:23.237569 (+ 37us) client_negotiation.cc:174] Beginning negotiation
0923 22:12:23.237632 (+ 63us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0923 22:12:23.237937 (+ 305us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0923 22:12:23.237940 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0923 22:12:23.237942 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0923 22:12:23.238006 (+ 64us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 22:12:23.238008 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 22:12:23.239254 (+ 1246us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 22:12:23.239255 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 22:12:23.240321 (+ 1066us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 22:12:23.240323 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 22:12:23.241244 (+ 921us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 22:12:23.241245 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 22:12:23.241314 (+ 69us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 22:12:23.241318 (+ 4us) client_negotiation.cc:769] Sending connection context
0923 22:12:23.241347 (+ 29us) client_negotiation.cc:240] Negotiation successful
0923 22:12:23.241544 (+ 197us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":161,"thread_start_us":53,"threads_started":1}
I20260923 22:12:23.241616 2611 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 22:12:23.237595 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48974 (local address 127.1.135.195:36101)
0923 22:12:23.237749 (+ 154us) server_negotiation.cc:206] Beginning negotiation
0923 22:12:23.237754 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0923 22:12:23.238146 (+ 392us) server_negotiation.cc:407] Connection header received
0923 22:12:23.238184 (+ 38us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 22:12:23.238186 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 22:12:23.238200 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 22:12:23.238263 (+ 63us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0923 22:12:23.238456 (+ 193us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 22:12:23.239387 (+ 931us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 22:12:23.240489 (+ 1102us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 22:12:23.241146 (+ 657us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 22:12:23.241216 (+ 70us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 22:12:23.241264 (+ 48us) server_negotiation.cc:1035] Waiting for connection context
0923 22:12:23.241501 (+ 237us) server_negotiation.cc:299] Negotiation successful
0923 22:12:23.241549 (+ 48us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":127,"thread_start_us":68,"threads_started":1}
I20260923 22:12:23.241786 2608 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 22:12:23.237385 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.135.195:36101 (local address 127.0.0.1:48974)
0923 22:12:23.238034 (+ 649us) negotiation.cc:107] Waiting for socket to connect
0923 22:12:23.238044 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0923 22:12:23.238091 (+ 47us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0923 22:12:23.238328 (+ 237us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0923 22:12:23.238329 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0923 22:12:23.238331 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0923 22:12:23.238370 (+ 39us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 22:12:23.238371 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 22:12:23.239482 (+ 1111us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 22:12:23.239483 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 22:12:23.240406 (+ 923us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 22:12:23.240408 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 22:12:23.241290 (+ 882us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 22:12:23.241291 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 22:12:23.241339 (+ 48us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 22:12:23.241342 (+ 3us) client_negotiation.cc:769] Sending connection context
0923 22:12:23.241385 (+ 43us) client_negotiation.cc:240] Negotiation successful
0923 22:12:23.241730 (+ 345us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":631,"thread_start_us":104,"threads_started":1}
I20260923 22:12:23.242163 2262 tablet_service.cc:1467] Processing CreateTablet for tablet 1f06c0e0a02d4da9afe1e4ac382137a4 (DEFAULT_TABLE table=test-table [id=85dcca53144842bcb97420b96dde0b61]), partition=RANGE (key) PARTITION UNBOUNDED
I20260923 22:12:23.242365 2530 tablet_service.cc:1467] Processing CreateTablet for tablet 1f06c0e0a02d4da9afe1e4ac382137a4 (DEFAULT_TABLE table=test-table [id=85dcca53144842bcb97420b96dde0b61]), partition=RANGE (key) PARTITION UNBOUNDED
I20260923 22:12:23.242475 2262 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 1f06c0e0a02d4da9afe1e4ac382137a4. 1 dirs total, 0 dirs full, 0 dirs failed
I20260923 22:12:23.242659 2530 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 1f06c0e0a02d4da9afe1e4ac382137a4. 1 dirs total, 0 dirs full, 0 dirs failed
I20260923 22:12:23.244407 2610 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 22:12:23.237454 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40208 (local address 127.1.135.194:43183)
0923 22:12:23.237778 (+ 324us) server_negotiation.cc:206] Beginning negotiation
0923 22:12:23.237782 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0923 22:12:23.237794 (+ 12us) server_negotiation.cc:407] Connection header received
0923 22:12:23.237830 (+ 36us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 22:12:23.237831 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 22:12:23.237843 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 22:12:23.237939 (+ 96us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0923 22:12:23.238065 (+ 126us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 22:12:23.239142 (+ 1077us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 22:12:23.240405 (+ 1263us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 22:12:23.241146 (+ 741us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 22:12:23.244191 (+ 3045us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 22:12:23.244272 (+ 81us) server_negotiation.cc:1035] Waiting for connection context
0923 22:12:23.244297 (+ 25us) server_negotiation.cc:299] Negotiation successful
0923 22:12:23.244335 (+ 38us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":288,"thread_start_us":79,"threads_started":1}
I20260923 22:12:23.245054 2613 tablet_bootstrap.cc:492] T 1f06c0e0a02d4da9afe1e4ac382137a4 P 0983183088f24c5bac2c87918556364c: Bootstrap starting.
I20260923 22:12:23.245126 2614 tablet_bootstrap.cc:492] T 1f06c0e0a02d4da9afe1e4ac382137a4 P f53d83ad2d734dddb2a49178ea23d7d9: Bootstrap starting.
I20260923 22:12:23.245687 2614 tablet_bootstrap.cc:654] T 1f06c0e0a02d4da9afe1e4ac382137a4 P f53d83ad2d734dddb2a49178ea23d7d9: Neither blocks nor log segments found. Creating new log.
I20260923 22:12:23.245709 2396 tablet_service.cc:1467] Processing CreateTablet for tablet 1f06c0e0a02d4da9afe1e4ac382137a4 (DEFAULT_TABLE table=test-table [id=85dcca53144842bcb97420b96dde0b61]), partition=RANGE (key) PARTITION UNBOUNDED
I20260923 22:12:23.245839 2613 tablet_bootstrap.cc:654] T 1f06c0e0a02d4da9afe1e4ac382137a4 P 0983183088f24c5bac2c87918556364c: Neither blocks nor log segments found. Creating new log.
I20260923 22:12:23.246078 2396 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 1f06c0e0a02d4da9afe1e4ac382137a4. 1 dirs total, 0 dirs full, 0 dirs failed
I20260923 22:12:23.246322 2614 log.cc:824] T 1f06c0e0a02d4da9afe1e4ac382137a4 P f53d83ad2d734dddb2a49178ea23d7d9: Log is configured to *not* fsync() on all Append() calls
I20260923 22:12:23.246444 2613 log.cc:824] T 1f06c0e0a02d4da9afe1e4ac382137a4 P 0983183088f24c5bac2c87918556364c: Log is configured to *not* fsync() on all Append() calls
I20260923 22:12:23.247491 2614 tablet_bootstrap.cc:492] T 1f06c0e0a02d4da9afe1e4ac382137a4 P f53d83ad2d734dddb2a49178ea23d7d9: No bootstrap required, opened a new log
I20260923 22:12:23.247572 2614 ts_tablet_manager.cc:1397] T 1f06c0e0a02d4da9afe1e4ac382137a4 P f53d83ad2d734dddb2a49178ea23d7d9: Time spent bootstrapping tablet: real 0.002s user 0.002s sys 0.000s
I20260923 22:12:23.248667 2613 tablet_bootstrap.cc:492] T 1f06c0e0a02d4da9afe1e4ac382137a4 P 0983183088f24c5bac2c87918556364c: No bootstrap required, opened a new log
I20260923 22:12:23.248744 2613 ts_tablet_manager.cc:1397] T 1f06c0e0a02d4da9afe1e4ac382137a4 P 0983183088f24c5bac2c87918556364c: Time spent bootstrapping tablet: real 0.004s user 0.002s sys 0.000s
I20260923 22:12:23.249032 2617 tablet_bootstrap.cc:492] T 1f06c0e0a02d4da9afe1e4ac382137a4 P a578afa357444ef7bec6d019e2634cd8: Bootstrap starting.
I20260923 22:12:23.249264 2614 raft_consensus.cc:348] T 1f06c0e0a02d4da9afe1e4ac382137a4 P f53d83ad2d734dddb2a49178ea23d7d9 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a578afa357444ef7bec6d019e2634cd8" member_type: VOTER last_known_addr { host: "127.1.135.194" port: 43183 } } peers { permanent_uuid: "0983183088f24c5bac2c87918556364c" member_type: VOTER last_known_addr { host: "127.1.135.193" port: 44397 } } peers { permanent_uuid: "f53d83ad2d734dddb2a49178ea23d7d9" member_type: VOTER last_known_addr { host: "127.1.135.195" port: 36101 } }
I20260923 22:12:23.249421 2614 raft_consensus.cc:374] T 1f06c0e0a02d4da9afe1e4ac382137a4 P f53d83ad2d734dddb2a49178ea23d7d9 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260923 22:12:23.249460 2614 raft_consensus.cc:729] T 1f06c0e0a02d4da9afe1e4ac382137a4 P f53d83ad2d734dddb2a49178ea23d7d9 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: f53d83ad2d734dddb2a49178ea23d7d9, State: Initialized, Role: FOLLOWER
I20260923 22:12:23.249617 2617 tablet_bootstrap.cc:654] T 1f06c0e0a02d4da9afe1e4ac382137a4 P a578afa357444ef7bec6d019e2634cd8: Neither blocks nor log segments found. Creating new log.
I20260923 22:12:23.249593 2614 consensus_queue.cc:260] T 1f06c0e0a02d4da9afe1e4ac382137a4 P f53d83ad2d734dddb2a49178ea23d7d9 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a578afa357444ef7bec6d019e2634cd8" member_type: VOTER last_known_addr { host: "127.1.135.194" port: 43183 } } peers { permanent_uuid: "0983183088f24c5bac2c87918556364c" member_type: VOTER last_known_addr { host: "127.1.135.193" port: 44397 } } peers { permanent_uuid: "f53d83ad2d734dddb2a49178ea23d7d9" member_type: VOTER last_known_addr { host: "127.1.135.195" port: 36101 } }
I20260923 22:12:23.249850 2614 ts_tablet_manager.cc:1428] T 1f06c0e0a02d4da9afe1e4ac382137a4 P f53d83ad2d734dddb2a49178ea23d7d9: Time spent starting tablet: real 0.002s user 0.002s sys 0.000s
I20260923 22:12:23.249984 2596 heartbeater.cc:503] Master 127.1.135.254:32983 was elected leader, sending a full tablet report...
I20260923 22:12:23.250110 2617 log.cc:824] T 1f06c0e0a02d4da9afe1e4ac382137a4 P a578afa357444ef7bec6d019e2634cd8: Log is configured to *not* fsync() on all Append() calls
I20260923 22:12:23.250761 2613 raft_consensus.cc:348] T 1f06c0e0a02d4da9afe1e4ac382137a4 P 0983183088f24c5bac2c87918556364c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a578afa357444ef7bec6d019e2634cd8" member_type: VOTER last_known_addr { host: "127.1.135.194" port: 43183 } } peers { permanent_uuid: "0983183088f24c5bac2c87918556364c" member_type: VOTER last_known_addr { host: "127.1.135.193" port: 44397 } } peers { permanent_uuid: "f53d83ad2d734dddb2a49178ea23d7d9" member_type: VOTER last_known_addr { host: "127.1.135.195" port: 36101 } }
I20260923 22:12:23.250950 2613 raft_consensus.cc:374] T 1f06c0e0a02d4da9afe1e4ac382137a4 P 0983183088f24c5bac2c87918556364c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260923 22:12:23.250993 2613 raft_consensus.cc:729] T 1f06c0e0a02d4da9afe1e4ac382137a4 P 0983183088f24c5bac2c87918556364c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 0983183088f24c5bac2c87918556364c, State: Initialized, Role: FOLLOWER
I20260923 22:12:23.251129 2617 tablet_bootstrap.cc:492] T 1f06c0e0a02d4da9afe1e4ac382137a4 P a578afa357444ef7bec6d019e2634cd8: No bootstrap required, opened a new log
I20260923 22:12:23.251102 2613 consensus_queue.cc:260] T 1f06c0e0a02d4da9afe1e4ac382137a4 P 0983183088f24c5bac2c87918556364c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a578afa357444ef7bec6d019e2634cd8" member_type: VOTER last_known_addr { host: "127.1.135.194" port: 43183 } } peers { permanent_uuid: "0983183088f24c5bac2c87918556364c" member_type: VOTER last_known_addr { host: "127.1.135.193" port: 44397 } } peers { permanent_uuid: "f53d83ad2d734dddb2a49178ea23d7d9" member_type: VOTER last_known_addr { host: "127.1.135.195" port: 36101 } }
I20260923 22:12:23.251188 2617 ts_tablet_manager.cc:1397] T 1f06c0e0a02d4da9afe1e4ac382137a4 P a578afa357444ef7bec6d019e2634cd8: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20260923 22:12:23.251354 2613 ts_tablet_manager.cc:1428] T 1f06c0e0a02d4da9afe1e4ac382137a4 P 0983183088f24c5bac2c87918556364c: Time spent starting tablet: real 0.003s user 0.003s sys 0.000s
I20260923 22:12:23.251366 2328 heartbeater.cc:503] Master 127.1.135.254:32983 was elected leader, sending a full tablet report...
I20260923 22:12:23.252847 2617 raft_consensus.cc:348] T 1f06c0e0a02d4da9afe1e4ac382137a4 P a578afa357444ef7bec6d019e2634cd8 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a578afa357444ef7bec6d019e2634cd8" member_type: VOTER last_known_addr { host: "127.1.135.194" port: 43183 } } peers { permanent_uuid: "0983183088f24c5bac2c87918556364c" member_type: VOTER last_known_addr { host: "127.1.135.193" port: 44397 } } peers { permanent_uuid: "f53d83ad2d734dddb2a49178ea23d7d9" member_type: VOTER last_known_addr { host: "127.1.135.195" port: 36101 } }
I20260923 22:12:23.252977 2617 raft_consensus.cc:374] T 1f06c0e0a02d4da9afe1e4ac382137a4 P a578afa357444ef7bec6d019e2634cd8 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260923 22:12:23.253000 2617 raft_consensus.cc:729] T 1f06c0e0a02d4da9afe1e4ac382137a4 P a578afa357444ef7bec6d019e2634cd8 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: a578afa357444ef7bec6d019e2634cd8, State: Initialized, Role: FOLLOWER
I20260923 22:12:23.253072 2617 consensus_queue.cc:260] T 1f06c0e0a02d4da9afe1e4ac382137a4 P a578afa357444ef7bec6d019e2634cd8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a578afa357444ef7bec6d019e2634cd8" member_type: VOTER last_known_addr { host: "127.1.135.194" port: 43183 } } peers { permanent_uuid: "0983183088f24c5bac2c87918556364c" member_type: VOTER last_known_addr { host: "127.1.135.193" port: 44397 } } peers { permanent_uuid: "f53d83ad2d734dddb2a49178ea23d7d9" member_type: VOTER last_known_addr { host: "127.1.135.195" port: 36101 } }
I20260923 22:12:23.253713 2617 ts_tablet_manager.cc:1428] T 1f06c0e0a02d4da9afe1e4ac382137a4 P a578afa357444ef7bec6d019e2634cd8: Time spent starting tablet: real 0.002s user 0.002s sys 0.000s
I20260923 22:12:23.253723 2462 heartbeater.cc:503] Master 127.1.135.254:32983 was elected leader, sending a full tablet report...
W20260923 22:12:23.307811 2463 tablet.cc:2369] T 1f06c0e0a02d4da9afe1e4ac382137a4 P a578afa357444ef7bec6d019e2634cd8: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260923 22:12:23.405720 2329 tablet.cc:2369] T 1f06c0e0a02d4da9afe1e4ac382137a4 P 0983183088f24c5bac2c87918556364c: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260923 22:12:23.435886 2620 raft_consensus.cc:482] T 1f06c0e0a02d4da9afe1e4ac382137a4 P 0983183088f24c5bac2c87918556364c [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260923 22:12:23.436034 2620 raft_consensus.cc:504] T 1f06c0e0a02d4da9afe1e4ac382137a4 P 0983183088f24c5bac2c87918556364c [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a578afa357444ef7bec6d019e2634cd8" member_type: VOTER last_known_addr { host: "127.1.135.194" port: 43183 } } peers { permanent_uuid: "0983183088f24c5bac2c87918556364c" member_type: VOTER last_known_addr { host: "127.1.135.193" port: 44397 } } peers { permanent_uuid: "f53d83ad2d734dddb2a49178ea23d7d9" member_type: VOTER last_known_addr { host: "127.1.135.195" port: 36101 } }
I20260923 22:12:23.436460 2620 leader_election.cc:288] T 1f06c0e0a02d4da9afe1e4ac382137a4 P 0983183088f24c5bac2c87918556364c [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers a578afa357444ef7bec6d019e2634cd8 (127.1.135.194:43183), f53d83ad2d734dddb2a49178ea23d7d9 (127.1.135.195:36101)
I20260923 22:12:23.439370 2610 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 22:12:23.436733 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.135.193:49973 (local address 127.1.135.194:43183)
0923 22:12:23.436781 (+ 48us) server_negotiation.cc:206] Beginning negotiation
0923 22:12:23.436783 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0923 22:12:23.436888 (+ 105us) server_negotiation.cc:407] Connection header received
0923 22:12:23.436975 (+ 87us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 22:12:23.436976 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 22:12:23.436988 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 22:12:23.437063 (+ 75us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0923 22:12:23.437248 (+ 185us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 22:12:23.437705 (+ 457us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 22:12:23.438653 (+ 948us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 22:12:23.439020 (+ 367us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 22:12:23.439104 (+ 84us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 22:12:23.439112 (+ 8us) server_negotiation.cc:1035] Waiting for connection context
0923 22:12:23.439268 (+ 156us) server_negotiation.cc:299] Negotiation successful
0923 22:12:23.439303 (+ 35us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":23}
I20260923 22:12:23.439371 2611 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 22:12:23.436800 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.135.193:36665 (local address 127.1.135.195:36101)
0923 22:12:23.436849 (+ 49us) server_negotiation.cc:206] Beginning negotiation
0923 22:12:23.436851 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0923 22:12:23.436861 (+ 10us) server_negotiation.cc:407] Connection header received
0923 22:12:23.436967 (+ 106us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 22:12:23.436968 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 22:12:23.436978 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 22:12:23.437047 (+ 69us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0923 22:12:23.437217 (+ 170us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 22:12:23.437693 (+ 476us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 22:12:23.438657 (+ 964us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 22:12:23.439017 (+ 360us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 22:12:23.439100 (+ 83us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 22:12:23.439109 (+ 9us) server_negotiation.cc:1035] Waiting for connection context
0923 22:12:23.439260 (+ 151us) server_negotiation.cc:299] Negotiation successful
0923 22:12:23.439296 (+ 36us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":25}
I20260923 22:12:23.439395 2623 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 22:12:23.436662 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.135.194:43183 (local address 127.1.135.193:49973)
0923 22:12:23.436818 (+ 156us) negotiation.cc:107] Waiting for socket to connect
0923 22:12:23.436830 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0923 22:12:23.436906 (+ 76us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0923 22:12:23.437068 (+ 162us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0923 22:12:23.437069 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0923 22:12:23.437071 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0923 22:12:23.437158 (+ 87us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 22:12:23.437159 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 22:12:23.437792 (+ 633us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 22:12:23.437793 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 22:12:23.438562 (+ 769us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 22:12:23.438564 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 22:12:23.439105 (+ 541us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 22:12:23.439106 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 22:12:23.439178 (+ 72us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 22:12:23.439186 (+ 8us) client_negotiation.cc:769] Sending connection context
0923 22:12:23.439260 (+ 74us) client_negotiation.cc:240] Negotiation successful
0923 22:12:23.439303 (+ 43us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":142,"mutex_wait_us":13,"thread_start_us":68,"threads_started":1}
I20260923 22:12:23.439399 2622 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 22:12:23.436660 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.135.195:36101 (local address 127.1.135.193:36665)
0923 22:12:23.436804 (+ 144us) negotiation.cc:107] Waiting for socket to connect
0923 22:12:23.436816 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0923 22:12:23.436880 (+ 64us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0923 22:12:23.437050 (+ 170us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0923 22:12:23.437051 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0923 22:12:23.437053 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0923 22:12:23.437141 (+ 88us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 22:12:23.437143 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 22:12:23.437779 (+ 636us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 22:12:23.437780 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 22:12:23.438560 (+ 780us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 22:12:23.438562 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 22:12:23.439108 (+ 546us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 22:12:23.439109 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 22:12:23.439187 (+ 78us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 22:12:23.439190 (+ 3us) client_negotiation.cc:769] Sending connection context
0923 22:12:23.439254 (+ 64us) client_negotiation.cc:240] Negotiation successful
0923 22:12:23.439296 (+ 42us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":129,"thread_start_us":78,"threads_started":1}
I20260923 22:12:23.439739 2416 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "1f06c0e0a02d4da9afe1e4ac382137a4" candidate_uuid: "0983183088f24c5bac2c87918556364c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a578afa357444ef7bec6d019e2634cd8" is_pre_election: true
I20260923 22:12:23.439898 2416 raft_consensus.cc:2454] T 1f06c0e0a02d4da9afe1e4ac382137a4 P a578afa357444ef7bec6d019e2634cd8 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0983183088f24c5bac2c87918556364c in term 0.
I20260923 22:12:23.439898 2550 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "1f06c0e0a02d4da9afe1e4ac382137a4" candidate_uuid: "0983183088f24c5bac2c87918556364c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f53d83ad2d734dddb2a49178ea23d7d9" is_pre_election: true
I20260923 22:12:23.440009 2550 raft_consensus.cc:2454] T 1f06c0e0a02d4da9afe1e4ac382137a4 P f53d83ad2d734dddb2a49178ea23d7d9 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0983183088f24c5bac2c87918556364c in term 0.
I20260923 22:12:23.440150 2215 leader_election.cc:302] T 1f06c0e0a02d4da9afe1e4ac382137a4 P 0983183088f24c5bac2c87918556364c [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0983183088f24c5bac2c87918556364c, a578afa357444ef7bec6d019e2634cd8; no voters:
I20260923 22:12:23.440284 2620 raft_consensus.cc:2781] T 1f06c0e0a02d4da9afe1e4ac382137a4 P 0983183088f24c5bac2c87918556364c [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260923 22:12:23.440351 2620 raft_consensus.cc:482] T 1f06c0e0a02d4da9afe1e4ac382137a4 P 0983183088f24c5bac2c87918556364c [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260923 22:12:23.440374 2620 raft_consensus.cc:3037] T 1f06c0e0a02d4da9afe1e4ac382137a4 P 0983183088f24c5bac2c87918556364c [term 0 FOLLOWER]: Advancing to term 1
I20260923 22:12:23.441026 2620 raft_consensus.cc:504] T 1f06c0e0a02d4da9afe1e4ac382137a4 P 0983183088f24c5bac2c87918556364c [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a578afa357444ef7bec6d019e2634cd8" member_type: VOTER last_known_addr { host: "127.1.135.194" port: 43183 } } peers { permanent_uuid: "0983183088f24c5bac2c87918556364c" member_type: VOTER last_known_addr { host: "127.1.135.193" port: 44397 } } peers { permanent_uuid: "f53d83ad2d734dddb2a49178ea23d7d9" member_type: VOTER last_known_addr { host: "127.1.135.195" port: 36101 } }
I20260923 22:12:23.441164 2620 leader_election.cc:288] T 1f06c0e0a02d4da9afe1e4ac382137a4 P 0983183088f24c5bac2c87918556364c [CANDIDATE]: Term 1 election: Requested vote from peers a578afa357444ef7bec6d019e2634cd8 (127.1.135.194:43183), f53d83ad2d734dddb2a49178ea23d7d9 (127.1.135.195:36101)
I20260923 22:12:23.441304 2416 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "1f06c0e0a02d4da9afe1e4ac382137a4" candidate_uuid: "0983183088f24c5bac2c87918556364c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a578afa357444ef7bec6d019e2634cd8"
I20260923 22:12:23.441352 2550 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "1f06c0e0a02d4da9afe1e4ac382137a4" candidate_uuid: "0983183088f24c5bac2c87918556364c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f53d83ad2d734dddb2a49178ea23d7d9"
I20260923 22:12:23.441399 2416 raft_consensus.cc:3037] T 1f06c0e0a02d4da9afe1e4ac382137a4 P a578afa357444ef7bec6d019e2634cd8 [term 0 FOLLOWER]: Advancing to term 1
I20260923 22:12:23.441406 2550 raft_consensus.cc:3037] T 1f06c0e0a02d4da9afe1e4ac382137a4 P f53d83ad2d734dddb2a49178ea23d7d9 [term 0 FOLLOWER]: Advancing to term 1
I20260923 22:12:23.442094 2550 raft_consensus.cc:2454] T 1f06c0e0a02d4da9afe1e4ac382137a4 P f53d83ad2d734dddb2a49178ea23d7d9 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0983183088f24c5bac2c87918556364c in term 1.
I20260923 22:12:23.442097 2416 raft_consensus.cc:2454] T 1f06c0e0a02d4da9afe1e4ac382137a4 P a578afa357444ef7bec6d019e2634cd8 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0983183088f24c5bac2c87918556364c in term 1.
I20260923 22:12:23.442253 2215 leader_election.cc:302] T 1f06c0e0a02d4da9afe1e4ac382137a4 P 0983183088f24c5bac2c87918556364c [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0983183088f24c5bac2c87918556364c, a578afa357444ef7bec6d019e2634cd8; no voters:
I20260923 22:12:23.442356 2620 raft_consensus.cc:2781] T 1f06c0e0a02d4da9afe1e4ac382137a4 P 0983183088f24c5bac2c87918556364c [term 1 FOLLOWER]: Leader election won for term 1
I20260923 22:12:23.442499 2620 raft_consensus.cc:686] T 1f06c0e0a02d4da9afe1e4ac382137a4 P 0983183088f24c5bac2c87918556364c [term 1 LEADER]: Becoming Leader. State: Replica: 0983183088f24c5bac2c87918556364c, State: Running, Role: LEADER
I20260923 22:12:23.442596 2620 consensus_queue.cc:237] T 1f06c0e0a02d4da9afe1e4ac382137a4 P 0983183088f24c5bac2c87918556364c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a578afa357444ef7bec6d019e2634cd8" member_type: VOTER last_known_addr { host: "127.1.135.194" port: 43183 } } peers { permanent_uuid: "0983183088f24c5bac2c87918556364c" member_type: VOTER last_known_addr { host: "127.1.135.193" port: 44397 } } peers { permanent_uuid: "f53d83ad2d734dddb2a49178ea23d7d9" member_type: VOTER last_known_addr { host: "127.1.135.195" port: 36101 } }
I20260923 22:12:23.443344 2136 catalog_manager.cc:5495] T 1f06c0e0a02d4da9afe1e4ac382137a4 P 0983183088f24c5bac2c87918556364c reported cstate change: term changed from 0 to 1, leader changed from <none> to 0983183088f24c5bac2c87918556364c (127.1.135.193). New cstate: current_term: 1 leader_uuid: "0983183088f24c5bac2c87918556364c" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a578afa357444ef7bec6d019e2634cd8" member_type: VOTER last_known_addr { host: "127.1.135.194" port: 43183 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "0983183088f24c5bac2c87918556364c" member_type: VOTER last_known_addr { host: "127.1.135.193" port: 44397 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "f53d83ad2d734dddb2a49178ea23d7d9" member_type: VOTER last_known_addr { host: "127.1.135.195" port: 36101 } health_report { overall_health: UNKNOWN } } }
W20260923 22:12:23.455358 2597 tablet.cc:2369] T 1f06c0e0a02d4da9afe1e4ac382137a4 P f53d83ad2d734dddb2a49178ea23d7d9: Can't schedule compaction. Clean time has not been advanced past its initial value.
Sep 23 22:12:23 dist-test-slave-s2fj krb5kdc[2092](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790201542, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.1.135.193@KRBTEST.COM
I20260923 22:12:23.472169 2612 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 22:12:23.464878 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43464 (local address 127.1.135.193:44397)
0923 22:12:23.464961 (+ 83us) server_negotiation.cc:206] Beginning negotiation
0923 22:12:23.464963 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0923 22:12:23.465046 (+ 83us) server_negotiation.cc:407] Connection header received
0923 22:12:23.465126 (+ 80us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 22:12:23.465127 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 22:12:23.465140 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 22:12:23.465212 (+ 72us) server_negotiation.cc:226] Negotiated authn=SASL
0923 22:12:23.465769 (+ 557us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 22:12:23.466260 (+ 491us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 22:12:23.467003 (+ 743us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 22:12:23.467192 (+ 189us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 22:12:23.467265 (+ 73us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 22:12:23.469522 (+ 2257us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0923 22:12:23.469533 (+ 11us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0923 22:12:23.469533 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0923 22:12:23.469540 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0923 22:12:23.470942 (+ 1402us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0923 22:12:23.471290 (+ 348us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0923 22:12:23.471300 (+ 10us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0923 22:12:23.471300 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0923 22:12:23.471351 (+ 51us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0923 22:12:23.471555 (+ 204us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0923 22:12:23.471556 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0923 22:12:23.471556 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0923 22:12:23.471690 (+ 134us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0923 22:12:23.471871 (+ 181us) server_negotiation.cc:1035] Waiting for connection context
0923 22:12:23.471959 (+ 88us) server_negotiation.cc:299] Negotiation successful
0923 22:12:23.472038 (+ 79us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":41}
Sep 23 22:12:23 dist-test-slave-s2fj krb5kdc[2092](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790201543, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-user@KRBTEST.COM:
Sep 23 22:12:23 dist-test-slave-s2fj krb5kdc[2092](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790201543, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for kudu/127.1.135.193@KRBTEST.COM
I20260923 22:12:23.494524 2612 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 22:12:23.487348 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43468 (local address 127.1.135.193:44397)
0923 22:12:23.487389 (+ 41us) server_negotiation.cc:206] Beginning negotiation
0923 22:12:23.487390 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0923 22:12:23.487863 (+ 473us) server_negotiation.cc:407] Connection header received
0923 22:12:23.487921 (+ 58us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 22:12:23.487922 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 22:12:23.487936 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 22:12:23.487985 (+ 49us) server_negotiation.cc:226] Negotiated authn=SASL
0923 22:12:23.488608 (+ 623us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 22:12:23.489074 (+ 466us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 22:12:23.489805 (+ 731us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 22:12:23.489994 (+ 189us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 22:12:23.490047 (+ 53us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 22:12:23.492259 (+ 2212us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0923 22:12:23.492260 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0923 22:12:23.492261 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0923 22:12:23.492266 (+ 5us) server_negotiation.cc:922] calling sasl_server_start()
0923 22:12:23.493640 (+ 1374us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0923 22:12:23.493917 (+ 277us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0923 22:12:23.493918 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0923 22:12:23.493918 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0923 22:12:23.493965 (+ 47us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0923 22:12:23.494102 (+ 137us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0923 22:12:23.494103 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0923 22:12:23.494103 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0923 22:12:23.494175 (+ 72us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0923 22:12:23.494256 (+ 81us) server_negotiation.cc:1035] Waiting for connection context
0923 22:12:23.494351 (+ 95us) server_negotiation.cc:299] Negotiation successful
0923 22:12:23.494405 (+ 54us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":21}
Sep 23 22:12:23 dist-test-slave-s2fj krb5kdc[2092](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790201543, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
Sep 23 22:12:23 dist-test-slave-s2fj krb5kdc[2092](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790201543, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.1.135.193@KRBTEST.COM
I20260923 22:12:23.515056 2612 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 22:12:23.508638 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43476 (local address 127.1.135.193:44397)
0923 22:12:23.508693 (+ 55us) server_negotiation.cc:206] Beginning negotiation
0923 22:12:23.508695 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0923 22:12:23.508783 (+ 88us) server_negotiation.cc:407] Connection header received
0923 22:12:23.508834 (+ 51us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 22:12:23.508835 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 22:12:23.508844 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 22:12:23.508883 (+ 39us) server_negotiation.cc:226] Negotiated authn=SASL
0923 22:12:23.509396 (+ 513us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 22:12:23.509840 (+ 444us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 22:12:23.510502 (+ 662us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 22:12:23.510670 (+ 168us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 22:12:23.510716 (+ 46us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 22:12:23.512792 (+ 2076us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0923 22:12:23.512799 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0923 22:12:23.512799 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0923 22:12:23.512805 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0923 22:12:23.514223 (+ 1418us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0923 22:12:23.514516 (+ 293us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0923 22:12:23.514516 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0923 22:12:23.514516 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0923 22:12:23.514562 (+ 46us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0923 22:12:23.514714 (+ 152us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0923 22:12:23.514714 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0923 22:12:23.514715 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0923 22:12:23.514792 (+ 77us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0923 22:12:23.514878 (+ 86us) server_negotiation.cc:1035] Waiting for connection context
0923 22:12:23.514940 (+ 62us) server_negotiation.cc:299] Negotiation successful
0923 22:12:23.514994 (+ 54us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":35}
W20260923 22:12:23.515527 2242 tablet_service.cc:2908] Rejecting scan request for tablet 1f06c0e0a02d4da9afe1e4ac382137a4: Uninitialized: clean time has not yet been initialized
I20260923 22:12:23.516422 1567 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskRiQe_a/build/release/bin/kudu with pid 2199
I20260923 22:12:23.521219 1567 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskRiQe_a/build/release/bin/kudu with pid 2333
I20260923 22:12:23.525468 1567 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskRiQe_a/build/release/bin/kudu with pid 2467
I20260923 22:12:23.529639 1567 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskRiQe_a/build/release/bin/kudu with pid 2105
2026-09-23T22:12:23Z chronyd exiting
[ OK ] SecurityITest.TestAuthorizationOnChecksum (2090 ms)
[ RUN ] SecurityITest.SmokeTestAsAuthorizedUser
Loading random data
Initializing database '/tmp/dist-test-taskRiQe_a/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790201539513534-1567-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Sep 23 22:12:23 dist-test-slave-s2fj krb5kdc[2638](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Sep 23 22:12:23 dist-test-slave-s2fj krb5kdc[2638](info): set up 2 sockets
Sep 23 22:12:23 dist-test-slave-s2fj krb5kdc[2638](info): commencing operation
krb5kdc: starting...
W20260923 22:12:24.559677 1567 mini_kdc.cc:121] Time spent starting KDC: real 1.009s user 0.002s sys 0.002s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskRiQe_a/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790201539513534-1567-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskRiQe_a/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790201539513534-1567-0/krb5kdc/test-user.keytab.
Sep 23 22:12:24 dist-test-slave-s2fj krb5kdc[2638](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790201544, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-09-23T22:12:24Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-09-23T22:12:24Z Disabled control of system clock
WARNING: no policy specified for kudu/127.1.135.254@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.1.135.254@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.1.135.254 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskRiQe_a/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790201539513534-1567-0/krb5kdc/kudu_127.1.135.254.keytab.
Entry for principal kudu/127.1.135.254 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskRiQe_a/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790201539513534-1567-0/krb5kdc/kudu_127.1.135.254.keytab.
I20260923 22:12:24.668941 1567 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskRiQe_a/build/release/bin/kudu
/tmp/dist-test-taskRiQe_a/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskRiQe_a/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790201539513534-1567-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskRiQe_a/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790201539513534-1567-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskRiQe_a/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790201539513534-1567-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskRiQe_a/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790201539513534-1567-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.1.135.254:40067
--webserver_interface=127.1.135.254
--webserver_port=0
--keytab_file=/tmp/dist-test-taskRiQe_a/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790201539513534-1567-0/krb5kdc/kudu_127.1.135.254.keytab
--principal=kudu/127.1.135.254
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.1.135.212:42617
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.1.135.254:40067
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--txn_manager_enabled=true with env {KRB5CCNAME=/tmp/dist-test-taskRiQe_a/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790201539513534-1567-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskRiQe_a/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790201539513534-1567-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskRiQe_a/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790201539513534-1567-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260923 22:12:24.746265 2651 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260923 22:12:24.746505 2651 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260923 22:12:24.746541 2651 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260923 22:12:24.747694 2651 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260923 22:12:24.747767 2651 flags.cc:425] Enabled experimental flag: --txn_manager_enabled=true
W20260923 22:12:24.747789 2651 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260923 22:12:24.747810 2651 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260923 22:12:24.747841 2651 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260923 22:12:24.747869 2651 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260923 22:12:24.749375 2651 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.1.135.212:42617
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskRiQe_a/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790201539513534-1567-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskRiQe_a/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790201539513534-1567-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.1.135.254:40067
--txn_manager_enabled=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskRiQe_a/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790201539513534-1567-0/krb5kdc/kudu_127.1.135.254.keytab
--principal=kudu/127.1.135.254
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.1.135.254:40067
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskRiQe_a/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790201539513534-1567-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.1.135.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.2651
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskRiQe_a/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790201539513534-1567-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 23 Sep 2026 20:53:37 UTC on ccycloud.quasar-rkviyx.root.comops.site
build id 4668083
I20260923 22:12:24.749660 2651 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260923 22:12:24.749871 2651 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260923 22:12:24.754416 2656 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260923 22:12:24.754422 2657 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260923 22:12:24.754618 2651 server_base.cc:1034] running on GCE node
W20260923 22:12:24.754642 2659 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260923 22:12:24.754833 2651 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260923 22:12:24.755100 2651 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260923 22:12:24.756259 2651 hybrid_clock.cc:648] HybridClock initialized: now 1790201544756255 us; error 35 us; skew 500 ppm
Sep 23 22:12:24 dist-test-slave-s2fj krb5kdc[2638](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790201544, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.135.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260923 22:12:24.758572 2651 init.cc:377] Logged in from keytab as kudu/127.1.135.254@KRBTEST.COM (short username kudu)
I20260923 22:12:24.758956 2651 webserver.cc:460] Webserver started at http://127.1.135.254:35773/ using document root <none> and password file <none>
I20260923 22:12:24.759128 2651 fs_manager.cc:359] Metadata directory not provided
I20260923 22:12:24.759167 2651 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260923 22:12:24.759248 2651 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260923 22:12:24.760232 2651 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskRiQe_a/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790201539513534-1567-0/minicluster-data/master-0/data/instance:
uuid: "f63d6b7613c94887b75f55a19fcea635"
format_stamp: "Formatted at 2026-09-23 22:12:24 on dist-test-slave-s2fj"
server_key: "91210fa643b3a2d54b4007d3300468be"
server_key_iv: "7d73f5d48aeaf33a843327779b2077cd"
server_key_version: "encryptionkey@0"
I20260923 22:12:24.760521 2651 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskRiQe_a/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790201539513534-1567-0/minicluster-data/master-0/wal/instance:
uuid: "f63d6b7613c94887b75f55a19fcea635"
format_stamp: "Formatted at 2026-09-23 22:12:24 on dist-test-slave-s2fj"
server_key: "91210fa643b3a2d54b4007d3300468be"
server_key_iv: "7d73f5d48aeaf33a843327779b2077cd"
server_key_version: "encryptionkey@0"
I20260923 22:12:24.761708 2651 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.001s sys 0.000s
I20260923 22:12:24.762317 2666 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260923 22:12:24.762491 2651 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260923 22:12:24.762573 2651 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskRiQe_a/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790201539513534-1567-0/minicluster-data/master-0/data,/tmp/dist-test-taskRiQe_a/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790201539513534-1567-0/minicluster-data/master-0/wal
uuid: "f63d6b7613c94887b75f55a19fcea635"
format_stamp: "Formatted at 2026-09-23 22:12:24 on dist-test-slave-s2fj"
server_key: "91210fa643b3a2d54b4007d3300468be"
server_key_iv: "7d73f5d48aeaf33a843327779b2077cd"
server_key_version: "encryptionkey@0"
I20260923 22:12:24.762621 2651 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskRiQe_a/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790201539513534-1567-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskRiQe_a/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790201539513534-1567-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskRiQe_a/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790201539513534-1567-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260923 22:12:24.776999 2651 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260923 22:12:24.779035 2651 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260923 22:12:24.779146 2651 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260923 22:12:24.783464 2651 rpc_server.cc:307] RPC server started. Bound to: 127.1.135.254:40067
I20260923 22:12:24.783560 2728 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.1.135.254:40067 every 8 connection(s)
I20260923 22:12:24.783851 2651 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskRiQe_a/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790201539513534-1567-0/minicluster-data/master-0/data/info.pb
I20260923 22:12:24.784520 2729 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260923 22:12:24.786583 2729 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P f63d6b7613c94887b75f55a19fcea635: Bootstrap starting.
I20260923 22:12:24.787153 2729 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P f63d6b7613c94887b75f55a19fcea635: Neither blocks nor log segments found. Creating new log.
I20260923 22:12:24.787446 2729 log.cc:824] T 00000000000000000000000000000000 P f63d6b7613c94887b75f55a19fcea635: Log is configured to *not* fsync() on all Append() calls
I20260923 22:12:24.788085 2729 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P f63d6b7613c94887b75f55a19fcea635: No bootstrap required, opened a new log
I20260923 22:12:24.790138 2729 raft_consensus.cc:348] T 00000000000000000000000000000000 P f63d6b7613c94887b75f55a19fcea635 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f63d6b7613c94887b75f55a19fcea635" member_type: VOTER last_known_addr { host: "127.1.135.254" port: 40067 } }
I20260923 22:12:24.790262 2729 raft_consensus.cc:374] T 00000000000000000000000000000000 P f63d6b7613c94887b75f55a19fcea635 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260923 22:12:24.790283 2729 raft_consensus.cc:729] T 00000000000000000000000000000000 P f63d6b7613c94887b75f55a19fcea635 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: f63d6b7613c94887b75f55a19fcea635, State: Initialized, Role: FOLLOWER
I20260923 22:12:24.790450 2729 consensus_queue.cc:260] T 00000000000000000000000000000000 P f63d6b7613c94887b75f55a19fcea635 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f63d6b7613c94887b75f55a19fcea635" member_type: VOTER last_known_addr { host: "127.1.135.254" port: 40067 } }
I20260923 22:12:24.790509 2729 raft_consensus.cc:388] T 00000000000000000000000000000000 P f63d6b7613c94887b75f55a19fcea635 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260923 22:12:24.790530 2729 raft_consensus.cc:482] T 00000000000000000000000000000000 P f63d6b7613c94887b75f55a19fcea635 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260923 22:12:24.790598 2729 raft_consensus.cc:3037] T 00000000000000000000000000000000 P f63d6b7613c94887b75f55a19fcea635 [term 0 FOLLOWER]: Advancing to term 1
I20260923 22:12:24.791079 2729 raft_consensus.cc:504] T 00000000000000000000000000000000 P f63d6b7613c94887b75f55a19fcea635 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f63d6b7613c94887b75f55a19fcea635" member_type: VOTER last_known_addr { host: "127.1.135.254" port: 40067 } }
I20260923 22:12:24.791206 2729 leader_election.cc:302] T 00000000000000000000000000000000 P f63d6b7613c94887b75f55a19fcea635 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: f63d6b7613c94887b75f55a19fcea635; no voters:
I20260923 22:12:24.791376 2729 leader_election.cc:288] T 00000000000000000000000000000000 P f63d6b7613c94887b75f55a19fcea635 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260923 22:12:24.791448 2732 raft_consensus.cc:2781] T 00000000000000000000000000000000 P f63d6b7613c94887b75f55a19fcea635 [term 1 FOLLOWER]: Leader election won for term 1
I20260923 22:12:24.791603 2729 sys_catalog.cc:564] T 00000000000000000000000000000000 P f63d6b7613c94887b75f55a19fcea635 [sys.catalog]: configured and running, proceeding with master startup.
I20260923 22:12:24.791630 2732 raft_consensus.cc:686] T 00000000000000000000000000000000 P f63d6b7613c94887b75f55a19fcea635 [term 1 LEADER]: Becoming Leader. State: Replica: f63d6b7613c94887b75f55a19fcea635, State: Running, Role: LEADER
I20260923 22:12:24.791704 2732 consensus_queue.cc:237] T 00000000000000000000000000000000 P f63d6b7613c94887b75f55a19fcea635 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f63d6b7613c94887b75f55a19fcea635" member_type: VOTER last_known_addr { host: "127.1.135.254" port: 40067 } }
I20260923 22:12:24.792026 2733 sys_catalog.cc:455] T 00000000000000000000000000000000 P f63d6b7613c94887b75f55a19fcea635 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "f63d6b7613c94887b75f55a19fcea635" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f63d6b7613c94887b75f55a19fcea635" member_type: VOTER last_known_addr { host: "127.1.135.254" port: 40067 } } }
I20260923 22:12:24.792032 2734 sys_catalog.cc:455] T 00000000000000000000000000000000 P f63d6b7613c94887b75f55a19fcea635 [sys.catalog]: SysCatalogTable state changed. Reason: New leader f63d6b7613c94887b75f55a19fcea635. Latest consensus state: current_term: 1 leader_uuid: "f63d6b7613c94887b75f55a19fcea635" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f63d6b7613c94887b75f55a19fcea635" member_type: VOTER last_known_addr { host: "127.1.135.254" port: 40067 } } }
I20260923 22:12:24.792156 2733 sys_catalog.cc:458] T 00000000000000000000000000000000 P f63d6b7613c94887b75f55a19fcea635 [sys.catalog]: This master's current role is: LEADER
I20260923 22:12:24.792189 2734 sys_catalog.cc:458] T 00000000000000000000000000000000 P f63d6b7613c94887b75f55a19fcea635 [sys.catalog]: This master's current role is: LEADER
I20260923 22:12:24.792407 2741 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260923 22:12:24.792919 2741 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260923 22:12:24.793130 1567 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskRiQe_a/build/release/bin/kudu as pid 2651
I20260923 22:12:24.793215 1567 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskRiQe_a/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790201539513534-1567-0/minicluster-data/master-0/wal/instance
I20260923 22:12:24.793376 1567 external_mini_cluster.cc:1428] Setting key bb0b258c699988ff616a2df91a2e4294
I20260923 22:12:24.794685 2741 catalog_manager.cc:1347] Generated new cluster ID: 9516038f152342c7a16ac7c8fef3cee7
I20260923 22:12:24.794735 2741 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
Sep 23 22:12:24 dist-test-slave-s2fj krb5kdc[2638](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790201544, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.1.135.254@KRBTEST.COM
I20260923 22:12:24.802311 2741 catalog_manager.cc:1370] Generated new certificate authority record
I20260923 22:12:24.802959 2741 catalog_manager.cc:1504] Loading token signing keys...
I20260923 22:12:24.805346 2751 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 22:12:24.793787 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56360 (local address 127.1.135.254:40067)
0923 22:12:24.793931 (+ 144us) server_negotiation.cc:206] Beginning negotiation
0923 22:12:24.793939 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0923 22:12:24.794268 (+ 329us) server_negotiation.cc:407] Connection header received
0923 22:12:24.794382 (+ 114us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 22:12:24.794387 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 22:12:24.794427 (+ 40us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 22:12:24.794555 (+ 128us) server_negotiation.cc:226] Negotiated authn=SASL
0923 22:12:24.795251 (+ 696us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 22:12:24.796720 (+ 1469us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 22:12:24.797721 (+ 1001us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 22:12:24.798961 (+ 1240us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 22:12:24.799070 (+ 109us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 22:12:24.801755 (+ 2685us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0923 22:12:24.801759 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0923 22:12:24.801760 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0923 22:12:24.801770 (+ 10us) server_negotiation.cc:922] calling sasl_server_start()
0923 22:12:24.803794 (+ 2024us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0923 22:12:24.804202 (+ 408us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0923 22:12:24.804205 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0923 22:12:24.804205 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0923 22:12:24.804295 (+ 90us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0923 22:12:24.804509 (+ 214us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0923 22:12:24.804510 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0923 22:12:24.804510 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0923 22:12:24.804742 (+ 232us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0923 22:12:24.804903 (+ 161us) server_negotiation.cc:1035] Waiting for connection context
0923 22:12:24.805019 (+ 116us) server_negotiation.cc:299] Negotiation successful
0923 22:12:24.805147 (+ 128us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":99,"thread_start_us":72,"threads_started":1}
I20260923 22:12:24.806815 2741 catalog_manager.cc:5868] T 00000000000000000000000000000000 P f63d6b7613c94887b75f55a19fcea635: Generated new TSK 0
I20260923 22:12:24.806972 2741 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.1.135.193@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.1.135.193@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.1.135.193 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskRiQe_a/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790201539513534-1567-0/krb5kdc/kudu_127.1.135.193.keytab.
Entry for principal kudu/127.1.135.193 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskRiQe_a/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790201539513534-1567-0/krb5kdc/kudu_127.1.135.193.keytab.
I20260923 22:12:24.829088 1567 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskRiQe_a/build/release/bin/kudu
/tmp/dist-test-taskRiQe_a/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskRiQe_a/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790201539513534-1567-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskRiQe_a/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790201539513534-1567-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskRiQe_a/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790201539513534-1567-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskRiQe_a/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790201539513534-1567-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.1.135.193:0
--local_ip_for_outbound_sockets=127.1.135.193
--webserver_interface=127.1.135.193
--webserver_port=0
--tserver_master_addrs=127.1.135.254:40067
--keytab_file=/tmp/dist-test-taskRiQe_a/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790201539513534-1567-0/krb5kdc/kudu_127.1.135.193.keytab
--principal=kudu/127.1.135.193
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.1.135.212:42617
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--enable_txn_system_client_init=true with env {KRB5CCNAME=/tmp/dist-test-taskRiQe_a/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790201539513534-1567-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskRiQe_a/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790201539513534-1567-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskRiQe_a/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790201539513534-1567-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260923 22:12:24.907845 2755 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260923 22:12:24.908032 2755 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260923 22:12:24.908056 2755 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260923 22:12:24.909217 2755 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260923 22:12:24.909257 2755 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260923 22:12:24.909276 2755 flags.cc:425] Enabled experimental flag: --enable_txn_system_client_init=true
W20260923 22:12:24.909293 2755 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.1.135.193
I20260923 22:12:24.910696 2755 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.1.135.212:42617
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskRiQe_a/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790201539513534-1567-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskRiQe_a/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790201539513534-1567-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskRiQe_a/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790201539513534-1567-0/krb5kdc/kudu_127.1.135.193.keytab
--principal=kudu/127.1.135.193
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.1.135.193:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskRiQe_a/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790201539513534-1567-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.1.135.193
--webserver_port=0
--enable_txn_system_client_init=true
--tserver_master_addrs=127.1.135.254:40067
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.2755
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.1.135.193
--log_dir=/tmp/dist-test-taskRiQe_a/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790201539513534-1567-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 23 Sep 2026 20:53:37 UTC on ccycloud.quasar-rkviyx.root.comops.site
build id 4668083
I20260923 22:12:24.910943 2755 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260923 22:12:24.911177 2755 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260923 22:12:24.915962 2763 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260923 22:12:24.915962 2760 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260923 22:12:24.915974 2761 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260923 22:12:24.916152 2755 server_base.cc:1034] running on GCE node
I20260923 22:12:24.916428 2755 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260923 22:12:24.916674 2755 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260923 22:12:24.917848 2755 hybrid_clock.cc:648] HybridClock initialized: now 1790201544917843 us; error 35 us; skew 500 ppm
Sep 23 22:12:24 dist-test-slave-s2fj krb5kdc[2638](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790201544, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.135.193@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260923 22:12:24.920264 2755 init.cc:377] Logged in from keytab as kudu/127.1.135.193@KRBTEST.COM (short username kudu)
I20260923 22:12:24.920604 2755 webserver.cc:460] Webserver started at http://127.1.135.193:45439/ using document root <none> and password file <none>
I20260923 22:12:24.920817 2755 fs_manager.cc:359] Metadata directory not provided
I20260923 22:12:24.920859 2755 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260923 22:12:24.921003 2755 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260923 22:12:24.922004 2755 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskRiQe_a/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790201539513534-1567-0/minicluster-data/ts-0/data/instance:
uuid: "88a6b9e139ed41b49b48710b7d8658de"
format_stamp: "Formatted at 2026-09-23 22:12:24 on dist-test-slave-s2fj"
server_key: "65d0d09f9159f87f1057d96e162023ac"
server_key_iv: "ffc7e25a7372c3d1c02a4932ecc5bd48"
server_key_version: "encryptionkey@0"
I20260923 22:12:24.922351 2755 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskRiQe_a/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790201539513534-1567-0/minicluster-data/ts-0/wal/instance:
uuid: "88a6b9e139ed41b49b48710b7d8658de"
format_stamp: "Formatted at 2026-09-23 22:12:24 on dist-test-slave-s2fj"
server_key: "65d0d09f9159f87f1057d96e162023ac"
server_key_iv: "ffc7e25a7372c3d1c02a4932ecc5bd48"
server_key_version: "encryptionkey@0"
I20260923 22:12:24.923615 2755 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260923 22:12:24.924327 2770 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260923 22:12:24.924516 2755 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.001s
I20260923 22:12:24.924587 2755 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskRiQe_a/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790201539513534-1567-0/minicluster-data/ts-0/data,/tmp/dist-test-taskRiQe_a/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790201539513534-1567-0/minicluster-data/ts-0/wal
uuid: "88a6b9e139ed41b49b48710b7d8658de"
format_stamp: "Formatted at 2026-09-23 22:12:24 on dist-test-slave-s2fj"
server_key: "65d0d09f9159f87f1057d96e162023ac"
server_key_iv: "ffc7e25a7372c3d1c02a4932ecc5bd48"
server_key_version: "encryptionkey@0"
I20260923 22:12:24.924671 2755 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskRiQe_a/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790201539513534-1567-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskRiQe_a/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790201539513534-1567-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskRiQe_a/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790201539513534-1567-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260923 22:12:24.945017 2755 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260923 22:12:24.947170 2755 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260923 22:12:24.947321 2755 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260923 22:12:24.947770 2755 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260923 22:12:24.947808 2755 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260923 22:12:24.947860 2755 ts_tablet_manager.cc:610] Registered 0 tablets
I20260923 22:12:24.947888 2755 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
Sep 23 22:12:24 dist-test-slave-s2fj krb5kdc[2638](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790201544, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.135.193@KRBTEST.COM for kudu/127.1.135.254@KRBTEST.COM
I20260923 22:12:24.954953 2755 rpc_server.cc:307] RPC server started. Bound to: 127.1.135.193:44317
I20260923 22:12:24.955060 2885 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.1.135.193:44317 every 8 connection(s)
I20260923 22:12:24.955319 2755 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskRiQe_a/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790201539513534-1567-0/minicluster-data/ts-0/data/info.pb
I20260923 22:12:24.957780 2751 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 22:12:24.948264 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.135.193:42605 (local address 127.1.135.254:40067)
0923 22:12:24.948330 (+ 66us) server_negotiation.cc:206] Beginning negotiation
0923 22:12:24.948332 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0923 22:12:24.948587 (+ 255us) server_negotiation.cc:407] Connection header received
0923 22:12:24.949289 (+ 702us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 22:12:24.949290 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 22:12:24.949303 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 22:12:24.949374 (+ 71us) server_negotiation.cc:226] Negotiated authn=SASL
0923 22:12:24.950252 (+ 878us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 22:12:24.951162 (+ 910us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 22:12:24.952498 (+ 1336us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 22:12:24.952707 (+ 209us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 22:12:24.952790 (+ 83us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 22:12:24.955365 (+ 2575us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0923 22:12:24.955366 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0923 22:12:24.955366 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0923 22:12:24.955374 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0923 22:12:24.956728 (+ 1354us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0923 22:12:24.957075 (+ 347us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0923 22:12:24.957076 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0923 22:12:24.957076 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0923 22:12:24.957125 (+ 49us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0923 22:12:24.957284 (+ 159us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0923 22:12:24.957284 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0923 22:12:24.957284 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0923 22:12:24.957376 (+ 92us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0923 22:12:24.957424 (+ 48us) server_negotiation.cc:1035] Waiting for connection context
0923 22:12:24.957654 (+ 230us) server_negotiation.cc:299] Negotiation successful
0923 22:12:24.957718 (+ 64us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":37}
I20260923 22:12:24.957899 2779 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 22:12:24.948242 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.135.254:40067 (local address 127.1.135.193:42605)
0923 22:12:24.948427 (+ 185us) negotiation.cc:107] Waiting for socket to connect
0923 22:12:24.948476 (+ 49us) client_negotiation.cc:174] Beginning negotiation
0923 22:12:24.949167 (+ 691us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0923 22:12:24.949458 (+ 291us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0923 22:12:24.949463 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0923 22:12:24.949847 (+ 384us) client_negotiation.cc:189] Negotiated authn=SASL
0923 22:12:24.950136 (+ 289us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 22:12:24.950142 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 22:12:24.951266 (+ 1124us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 22:12:24.951268 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 22:12:24.952409 (+ 1141us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 22:12:24.952411 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 22:12:24.952810 (+ 399us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 22:12:24.952812 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 22:12:24.952930 (+ 118us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 22:12:24.953376 (+ 446us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0923 22:12:24.953387 (+ 11us) client_negotiation.cc:656] calling sasl_client_start()
0923 22:12:24.955264 (+ 1877us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0923 22:12:24.956797 (+ 1533us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0923 22:12:24.956802 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0923 22:12:24.956804 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0923 22:12:24.957009 (+ 205us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0923 22:12:24.957165 (+ 156us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0923 22:12:24.957165 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0923 22:12:24.957165 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0923 22:12:24.957242 (+ 77us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0923 22:12:24.957428 (+ 186us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0923 22:12:24.957432 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0923 22:12:24.957548 (+ 116us) client_negotiation.cc:769] Sending connection context
0923 22:12:24.957626 (+ 78us) client_negotiation.cc:240] Negotiation successful
0923 22:12:24.957729 (+ 103us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":130,"thread_start_us":89,"threads_started":1}
I20260923 22:12:24.958407 2886 heartbeater.cc:347] Connected to a master server at 127.1.135.254:40067
I20260923 22:12:24.958513 2886 heartbeater.cc:464] Registering TS with master...
I20260923 22:12:24.958673 2886 heartbeater.cc:511] Master 127.1.135.254:40067 requested a full tablet report, sending...
I20260923 22:12:24.959224 2682 ts_manager.cc:194] Registered new tserver with Master: 88a6b9e139ed41b49b48710b7d8658de (127.1.135.193:44317)
I20260923 22:12:24.959817 2682 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.1.135.193@KRBTEST.COM'} at 127.1.135.193:42605
I20260923 22:12:24.963405 1567 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskRiQe_a/build/release/bin/kudu as pid 2755
I20260923 22:12:24.963515 1567 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskRiQe_a/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790201539513534-1567-0/minicluster-data/ts-0/wal/instance
I20260923 22:12:24.963675 1567 external_mini_cluster.cc:1428] Setting key 4ffafab5bb73d2553a7df3443c0a0986
I20260923 22:12:24.964622 2892 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 22:12:24.959363 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.135.254:40067 (local address 127.1.135.193:53221)
0923 22:12:24.959536 (+ 173us) negotiation.cc:107] Waiting for socket to connect
0923 22:12:24.959545 (+ 9us) client_negotiation.cc:174] Beginning negotiation
0923 22:12:24.959622 (+ 77us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0923 22:12:24.959735 (+ 113us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0923 22:12:24.959736 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0923 22:12:24.959904 (+ 168us) client_negotiation.cc:189] Negotiated authn=SASL
0923 22:12:24.959976 (+ 72us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 22:12:24.959978 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 22:12:24.960824 (+ 846us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 22:12:24.960825 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 22:12:24.961301 (+ 476us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 22:12:24.961303 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 22:12:24.961688 (+ 385us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 22:12:24.961689 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 22:12:24.961762 (+ 73us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 22:12:24.962082 (+ 320us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0923 22:12:24.962086 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0923 22:12:24.962325 (+ 239us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0923 22:12:24.963527 (+ 1202us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0923 22:12:24.963528 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0923 22:12:24.963529 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0923 22:12:24.963717 (+ 188us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0923 22:12:24.963945 (+ 228us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0923 22:12:24.963946 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0923 22:12:24.963946 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0923 22:12:24.964009 (+ 63us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0923 22:12:24.964291 (+ 282us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0923 22:12:24.964292 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0923 22:12:24.964335 (+ 43us) client_negotiation.cc:769] Sending connection context
0923 22:12:24.964426 (+ 91us) client_negotiation.cc:240] Negotiation successful
0923 22:12:24.964534 (+ 108us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":152,"thread_start_us":127,"threads_started":1}
I20260923 22:12:24.964728 2751 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 22:12:24.959359 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.135.193:53221 (local address 127.1.135.254:40067)
0923 22:12:24.959390 (+ 31us) server_negotiation.cc:206] Beginning negotiation
0923 22:12:24.959391 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0923 22:12:24.959610 (+ 219us) server_negotiation.cc:407] Connection header received
0923 22:12:24.959677 (+ 67us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 22:12:24.959678 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 22:12:24.959688 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 22:12:24.959734 (+ 46us) server_negotiation.cc:226] Negotiated authn=SASL
0923 22:12:24.960034 (+ 300us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 22:12:24.960728 (+ 694us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 22:12:24.961403 (+ 675us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 22:12:24.961600 (+ 197us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 22:12:24.961689 (+ 89us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 22:12:24.962413 (+ 724us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0923 22:12:24.962415 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0923 22:12:24.962415 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0923 22:12:24.962422 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0923 22:12:24.963443 (+ 1021us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0923 22:12:24.963803 (+ 360us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0923 22:12:24.963805 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0923 22:12:24.963805 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0923 22:12:24.963864 (+ 59us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0923 22:12:24.964114 (+ 250us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0923 22:12:24.964115 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0923 22:12:24.964116 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0923 22:12:24.964227 (+ 111us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0923 22:12:24.964281 (+ 54us) server_negotiation.cc:1035] Waiting for connection context
0923 22:12:24.964571 (+ 290us) server_negotiation.cc:299] Negotiation successful
0923 22:12:24.964628 (+ 57us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":15}
WARNING: no policy specified for kudu/127.1.135.194@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.1.135.194@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.1.135.194 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskRiQe_a/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790201539513534-1567-0/krb5kdc/kudu_127.1.135.194.keytab.
Entry for principal kudu/127.1.135.194 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskRiQe_a/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790201539513534-1567-0/krb5kdc/kudu_127.1.135.194.keytab.
I20260923 22:12:24.986150 1567 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskRiQe_a/build/release/bin/kudu
/tmp/dist-test-taskRiQe_a/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskRiQe_a/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790201539513534-1567-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskRiQe_a/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790201539513534-1567-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskRiQe_a/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790201539513534-1567-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskRiQe_a/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790201539513534-1567-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.1.135.194:0
--local_ip_for_outbound_sockets=127.1.135.194
--webserver_interface=127.1.135.194
--webserver_port=0
--tserver_master_addrs=127.1.135.254:40067
--keytab_file=/tmp/dist-test-taskRiQe_a/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790201539513534-1567-0/krb5kdc/kudu_127.1.135.194.keytab
--principal=kudu/127.1.135.194
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.1.135.212:42617
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--enable_txn_system_client_init=true with env {KRB5CCNAME=/tmp/dist-test-taskRiQe_a/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790201539513534-1567-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskRiQe_a/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790201539513534-1567-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskRiQe_a/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790201539513534-1567-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260923 22:12:25.064312 2895 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260923 22:12:25.064553 2895 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260923 22:12:25.064581 2895 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260923 22:12:25.065841 2895 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260923 22:12:25.065881 2895 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260923 22:12:25.065922 2895 flags.cc:425] Enabled experimental flag: --enable_txn_system_client_init=true
W20260923 22:12:25.065958 2895 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.1.135.194
I20260923 22:12:25.067381 2895 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.1.135.212:42617
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskRiQe_a/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790201539513534-1567-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskRiQe_a/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790201539513534-1567-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskRiQe_a/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790201539513534-1567-0/krb5kdc/kudu_127.1.135.194.keytab
--principal=kudu/127.1.135.194
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.1.135.194:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskRiQe_a/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790201539513534-1567-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.1.135.194
--webserver_port=0
--enable_txn_system_client_init=true
--tserver_master_addrs=127.1.135.254:40067
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.2895
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.1.135.194
--log_dir=/tmp/dist-test-taskRiQe_a/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790201539513534-1567-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 23 Sep 2026 20:53:37 UTC on ccycloud.quasar-rkviyx.root.comops.site
build id 4668083
I20260923 22:12:25.067610 2895 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260923 22:12:25.067830 2895 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260923 22:12:25.072659 2903 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260923 22:12:25.072639 2900 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260923 22:12:25.072639 2901 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260923 22:12:25.072662 2895 server_base.cc:1034] running on GCE node
I20260923 22:12:25.073043 2895 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260923 22:12:25.073285 2895 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260923 22:12:25.074460 2895 hybrid_clock.cc:648] HybridClock initialized: now 1790201545074443 us; error 32 us; skew 500 ppm
Sep 23 22:12:25 dist-test-slave-s2fj krb5kdc[2638](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790201545, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.135.194@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260923 22:12:25.076887 2895 init.cc:377] Logged in from keytab as kudu/127.1.135.194@KRBTEST.COM (short username kudu)
I20260923 22:12:25.077347 2895 webserver.cc:460] Webserver started at http://127.1.135.194:44139/ using document root <none> and password file <none>
I20260923 22:12:25.077571 2895 fs_manager.cc:359] Metadata directory not provided
I20260923 22:12:25.077615 2895 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260923 22:12:25.077775 2895 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260923 22:12:25.078743 2895 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskRiQe_a/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790201539513534-1567-0/minicluster-data/ts-1/data/instance:
uuid: "a4124d4f2ad348d09b2f8903f9f35ba5"
format_stamp: "Formatted at 2026-09-23 22:12:25 on dist-test-slave-s2fj"
server_key: "02ff36653e38d2db65db62337b543615"
server_key_iv: "bd79d41c160ba3cfe3294e21e23fef1c"
server_key_version: "encryptionkey@0"
I20260923 22:12:25.079083 2895 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskRiQe_a/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790201539513534-1567-0/minicluster-data/ts-1/wal/instance:
uuid: "a4124d4f2ad348d09b2f8903f9f35ba5"
format_stamp: "Formatted at 2026-09-23 22:12:25 on dist-test-slave-s2fj"
server_key: "02ff36653e38d2db65db62337b543615"
server_key_iv: "bd79d41c160ba3cfe3294e21e23fef1c"
server_key_version: "encryptionkey@0"
I20260923 22:12:25.080387 2895 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.003s
I20260923 22:12:25.081037 2910 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260923 22:12:25.081194 2895 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.001s
I20260923 22:12:25.081292 2895 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskRiQe_a/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790201539513534-1567-0/minicluster-data/ts-1/data,/tmp/dist-test-taskRiQe_a/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790201539513534-1567-0/minicluster-data/ts-1/wal
uuid: "a4124d4f2ad348d09b2f8903f9f35ba5"
format_stamp: "Formatted at 2026-09-23 22:12:25 on dist-test-slave-s2fj"
server_key: "02ff36653e38d2db65db62337b543615"
server_key_iv: "bd79d41c160ba3cfe3294e21e23fef1c"
server_key_version: "encryptionkey@0"
I20260923 22:12:25.081369 2895 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskRiQe_a/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790201539513534-1567-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskRiQe_a/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790201539513534-1567-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskRiQe_a/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790201539513534-1567-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260923 22:12:25.098510 2895 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260923 22:12:25.100728 2895 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260923 22:12:25.100888 2895 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260923 22:12:25.101414 2895 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260923 22:12:25.101472 2895 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260923 22:12:25.101512 2895 ts_tablet_manager.cc:610] Registered 0 tablets
I20260923 22:12:25.101533 2895 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
Sep 23 22:12:25 dist-test-slave-s2fj krb5kdc[2638](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790201545, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.135.194@KRBTEST.COM for kudu/127.1.135.254@KRBTEST.COM
I20260923 22:12:25.108443 2895 rpc_server.cc:307] RPC server started. Bound to: 127.1.135.194:36581
I20260923 22:12:25.108549 3025 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.1.135.194:36581 every 8 connection(s)
I20260923 22:12:25.108816 2895 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskRiQe_a/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790201539513534-1567-0/minicluster-data/ts-1/data/info.pb
I20260923 22:12:25.110415 1567 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskRiQe_a/build/release/bin/kudu as pid 2895
I20260923 22:12:25.110531 1567 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskRiQe_a/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790201539513534-1567-0/minicluster-data/ts-1/wal/instance
I20260923 22:12:25.110705 1567 external_mini_cluster.cc:1428] Setting key 28d51c4f1412f8f14ff14819517e1c3f
I20260923 22:12:25.110697 2751 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 22:12:25.101724 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.135.194:51299 (local address 127.1.135.254:40067)
0923 22:12:25.101796 (+ 72us) server_negotiation.cc:206] Beginning negotiation
0923 22:12:25.101798 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0923 22:12:25.102044 (+ 246us) server_negotiation.cc:407] Connection header received
0923 22:12:25.102462 (+ 418us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 22:12:25.102465 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 22:12:25.102479 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 22:12:25.102572 (+ 93us) server_negotiation.cc:226] Negotiated authn=SASL
0923 22:12:25.103146 (+ 574us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 22:12:25.103909 (+ 763us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 22:12:25.104988 (+ 1079us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 22:12:25.105197 (+ 209us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 22:12:25.105280 (+ 83us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 22:12:25.107747 (+ 2467us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0923 22:12:25.107748 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0923 22:12:25.107749 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0923 22:12:25.107756 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0923 22:12:25.109360 (+ 1604us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0923 22:12:25.109752 (+ 392us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0923 22:12:25.109753 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0923 22:12:25.109754 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0923 22:12:25.109813 (+ 59us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0923 22:12:25.110054 (+ 241us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0923 22:12:25.110055 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0923 22:12:25.110055 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0923 22:12:25.110168 (+ 113us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0923 22:12:25.110241 (+ 73us) server_negotiation.cc:1035] Waiting for connection context
0923 22:12:25.110520 (+ 279us) server_negotiation.cc:299] Negotiation successful
0923 22:12:25.110608 (+ 88us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":41}
I20260923 22:12:25.110864 2919 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 22:12:25.101733 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.135.254:40067 (local address 127.1.135.194:51299)
0923 22:12:25.101898 (+ 165us) negotiation.cc:107] Waiting for socket to connect
0923 22:12:25.101935 (+ 37us) client_negotiation.cc:174] Beginning negotiation
0923 22:12:25.102364 (+ 429us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0923 22:12:25.102630 (+ 266us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0923 22:12:25.102633 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0923 22:12:25.102837 (+ 204us) client_negotiation.cc:189] Negotiated authn=SASL
0923 22:12:25.103045 (+ 208us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 22:12:25.103051 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 22:12:25.104019 (+ 968us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 22:12:25.104020 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 22:12:25.104876 (+ 856us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 22:12:25.104878 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 22:12:25.105288 (+ 410us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 22:12:25.105290 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 22:12:25.105405 (+ 115us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 22:12:25.105886 (+ 481us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0923 22:12:25.105900 (+ 14us) client_negotiation.cc:656] calling sasl_client_start()
0923 22:12:25.107656 (+ 1756us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0923 22:12:25.109449 (+ 1793us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0923 22:12:25.109454 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0923 22:12:25.109456 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0923 22:12:25.109665 (+ 209us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0923 22:12:25.109875 (+ 210us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0923 22:12:25.109876 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0923 22:12:25.109877 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0923 22:12:25.109986 (+ 109us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0923 22:12:25.110232 (+ 246us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0923 22:12:25.110237 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0923 22:12:25.110380 (+ 143us) client_negotiation.cc:769] Sending connection context
0923 22:12:25.110496 (+ 116us) client_negotiation.cc:240] Negotiation successful
0923 22:12:25.110640 (+ 144us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":125,"thread_start_us":98,"threads_started":1}
I20260923 22:12:25.111260 3026 heartbeater.cc:347] Connected to a master server at 127.1.135.254:40067
I20260923 22:12:25.111344 3026 heartbeater.cc:464] Registering TS with master...
I20260923 22:12:25.111515 3026 heartbeater.cc:511] Master 127.1.135.254:40067 requested a full tablet report, sending...
I20260923 22:12:25.111857 2682 ts_manager.cc:194] Registered new tserver with Master: a4124d4f2ad348d09b2f8903f9f35ba5 (127.1.135.194:36581)
I20260923 22:12:25.112376 2682 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.1.135.194@KRBTEST.COM'} at 127.1.135.194:51299
I20260923 22:12:25.118853 3032 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 22:12:25.112236 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.135.254:40067 (local address 127.1.135.194:59989)
0923 22:12:25.113012 (+ 776us) negotiation.cc:107] Waiting for socket to connect
0923 22:12:25.113023 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0923 22:12:25.113097 (+ 74us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0923 22:12:25.113217 (+ 120us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0923 22:12:25.113218 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0923 22:12:25.113415 (+ 197us) client_negotiation.cc:189] Negotiated authn=SASL
0923 22:12:25.113490 (+ 75us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 22:12:25.113491 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 22:12:25.114132 (+ 641us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 22:12:25.114133 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 22:12:25.115050 (+ 917us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 22:12:25.115052 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 22:12:25.115387 (+ 335us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 22:12:25.115388 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 22:12:25.115475 (+ 87us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 22:12:25.115933 (+ 458us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0923 22:12:25.115938 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0923 22:12:25.116264 (+ 326us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0923 22:12:25.117910 (+ 1646us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0923 22:12:25.117911 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0923 22:12:25.117912 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0923 22:12:25.118061 (+ 149us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0923 22:12:25.118267 (+ 206us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0923 22:12:25.118268 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0923 22:12:25.118268 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0923 22:12:25.118315 (+ 47us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0923 22:12:25.118642 (+ 327us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0923 22:12:25.118642 (+ 0us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0923 22:12:25.118675 (+ 33us) client_negotiation.cc:769] Sending connection context
0923 22:12:25.118725 (+ 50us) client_negotiation.cc:240] Negotiation successful
0923 22:12:25.118793 (+ 68us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":752,"thread_start_us":63,"threads_started":1}
I20260923 22:12:25.118949 2751 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 22:12:25.112302 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.135.194:59989 (local address 127.1.135.254:40067)
0923 22:12:25.112949 (+ 647us) server_negotiation.cc:206] Beginning negotiation
0923 22:12:25.112951 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0923 22:12:25.113075 (+ 124us) server_negotiation.cc:407] Connection header received
0923 22:12:25.113150 (+ 75us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 22:12:25.113151 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 22:12:25.113160 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 22:12:25.113204 (+ 44us) server_negotiation.cc:226] Negotiated authn=SASL
0923 22:12:25.113554 (+ 350us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 22:12:25.114039 (+ 485us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 22:12:25.115134 (+ 1095us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 22:12:25.115314 (+ 180us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 22:12:25.115366 (+ 52us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 22:12:25.116348 (+ 982us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0923 22:12:25.116350 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0923 22:12:25.116350 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0923 22:12:25.116357 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0923 22:12:25.117818 (+ 1461us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0923 22:12:25.118144 (+ 326us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0923 22:12:25.118145 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0923 22:12:25.118145 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0923 22:12:25.118201 (+ 56us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0923 22:12:25.118371 (+ 170us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0923 22:12:25.118372 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0923 22:12:25.118372 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0923 22:12:25.118576 (+ 204us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0923 22:12:25.118649 (+ 73us) server_negotiation.cc:1035] Waiting for connection context
0923 22:12:25.118754 (+ 105us) server_negotiation.cc:299] Negotiation successful
0923 22:12:25.118820 (+ 66us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":625}
WARNING: no policy specified for kudu/127.1.135.195@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.1.135.195@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.1.135.195 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskRiQe_a/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790201539513534-1567-0/krb5kdc/kudu_127.1.135.195.keytab.
Entry for principal kudu/127.1.135.195 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskRiQe_a/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790201539513534-1567-0/krb5kdc/kudu_127.1.135.195.keytab.
I20260923 22:12:25.136668 1567 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskRiQe_a/build/release/bin/kudu
/tmp/dist-test-taskRiQe_a/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskRiQe_a/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790201539513534-1567-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskRiQe_a/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790201539513534-1567-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskRiQe_a/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790201539513534-1567-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskRiQe_a/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790201539513534-1567-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.1.135.195:0
--local_ip_for_outbound_sockets=127.1.135.195
--webserver_interface=127.1.135.195
--webserver_port=0
--tserver_master_addrs=127.1.135.254:40067
--keytab_file=/tmp/dist-test-taskRiQe_a/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790201539513534-1567-0/krb5kdc/kudu_127.1.135.195.keytab
--principal=kudu/127.1.135.195
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.1.135.212:42617
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--enable_txn_system_client_init=true with env {KRB5CCNAME=/tmp/dist-test-taskRiQe_a/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790201539513534-1567-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskRiQe_a/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790201539513534-1567-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskRiQe_a/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790201539513534-1567-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260923 22:12:25.219520 3035 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260923 22:12:25.219705 3035 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260923 22:12:25.219728 3035 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260923 22:12:25.220950 3035 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260923 22:12:25.220991 3035 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260923 22:12:25.221012 3035 flags.cc:425] Enabled experimental flag: --enable_txn_system_client_init=true
W20260923 22:12:25.221071 3035 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.1.135.195
I20260923 22:12:25.222441 3035 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.1.135.212:42617
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskRiQe_a/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790201539513534-1567-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskRiQe_a/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790201539513534-1567-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskRiQe_a/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790201539513534-1567-0/krb5kdc/kudu_127.1.135.195.keytab
--principal=kudu/127.1.135.195
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.1.135.195:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskRiQe_a/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790201539513534-1567-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.1.135.195
--webserver_port=0
--enable_txn_system_client_init=true
--tserver_master_addrs=127.1.135.254:40067
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.3035
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.1.135.195
--log_dir=/tmp/dist-test-taskRiQe_a/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790201539513534-1567-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 23 Sep 2026 20:53:37 UTC on ccycloud.quasar-rkviyx.root.comops.site
build id 4668083
I20260923 22:12:25.222684 3035 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260923 22:12:25.222895 3035 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260923 22:12:25.227573 3041 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260923 22:12:25.227633 3035 server_base.cc:1034] running on GCE node
W20260923 22:12:25.227608 3043 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260923 22:12:25.227574 3040 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260923 22:12:25.228226 3035 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260923 22:12:25.228479 3035 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260923 22:12:25.229619 3035 hybrid_clock.cc:648] HybridClock initialized: now 1790201545229608 us; error 26 us; skew 500 ppm
Sep 23 22:12:25 dist-test-slave-s2fj krb5kdc[2638](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790201545, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.135.195@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260923 22:12:25.231963 3035 init.cc:377] Logged in from keytab as kudu/127.1.135.195@KRBTEST.COM (short username kudu)
I20260923 22:12:25.232415 3035 webserver.cc:460] Webserver started at http://127.1.135.195:39421/ using document root <none> and password file <none>
I20260923 22:12:25.232625 3035 fs_manager.cc:359] Metadata directory not provided
I20260923 22:12:25.232693 3035 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260923 22:12:25.232822 3035 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260923 22:12:25.233788 3035 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskRiQe_a/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790201539513534-1567-0/minicluster-data/ts-2/data/instance:
uuid: "e067e9390e544580989450a94afe6ca4"
format_stamp: "Formatted at 2026-09-23 22:12:25 on dist-test-slave-s2fj"
server_key: "94cf26efcf13d933dee7429a0e94d18d"
server_key_iv: "117b9f340fbe86ba9ad88e6038d6f5d1"
server_key_version: "encryptionkey@0"
I20260923 22:12:25.234153 3035 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskRiQe_a/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790201539513534-1567-0/minicluster-data/ts-2/wal/instance:
uuid: "e067e9390e544580989450a94afe6ca4"
format_stamp: "Formatted at 2026-09-23 22:12:25 on dist-test-slave-s2fj"
server_key: "94cf26efcf13d933dee7429a0e94d18d"
server_key_iv: "117b9f340fbe86ba9ad88e6038d6f5d1"
server_key_version: "encryptionkey@0"
I20260923 22:12:25.235419 3035 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.001s sys 0.002s
I20260923 22:12:25.236097 3050 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260923 22:12:25.236267 3035 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260923 22:12:25.236369 3035 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskRiQe_a/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790201539513534-1567-0/minicluster-data/ts-2/data,/tmp/dist-test-taskRiQe_a/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790201539513534-1567-0/minicluster-data/ts-2/wal
uuid: "e067e9390e544580989450a94afe6ca4"
format_stamp: "Formatted at 2026-09-23 22:12:25 on dist-test-slave-s2fj"
server_key: "94cf26efcf13d933dee7429a0e94d18d"
server_key_iv: "117b9f340fbe86ba9ad88e6038d6f5d1"
server_key_version: "encryptionkey@0"
I20260923 22:12:25.236459 3035 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskRiQe_a/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790201539513534-1567-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskRiQe_a/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790201539513534-1567-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskRiQe_a/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790201539513534-1567-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260923 22:12:25.257508 3035 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260923 22:12:25.259727 3035 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260923 22:12:25.259879 3035 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260923 22:12:25.260404 3035 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260923 22:12:25.260463 3035 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260923 22:12:25.260509 3035 ts_tablet_manager.cc:610] Registered 0 tablets
I20260923 22:12:25.260535 3035 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260923 22:12:25.267037 3035 rpc_server.cc:307] RPC server started. Bound to: 127.1.135.195:44437
I20260923 22:12:25.267109 3165 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.1.135.195:44437 every 8 connection(s)
Sep 23 22:12:25 dist-test-slave-s2fj krb5kdc[2638](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790201545, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.135.195@KRBTEST.COM for kudu/127.1.135.254@KRBTEST.COM
I20260923 22:12:25.267426 3035 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskRiQe_a/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790201539513534-1567-0/minicluster-data/ts-2/data/info.pb
I20260923 22:12:25.270417 2751 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 22:12:25.260805 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.135.195:60567 (local address 127.1.135.254:40067)
0923 22:12:25.260864 (+ 59us) server_negotiation.cc:206] Beginning negotiation
0923 22:12:25.260866 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0923 22:12:25.261096 (+ 230us) server_negotiation.cc:407] Connection header received
0923 22:12:25.261580 (+ 484us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 22:12:25.261581 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 22:12:25.261595 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 22:12:25.261659 (+ 64us) server_negotiation.cc:226] Negotiated authn=SASL
0923 22:12:25.262503 (+ 844us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 22:12:25.263358 (+ 855us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 22:12:25.264938 (+ 1580us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 22:12:25.265259 (+ 321us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 22:12:25.265342 (+ 83us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 22:12:25.267872 (+ 2530us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0923 22:12:25.267874 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0923 22:12:25.267874 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0923 22:12:25.267881 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0923 22:12:25.269264 (+ 1383us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0923 22:12:25.269640 (+ 376us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0923 22:12:25.269641 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0923 22:12:25.269641 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0923 22:12:25.269702 (+ 61us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0923 22:12:25.269918 (+ 216us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0923 22:12:25.269919 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0923 22:12:25.269920 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0923 22:12:25.270028 (+ 108us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0923 22:12:25.270090 (+ 62us) server_negotiation.cc:1035] Waiting for connection context
0923 22:12:25.270276 (+ 186us) server_negotiation.cc:299] Negotiation successful
0923 22:12:25.270331 (+ 55us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":17}
I20260923 22:12:25.270570 3059 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 22:12:25.260830 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.135.254:40067 (local address 127.1.135.195:60567)
0923 22:12:25.260983 (+ 153us) negotiation.cc:107] Waiting for socket to connect
0923 22:12:25.261014 (+ 31us) client_negotiation.cc:174] Beginning negotiation
0923 22:12:25.261479 (+ 465us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0923 22:12:25.261755 (+ 276us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0923 22:12:25.261758 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0923 22:12:25.262127 (+ 369us) client_negotiation.cc:189] Negotiated authn=SASL
0923 22:12:25.262376 (+ 249us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 22:12:25.262381 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 22:12:25.263468 (+ 1087us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 22:12:25.263469 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 22:12:25.264832 (+ 1363us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 22:12:25.264835 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 22:12:25.265342 (+ 507us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 22:12:25.265343 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 22:12:25.265446 (+ 103us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 22:12:25.265924 (+ 478us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0923 22:12:25.265933 (+ 9us) client_negotiation.cc:656] calling sasl_client_start()
0923 22:12:25.267777 (+ 1844us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0923 22:12:25.269347 (+ 1570us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0923 22:12:25.269351 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0923 22:12:25.269353 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0923 22:12:25.269560 (+ 207us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0923 22:12:25.269759 (+ 199us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0923 22:12:25.269759 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0923 22:12:25.269760 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0923 22:12:25.269843 (+ 83us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0923 22:12:25.270090 (+ 247us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0923 22:12:25.270093 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0923 22:12:25.270189 (+ 96us) client_negotiation.cc:769] Sending connection context
0923 22:12:25.270261 (+ 72us) client_negotiation.cc:240] Negotiation successful
0923 22:12:25.270361 (+ 100us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":101,"thread_start_us":80,"threads_started":1}
I20260923 22:12:25.270900 1567 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskRiQe_a/build/release/bin/kudu as pid 3035
I20260923 22:12:25.271006 1567 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskRiQe_a/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790201539513534-1567-0/minicluster-data/ts-2/wal/instance
I20260923 22:12:25.271181 1567 external_mini_cluster.cc:1428] Setting key bee50cc5e539f319f4cd68b024befba7
I20260923 22:12:25.271579 3166 heartbeater.cc:347] Connected to a master server at 127.1.135.254:40067
I20260923 22:12:25.271661 3166 heartbeater.cc:464] Registering TS with master...
I20260923 22:12:25.271829 3166 heartbeater.cc:511] Master 127.1.135.254:40067 requested a full tablet report, sending...
I20260923 22:12:25.272150 2682 ts_manager.cc:194] Registered new tserver with Master: e067e9390e544580989450a94afe6ca4 (127.1.135.195:44437)
I20260923 22:12:25.272506 2682 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.1.135.195@KRBTEST.COM'} at 127.1.135.195:60567
I20260923 22:12:25.277372 2751 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 22:12:25.272587 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.135.195:44681 (local address 127.1.135.254:40067)
0923 22:12:25.272625 (+ 38us) server_negotiation.cc:206] Beginning negotiation
0923 22:12:25.272626 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0923 22:12:25.272744 (+ 118us) server_negotiation.cc:407] Connection header received
0923 22:12:25.272797 (+ 53us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 22:12:25.272798 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 22:12:25.272808 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 22:12:25.272855 (+ 47us) server_negotiation.cc:226] Negotiated authn=SASL
0923 22:12:25.273146 (+ 291us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 22:12:25.273635 (+ 489us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 22:12:25.274249 (+ 614us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 22:12:25.274425 (+ 176us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 22:12:25.274471 (+ 46us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 22:12:25.275158 (+ 687us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0923 22:12:25.275158 (+ 0us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0923 22:12:25.275159 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0923 22:12:25.275164 (+ 5us) server_negotiation.cc:922] calling sasl_server_start()
0923 22:12:25.276509 (+ 1345us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0923 22:12:25.276810 (+ 301us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0923 22:12:25.276811 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0923 22:12:25.276811 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0923 22:12:25.276854 (+ 43us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0923 22:12:25.277034 (+ 180us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0923 22:12:25.277034 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0923 22:12:25.277034 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0923 22:12:25.277117 (+ 83us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0923 22:12:25.277163 (+ 46us) server_negotiation.cc:1035] Waiting for connection context
0923 22:12:25.277269 (+ 106us) server_negotiation.cc:299] Negotiation successful
0923 22:12:25.277318 (+ 49us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":17}
I20260923 22:12:25.277374 3172 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 22:12:25.272536 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.135.254:40067 (local address 127.1.135.195:44681)
0923 22:12:25.272672 (+ 136us) negotiation.cc:107] Waiting for socket to connect
0923 22:12:25.272682 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0923 22:12:25.272738 (+ 56us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0923 22:12:25.272871 (+ 133us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0923 22:12:25.272872 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0923 22:12:25.273032 (+ 160us) client_negotiation.cc:189] Negotiated authn=SASL
0923 22:12:25.273075 (+ 43us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 22:12:25.273076 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 22:12:25.273716 (+ 640us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 22:12:25.273717 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 22:12:25.274182 (+ 465us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 22:12:25.274183 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 22:12:25.274484 (+ 301us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 22:12:25.274485 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 22:12:25.274539 (+ 54us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 22:12:25.274848 (+ 309us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0923 22:12:25.274851 (+ 3us) client_negotiation.cc:656] calling sasl_client_start()
0923 22:12:25.275085 (+ 234us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0923 22:12:25.276583 (+ 1498us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0923 22:12:25.276584 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0923 22:12:25.276584 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0923 22:12:25.276751 (+ 167us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0923 22:12:25.276906 (+ 155us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0923 22:12:25.276907 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0923 22:12:25.276907 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0923 22:12:25.276953 (+ 46us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0923 22:12:25.277175 (+ 222us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0923 22:12:25.277175 (+ 0us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0923 22:12:25.277206 (+ 31us) client_negotiation.cc:769] Sending connection context
0923 22:12:25.277255 (+ 49us) client_negotiation.cc:240] Negotiation successful
0923 22:12:25.277320 (+ 65us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":115,"thread_start_us":79,"threads_started":1}
I20260923 22:12:25.282755 1567 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
Sep 23 22:12:25 dist-test-slave-s2fj krb5kdc[2638](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790201545, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-user@KRBTEST.COM:
Sep 23 22:12:25 dist-test-slave-s2fj krb5kdc[2638](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790201545, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for kudu/127.1.135.254@KRBTEST.COM
I20260923 22:12:25.313406 2751 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 22:12:25.306196 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56362 (local address 127.1.135.254:40067)
0923 22:12:25.306273 (+ 77us) server_negotiation.cc:206] Beginning negotiation
0923 22:12:25.306275 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0923 22:12:25.306285 (+ 10us) server_negotiation.cc:407] Connection header received
0923 22:12:25.306310 (+ 25us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 22:12:25.306310 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 22:12:25.306321 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 22:12:25.306429 (+ 108us) server_negotiation.cc:226] Negotiated authn=SASL
0923 22:12:25.307250 (+ 821us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 22:12:25.307833 (+ 583us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 22:12:25.308810 (+ 977us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 22:12:25.309017 (+ 207us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 22:12:25.309117 (+ 100us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 22:12:25.311123 (+ 2006us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0923 22:12:25.311124 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0923 22:12:25.311125 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0923 22:12:25.311132 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0923 22:12:25.312329 (+ 1197us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0923 22:12:25.312677 (+ 348us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0923 22:12:25.312678 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0923 22:12:25.312679 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0923 22:12:25.312734 (+ 55us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0923 22:12:25.312945 (+ 211us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0923 22:12:25.312946 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0923 22:12:25.312946 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0923 22:12:25.313044 (+ 98us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0923 22:12:25.313142 (+ 98us) server_negotiation.cc:1035] Waiting for connection context
0923 22:12:25.313239 (+ 97us) server_negotiation.cc:299] Negotiation successful
0923 22:12:25.313309 (+ 70us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":27}
I20260923 22:12:25.315032 2682 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:56362:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260923 22:12:25.316677 2682 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260923 22:12:25.322860 3184 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 22:12:25.318849 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.135.195:44437 (local address 127.0.0.1:35644)
0923 22:12:25.319336 (+ 487us) negotiation.cc:107] Waiting for socket to connect
0923 22:12:25.319346 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0923 22:12:25.319397 (+ 51us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0923 22:12:25.319633 (+ 236us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0923 22:12:25.319634 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0923 22:12:25.319636 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0923 22:12:25.319682 (+ 46us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 22:12:25.319684 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 22:12:25.320871 (+ 1187us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 22:12:25.320872 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 22:12:25.321705 (+ 833us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 22:12:25.321707 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 22:12:25.322550 (+ 843us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 22:12:25.322551 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 22:12:25.322627 (+ 76us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 22:12:25.322639 (+ 12us) client_negotiation.cc:769] Sending connection context
0923 22:12:25.322735 (+ 96us) client_negotiation.cc:240] Negotiation successful
0923 22:12:25.322784 (+ 49us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":468,"mutex_wait_us":31,"thread_start_us":73,"threads_started":1}
I20260923 22:12:25.322867 3187 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 22:12:25.319047 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35644 (local address 127.1.135.195:44437)
0923 22:12:25.319286 (+ 239us) server_negotiation.cc:206] Beginning negotiation
0923 22:12:25.319289 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0923 22:12:25.319405 (+ 116us) server_negotiation.cc:407] Connection header received
0923 22:12:25.319454 (+ 49us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 22:12:25.319456 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 22:12:25.319471 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 22:12:25.319556 (+ 85us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0923 22:12:25.319755 (+ 199us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 22:12:25.320777 (+ 1022us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 22:12:25.321794 (+ 1017us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 22:12:25.322456 (+ 662us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 22:12:25.322547 (+ 91us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 22:12:25.322602 (+ 55us) server_negotiation.cc:1035] Waiting for connection context
0923 22:12:25.322751 (+ 149us) server_negotiation.cc:299] Negotiation successful
0923 22:12:25.322783 (+ 32us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":216,"thread_start_us":66,"threads_started":1}
I20260923 22:12:25.323174 3185 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 22:12:25.318866 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.135.194:36581 (local address 127.0.0.1:47168)
0923 22:12:25.319482 (+ 616us) negotiation.cc:107] Waiting for socket to connect
0923 22:12:25.319489 (+ 7us) client_negotiation.cc:174] Beginning negotiation
0923 22:12:25.319582 (+ 93us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0923 22:12:25.319772 (+ 190us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0923 22:12:25.319773 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0923 22:12:25.319774 (+ 1us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0923 22:12:25.319810 (+ 36us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 22:12:25.319811 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 22:12:25.320898 (+ 1087us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 22:12:25.320899 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 22:12:25.322023 (+ 1124us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 22:12:25.322025 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 22:12:25.322953 (+ 928us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 22:12:25.322954 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 22:12:25.323011 (+ 57us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 22:12:25.323015 (+ 4us) client_negotiation.cc:769] Sending connection context
0923 22:12:25.323067 (+ 52us) client_negotiation.cc:240] Negotiation successful
0923 22:12:25.323103 (+ 36us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":596,"thread_start_us":81,"threads_started":1}
I20260923 22:12:25.323236 3188 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 22:12:25.319075 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47168 (local address 127.1.135.194:36581)
0923 22:12:25.319231 (+ 156us) server_negotiation.cc:206] Beginning negotiation
0923 22:12:25.319234 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0923 22:12:25.319551 (+ 317us) server_negotiation.cc:407] Connection header received
0923 22:12:25.319658 (+ 107us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 22:12:25.319661 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 22:12:25.319676 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 22:12:25.319774 (+ 98us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0923 22:12:25.319871 (+ 97us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 22:12:25.320808 (+ 937us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 22:12:25.322105 (+ 1297us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 22:12:25.322792 (+ 687us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 22:12:25.322859 (+ 67us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 22:12:25.322908 (+ 49us) server_negotiation.cc:1035] Waiting for connection context
0923 22:12:25.323102 (+ 194us) server_negotiation.cc:299] Negotiation successful
0923 22:12:25.323146 (+ 44us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":132,"thread_start_us":66,"threads_started":1}
I20260923 22:12:25.323316 3183 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 22:12:25.318846 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.135.193:44317 (local address 127.0.0.1:42500)
0923 22:12:25.318988 (+ 142us) negotiation.cc:107] Waiting for socket to connect
0923 22:12:25.319025 (+ 37us) client_negotiation.cc:174] Beginning negotiation
0923 22:12:25.319087 (+ 62us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0923 22:12:25.319267 (+ 180us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0923 22:12:25.319270 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0923 22:12:25.319271 (+ 1us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0923 22:12:25.319329 (+ 58us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 22:12:25.319331 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 22:12:25.320687 (+ 1356us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 22:12:25.320688 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 22:12:25.321996 (+ 1308us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 22:12:25.321999 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 22:12:25.323097 (+ 1098us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 22:12:25.323098 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 22:12:25.323158 (+ 60us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 22:12:25.323161 (+ 3us) client_negotiation.cc:769] Sending connection context
0923 22:12:25.323206 (+ 45us) client_negotiation.cc:240] Negotiation successful
0923 22:12:25.323245 (+ 39us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":119,"mutex_wait_us":25,"thread_start_us":37,"threads_started":1}
I20260923 22:12:25.323535 3186 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 22:12:25.318934 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42500 (local address 127.1.135.193:44317)
0923 22:12:25.319109 (+ 175us) server_negotiation.cc:206] Beginning negotiation
0923 22:12:25.319114 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0923 22:12:25.319127 (+ 13us) server_negotiation.cc:407] Connection header received
0923 22:12:25.319166 (+ 39us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 22:12:25.319168 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 22:12:25.319183 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 22:12:25.319256 (+ 73us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0923 22:12:25.319432 (+ 176us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 22:12:25.320604 (+ 1172us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 22:12:25.322098 (+ 1494us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 22:12:25.322792 (+ 694us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 22:12:25.322859 (+ 67us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 22:12:25.322908 (+ 49us) server_negotiation.cc:1035] Waiting for connection context
0923 22:12:25.323404 (+ 496us) server_negotiation.cc:299] Negotiation successful
0923 22:12:25.323449 (+ 45us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":147,"thread_start_us":72,"threads_started":1}
I20260923 22:12:25.323690 3100 tablet_service.cc:1467] Processing CreateTablet for tablet 81c4606146c44f60b51e88b7a5ad31ae (DEFAULT_TABLE table=test-table [id=ea683e7d69a44f27919f313c1d8be96d]), partition=RANGE (key) PARTITION UNBOUNDED
I20260923 22:12:25.323699 2960 tablet_service.cc:1467] Processing CreateTablet for tablet 81c4606146c44f60b51e88b7a5ad31ae (DEFAULT_TABLE table=test-table [id=ea683e7d69a44f27919f313c1d8be96d]), partition=RANGE (key) PARTITION UNBOUNDED
I20260923 22:12:25.324015 3100 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 81c4606146c44f60b51e88b7a5ad31ae. 1 dirs total, 0 dirs full, 0 dirs failed
I20260923 22:12:25.324015 2960 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 81c4606146c44f60b51e88b7a5ad31ae. 1 dirs total, 0 dirs full, 0 dirs failed
I20260923 22:12:25.324045 2820 tablet_service.cc:1467] Processing CreateTablet for tablet 81c4606146c44f60b51e88b7a5ad31ae (DEFAULT_TABLE table=test-table [id=ea683e7d69a44f27919f313c1d8be96d]), partition=RANGE (key) PARTITION UNBOUNDED
I20260923 22:12:25.324389 2820 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 81c4606146c44f60b51e88b7a5ad31ae. 1 dirs total, 0 dirs full, 0 dirs failed
I20260923 22:12:25.327105 3189 tablet_bootstrap.cc:492] T 81c4606146c44f60b51e88b7a5ad31ae P 88a6b9e139ed41b49b48710b7d8658de: Bootstrap starting.
I20260923 22:12:25.327605 3190 tablet_bootstrap.cc:492] T 81c4606146c44f60b51e88b7a5ad31ae P e067e9390e544580989450a94afe6ca4: Bootstrap starting.
I20260923 22:12:25.327740 3189 tablet_bootstrap.cc:654] T 81c4606146c44f60b51e88b7a5ad31ae P 88a6b9e139ed41b49b48710b7d8658de: Neither blocks nor log segments found. Creating new log.
I20260923 22:12:25.327934 3191 tablet_bootstrap.cc:492] T 81c4606146c44f60b51e88b7a5ad31ae P a4124d4f2ad348d09b2f8903f9f35ba5: Bootstrap starting.
I20260923 22:12:25.328176 3189 log.cc:824] T 81c4606146c44f60b51e88b7a5ad31ae P 88a6b9e139ed41b49b48710b7d8658de: Log is configured to *not* fsync() on all Append() calls
I20260923 22:12:25.328599 3191 tablet_bootstrap.cc:654] T 81c4606146c44f60b51e88b7a5ad31ae P a4124d4f2ad348d09b2f8903f9f35ba5: Neither blocks nor log segments found. Creating new log.
I20260923 22:12:25.328593 3190 tablet_bootstrap.cc:654] T 81c4606146c44f60b51e88b7a5ad31ae P e067e9390e544580989450a94afe6ca4: Neither blocks nor log segments found. Creating new log.
I20260923 22:12:25.328884 3191 log.cc:824] T 81c4606146c44f60b51e88b7a5ad31ae P a4124d4f2ad348d09b2f8903f9f35ba5: Log is configured to *not* fsync() on all Append() calls
I20260923 22:12:25.328912 3190 log.cc:824] T 81c4606146c44f60b51e88b7a5ad31ae P e067e9390e544580989450a94afe6ca4: Log is configured to *not* fsync() on all Append() calls
I20260923 22:12:25.328966 3189 tablet_bootstrap.cc:492] T 81c4606146c44f60b51e88b7a5ad31ae P 88a6b9e139ed41b49b48710b7d8658de: No bootstrap required, opened a new log
I20260923 22:12:25.329046 3189 ts_tablet_manager.cc:1397] T 81c4606146c44f60b51e88b7a5ad31ae P 88a6b9e139ed41b49b48710b7d8658de: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20260923 22:12:25.329559 3191 tablet_bootstrap.cc:492] T 81c4606146c44f60b51e88b7a5ad31ae P a4124d4f2ad348d09b2f8903f9f35ba5: No bootstrap required, opened a new log
I20260923 22:12:25.329609 3190 tablet_bootstrap.cc:492] T 81c4606146c44f60b51e88b7a5ad31ae P e067e9390e544580989450a94afe6ca4: No bootstrap required, opened a new log
I20260923 22:12:25.329639 3191 ts_tablet_manager.cc:1397] T 81c4606146c44f60b51e88b7a5ad31ae P a4124d4f2ad348d09b2f8903f9f35ba5: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20260923 22:12:25.329672 3190 ts_tablet_manager.cc:1397] T 81c4606146c44f60b51e88b7a5ad31ae P e067e9390e544580989450a94afe6ca4: Time spent bootstrapping tablet: real 0.002s user 0.002s sys 0.000s
I20260923 22:12:25.331037 3189 raft_consensus.cc:348] T 81c4606146c44f60b51e88b7a5ad31ae P 88a6b9e139ed41b49b48710b7d8658de [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e067e9390e544580989450a94afe6ca4" member_type: VOTER last_known_addr { host: "127.1.135.195" port: 44437 } } peers { permanent_uuid: "88a6b9e139ed41b49b48710b7d8658de" member_type: VOTER last_known_addr { host: "127.1.135.193" port: 44317 } } peers { permanent_uuid: "a4124d4f2ad348d09b2f8903f9f35ba5" member_type: VOTER last_known_addr { host: "127.1.135.194" port: 36581 } }
I20260923 22:12:25.331199 3189 raft_consensus.cc:374] T 81c4606146c44f60b51e88b7a5ad31ae P 88a6b9e139ed41b49b48710b7d8658de [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260923 22:12:25.331120 3191 raft_consensus.cc:348] T 81c4606146c44f60b51e88b7a5ad31ae P a4124d4f2ad348d09b2f8903f9f35ba5 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e067e9390e544580989450a94afe6ca4" member_type: VOTER last_known_addr { host: "127.1.135.195" port: 44437 } } peers { permanent_uuid: "88a6b9e139ed41b49b48710b7d8658de" member_type: VOTER last_known_addr { host: "127.1.135.193" port: 44317 } } peers { permanent_uuid: "a4124d4f2ad348d09b2f8903f9f35ba5" member_type: VOTER last_known_addr { host: "127.1.135.194" port: 36581 } }
I20260923 22:12:25.331234 3189 raft_consensus.cc:729] T 81c4606146c44f60b51e88b7a5ad31ae P 88a6b9e139ed41b49b48710b7d8658de [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 88a6b9e139ed41b49b48710b7d8658de, State: Initialized, Role: FOLLOWER
I20260923 22:12:25.331269 3191 raft_consensus.cc:374] T 81c4606146c44f60b51e88b7a5ad31ae P a4124d4f2ad348d09b2f8903f9f35ba5 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260923 22:12:25.331301 3191 raft_consensus.cc:729] T 81c4606146c44f60b51e88b7a5ad31ae P a4124d4f2ad348d09b2f8903f9f35ba5 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: a4124d4f2ad348d09b2f8903f9f35ba5, State: Initialized, Role: FOLLOWER
I20260923 22:12:25.331339 3189 consensus_queue.cc:260] T 81c4606146c44f60b51e88b7a5ad31ae P 88a6b9e139ed41b49b48710b7d8658de [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e067e9390e544580989450a94afe6ca4" member_type: VOTER last_known_addr { host: "127.1.135.195" port: 44437 } } peers { permanent_uuid: "88a6b9e139ed41b49b48710b7d8658de" member_type: VOTER last_known_addr { host: "127.1.135.193" port: 44317 } } peers { permanent_uuid: "a4124d4f2ad348d09b2f8903f9f35ba5" member_type: VOTER last_known_addr { host: "127.1.135.194" port: 36581 } }
I20260923 22:12:25.331422 3191 consensus_queue.cc:260] T 81c4606146c44f60b51e88b7a5ad31ae P a4124d4f2ad348d09b2f8903f9f35ba5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e067e9390e544580989450a94afe6ca4" member_type: VOTER last_known_addr { host: "127.1.135.195" port: 44437 } } peers { permanent_uuid: "88a6b9e139ed41b49b48710b7d8658de" member_type: VOTER last_known_addr { host: "127.1.135.193" port: 44317 } } peers { permanent_uuid: "a4124d4f2ad348d09b2f8903f9f35ba5" member_type: VOTER last_known_addr { host: "127.1.135.194" port: 36581 } }
I20260923 22:12:25.331568 3189 ts_tablet_manager.cc:1428] T 81c4606146c44f60b51e88b7a5ad31ae P 88a6b9e139ed41b49b48710b7d8658de: Time spent starting tablet: real 0.002s user 0.003s sys 0.000s
I20260923 22:12:25.331537 3190 raft_consensus.cc:348] T 81c4606146c44f60b51e88b7a5ad31ae P e067e9390e544580989450a94afe6ca4 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e067e9390e544580989450a94afe6ca4" member_type: VOTER last_known_addr { host: "127.1.135.195" port: 44437 } } peers { permanent_uuid: "88a6b9e139ed41b49b48710b7d8658de" member_type: VOTER last_known_addr { host: "127.1.135.193" port: 44317 } } peers { permanent_uuid: "a4124d4f2ad348d09b2f8903f9f35ba5" member_type: VOTER last_known_addr { host: "127.1.135.194" port: 36581 } }
I20260923 22:12:25.331677 2886 heartbeater.cc:503] Master 127.1.135.254:40067 was elected leader, sending a full tablet report...
I20260923 22:12:25.331681 3191 ts_tablet_manager.cc:1428] T 81c4606146c44f60b51e88b7a5ad31ae P a4124d4f2ad348d09b2f8903f9f35ba5: Time spent starting tablet: real 0.002s user 0.002s sys 0.000s
I20260923 22:12:25.331686 3190 raft_consensus.cc:374] T 81c4606146c44f60b51e88b7a5ad31ae P e067e9390e544580989450a94afe6ca4 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260923 22:12:25.331723 3190 raft_consensus.cc:729] T 81c4606146c44f60b51e88b7a5ad31ae P e067e9390e544580989450a94afe6ca4 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e067e9390e544580989450a94afe6ca4, State: Initialized, Role: FOLLOWER
I20260923 22:12:25.331791 3026 heartbeater.cc:503] Master 127.1.135.254:40067 was elected leader, sending a full tablet report...
I20260923 22:12:25.331825 3190 consensus_queue.cc:260] T 81c4606146c44f60b51e88b7a5ad31ae P e067e9390e544580989450a94afe6ca4 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e067e9390e544580989450a94afe6ca4" member_type: VOTER last_known_addr { host: "127.1.135.195" port: 44437 } } peers { permanent_uuid: "88a6b9e139ed41b49b48710b7d8658de" member_type: VOTER last_known_addr { host: "127.1.135.193" port: 44317 } } peers { permanent_uuid: "a4124d4f2ad348d09b2f8903f9f35ba5" member_type: VOTER last_known_addr { host: "127.1.135.194" port: 36581 } }
I20260923 22:12:25.332073 3190 ts_tablet_manager.cc:1428] T 81c4606146c44f60b51e88b7a5ad31ae P e067e9390e544580989450a94afe6ca4: Time spent starting tablet: real 0.002s user 0.003s sys 0.000s
I20260923 22:12:25.332394 3166 heartbeater.cc:503] Master 127.1.135.254:40067 was elected leader, sending a full tablet report...
W20260923 22:12:25.359385 3027 tablet.cc:2369] T 81c4606146c44f60b51e88b7a5ad31ae P a4124d4f2ad348d09b2f8903f9f35ba5: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260923 22:12:25.366082 3196 raft_consensus.cc:482] T 81c4606146c44f60b51e88b7a5ad31ae P a4124d4f2ad348d09b2f8903f9f35ba5 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260923 22:12:25.366178 3196 raft_consensus.cc:504] T 81c4606146c44f60b51e88b7a5ad31ae P a4124d4f2ad348d09b2f8903f9f35ba5 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e067e9390e544580989450a94afe6ca4" member_type: VOTER last_known_addr { host: "127.1.135.195" port: 44437 } } peers { permanent_uuid: "88a6b9e139ed41b49b48710b7d8658de" member_type: VOTER last_known_addr { host: "127.1.135.193" port: 44317 } } peers { permanent_uuid: "a4124d4f2ad348d09b2f8903f9f35ba5" member_type: VOTER last_known_addr { host: "127.1.135.194" port: 36581 } }
I20260923 22:12:25.366554 3196 leader_election.cc:288] T 81c4606146c44f60b51e88b7a5ad31ae P a4124d4f2ad348d09b2f8903f9f35ba5 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers e067e9390e544580989450a94afe6ca4 (127.1.135.195:44437), 88a6b9e139ed41b49b48710b7d8658de (127.1.135.193:44317)
I20260923 22:12:25.369411 2919 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 22:12:25.366724 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.135.195:44437 (local address 127.1.135.194:50257)
0923 22:12:25.366779 (+ 55us) negotiation.cc:107] Waiting for socket to connect
0923 22:12:25.366791 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0923 22:12:25.366833 (+ 42us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0923 22:12:25.366978 (+ 145us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0923 22:12:25.366979 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0923 22:12:25.366981 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0923 22:12:25.367048 (+ 67us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 22:12:25.367050 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 22:12:25.367771 (+ 721us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 22:12:25.367772 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 22:12:25.368647 (+ 875us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 22:12:25.368649 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 22:12:25.369191 (+ 542us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 22:12:25.369192 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 22:12:25.369294 (+ 102us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 22:12:25.369297 (+ 3us) client_negotiation.cc:769] Sending connection context
0923 22:12:25.369339 (+ 42us) client_negotiation.cc:240] Negotiation successful
0923 22:12:25.369365 (+ 26us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":37}
I20260923 22:12:25.369494 3187 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 22:12:25.366811 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.135.194:50257 (local address 127.1.135.195:44437)
0923 22:12:25.366851 (+ 40us) server_negotiation.cc:206] Beginning negotiation
0923 22:12:25.366853 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0923 22:12:25.366862 (+ 9us) server_negotiation.cc:407] Connection header received
0923 22:12:25.366893 (+ 31us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 22:12:25.366894 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 22:12:25.366901 (+ 7us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 22:12:25.366956 (+ 55us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0923 22:12:25.367130 (+ 174us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 22:12:25.367692 (+ 562us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 22:12:25.368737 (+ 1045us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 22:12:25.369098 (+ 361us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 22:12:25.369195 (+ 97us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 22:12:25.369202 (+ 7us) server_negotiation.cc:1035] Waiting for connection context
0923 22:12:25.369362 (+ 160us) server_negotiation.cc:299] Negotiation successful
0923 22:12:25.369413 (+ 51us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":21}
I20260923 22:12:25.369882 3186 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 22:12:25.366805 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.135.194:37897 (local address 127.1.135.193:44317)
0923 22:12:25.366844 (+ 39us) server_negotiation.cc:206] Beginning negotiation
0923 22:12:25.366846 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0923 22:12:25.367012 (+ 166us) server_negotiation.cc:407] Connection header received
0923 22:12:25.367065 (+ 53us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 22:12:25.367066 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 22:12:25.367076 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 22:12:25.367131 (+ 55us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0923 22:12:25.367316 (+ 185us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 22:12:25.367814 (+ 498us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 22:12:25.368745 (+ 931us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 22:12:25.369100 (+ 355us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 22:12:25.369188 (+ 88us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 22:12:25.369196 (+ 8us) server_negotiation.cc:1035] Waiting for connection context
0923 22:12:25.369761 (+ 565us) server_negotiation.cc:299] Negotiation successful
0923 22:12:25.369805 (+ 44us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":22}
I20260923 22:12:25.369920 3120 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "81c4606146c44f60b51e88b7a5ad31ae" candidate_uuid: "a4124d4f2ad348d09b2f8903f9f35ba5" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e067e9390e544580989450a94afe6ca4" is_pre_election: true
I20260923 22:12:25.369918 3198 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 22:12:25.366733 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.135.193:44317 (local address 127.1.135.194:37897)
0923 22:12:25.366924 (+ 191us) negotiation.cc:107] Waiting for socket to connect
0923 22:12:25.366935 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0923 22:12:25.366989 (+ 54us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0923 22:12:25.367195 (+ 206us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0923 22:12:25.367196 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0923 22:12:25.367198 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0923 22:12:25.367256 (+ 58us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 22:12:25.367257 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 22:12:25.367890 (+ 633us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 22:12:25.367891 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 22:12:25.368649 (+ 758us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 22:12:25.368652 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 22:12:25.369196 (+ 544us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 22:12:25.369197 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 22:12:25.369679 (+ 482us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 22:12:25.369684 (+ 5us) client_negotiation.cc:769] Sending connection context
0923 22:12:25.369740 (+ 56us) client_negotiation.cc:240] Negotiation successful
0923 22:12:25.369779 (+ 39us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":177,"mutex_wait_us":3,"thread_start_us":93,"threads_started":1}
I20260923 22:12:25.370052 3120 raft_consensus.cc:2454] T 81c4606146c44f60b51e88b7a5ad31ae P e067e9390e544580989450a94afe6ca4 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a4124d4f2ad348d09b2f8903f9f35ba5 in term 0.
I20260923 22:12:25.370273 2840 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "81c4606146c44f60b51e88b7a5ad31ae" candidate_uuid: "a4124d4f2ad348d09b2f8903f9f35ba5" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "88a6b9e139ed41b49b48710b7d8658de" is_pre_election: true
I20260923 22:12:25.370332 2911 leader_election.cc:302] T 81c4606146c44f60b51e88b7a5ad31ae P a4124d4f2ad348d09b2f8903f9f35ba5 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: a4124d4f2ad348d09b2f8903f9f35ba5, e067e9390e544580989450a94afe6ca4; no voters:
I20260923 22:12:25.370394 2840 raft_consensus.cc:2454] T 81c4606146c44f60b51e88b7a5ad31ae P 88a6b9e139ed41b49b48710b7d8658de [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a4124d4f2ad348d09b2f8903f9f35ba5 in term 0.
I20260923 22:12:25.370445 3196 raft_consensus.cc:2781] T 81c4606146c44f60b51e88b7a5ad31ae P a4124d4f2ad348d09b2f8903f9f35ba5 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260923 22:12:25.370492 3196 raft_consensus.cc:482] T 81c4606146c44f60b51e88b7a5ad31ae P a4124d4f2ad348d09b2f8903f9f35ba5 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260923 22:12:25.370517 3196 raft_consensus.cc:3037] T 81c4606146c44f60b51e88b7a5ad31ae P a4124d4f2ad348d09b2f8903f9f35ba5 [term 0 FOLLOWER]: Advancing to term 1
I20260923 22:12:25.371271 3196 raft_consensus.cc:504] T 81c4606146c44f60b51e88b7a5ad31ae P a4124d4f2ad348d09b2f8903f9f35ba5 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e067e9390e544580989450a94afe6ca4" member_type: VOTER last_known_addr { host: "127.1.135.195" port: 44437 } } peers { permanent_uuid: "88a6b9e139ed41b49b48710b7d8658de" member_type: VOTER last_known_addr { host: "127.1.135.193" port: 44317 } } peers { permanent_uuid: "a4124d4f2ad348d09b2f8903f9f35ba5" member_type: VOTER last_known_addr { host: "127.1.135.194" port: 36581 } }
I20260923 22:12:25.371402 3196 leader_election.cc:288] T 81c4606146c44f60b51e88b7a5ad31ae P a4124d4f2ad348d09b2f8903f9f35ba5 [CANDIDATE]: Term 1 election: Requested vote from peers e067e9390e544580989450a94afe6ca4 (127.1.135.195:44437), 88a6b9e139ed41b49b48710b7d8658de (127.1.135.193:44317)
I20260923 22:12:25.371541 2840 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "81c4606146c44f60b51e88b7a5ad31ae" candidate_uuid: "a4124d4f2ad348d09b2f8903f9f35ba5" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "88a6b9e139ed41b49b48710b7d8658de"
I20260923 22:12:25.371541 3120 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "81c4606146c44f60b51e88b7a5ad31ae" candidate_uuid: "a4124d4f2ad348d09b2f8903f9f35ba5" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e067e9390e544580989450a94afe6ca4"
I20260923 22:12:25.371621 2840 raft_consensus.cc:3037] T 81c4606146c44f60b51e88b7a5ad31ae P 88a6b9e139ed41b49b48710b7d8658de [term 0 FOLLOWER]: Advancing to term 1
I20260923 22:12:25.371621 3120 raft_consensus.cc:3037] T 81c4606146c44f60b51e88b7a5ad31ae P e067e9390e544580989450a94afe6ca4 [term 0 FOLLOWER]: Advancing to term 1
I20260923 22:12:25.372500 2840 raft_consensus.cc:2454] T 81c4606146c44f60b51e88b7a5ad31ae P 88a6b9e139ed41b49b48710b7d8658de [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a4124d4f2ad348d09b2f8903f9f35ba5 in term 1.
I20260923 22:12:25.372500 3120 raft_consensus.cc:2454] T 81c4606146c44f60b51e88b7a5ad31ae P e067e9390e544580989450a94afe6ca4 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a4124d4f2ad348d09b2f8903f9f35ba5 in term 1.
I20260923 22:12:25.372668 2914 leader_election.cc:302] T 81c4606146c44f60b51e88b7a5ad31ae P a4124d4f2ad348d09b2f8903f9f35ba5 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 88a6b9e139ed41b49b48710b7d8658de, a4124d4f2ad348d09b2f8903f9f35ba5; no voters:
I20260923 22:12:25.372754 3196 raft_consensus.cc:2781] T 81c4606146c44f60b51e88b7a5ad31ae P a4124d4f2ad348d09b2f8903f9f35ba5 [term 1 FOLLOWER]: Leader election won for term 1
I20260923 22:12:25.372880 3196 raft_consensus.cc:686] T 81c4606146c44f60b51e88b7a5ad31ae P a4124d4f2ad348d09b2f8903f9f35ba5 [term 1 LEADER]: Becoming Leader. State: Replica: a4124d4f2ad348d09b2f8903f9f35ba5, State: Running, Role: LEADER
I20260923 22:12:25.373009 3196 consensus_queue.cc:237] T 81c4606146c44f60b51e88b7a5ad31ae P a4124d4f2ad348d09b2f8903f9f35ba5 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e067e9390e544580989450a94afe6ca4" member_type: VOTER last_known_addr { host: "127.1.135.195" port: 44437 } } peers { permanent_uuid: "88a6b9e139ed41b49b48710b7d8658de" member_type: VOTER last_known_addr { host: "127.1.135.193" port: 44317 } } peers { permanent_uuid: "a4124d4f2ad348d09b2f8903f9f35ba5" member_type: VOTER last_known_addr { host: "127.1.135.194" port: 36581 } }
I20260923 22:12:25.373764 2681 catalog_manager.cc:5495] T 81c4606146c44f60b51e88b7a5ad31ae P a4124d4f2ad348d09b2f8903f9f35ba5 reported cstate change: term changed from 0 to 1, leader changed from <none> to a4124d4f2ad348d09b2f8903f9f35ba5 (127.1.135.194). New cstate: current_term: 1 leader_uuid: "a4124d4f2ad348d09b2f8903f9f35ba5" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e067e9390e544580989450a94afe6ca4" member_type: VOTER last_known_addr { host: "127.1.135.195" port: 44437 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "88a6b9e139ed41b49b48710b7d8658de" member_type: VOTER last_known_addr { host: "127.1.135.193" port: 44317 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "a4124d4f2ad348d09b2f8903f9f35ba5" member_type: VOTER last_known_addr { host: "127.1.135.194" port: 36581 } health_report { overall_health: HEALTHY } } }
I20260923 22:12:25.383837 3188 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 22:12:25.381512 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47170 (local address 127.1.135.194:36581)
0923 22:12:25.381564 (+ 52us) server_negotiation.cc:206] Beginning negotiation
0923 22:12:25.381566 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0923 22:12:25.381574 (+ 8us) server_negotiation.cc:407] Connection header received
0923 22:12:25.381653 (+ 79us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 22:12:25.381653 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 22:12:25.381668 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 22:12:25.381744 (+ 76us) server_negotiation.cc:226] Negotiated authn=TOKEN
0923 22:12:25.381877 (+ 133us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 22:12:25.382295 (+ 418us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 22:12:25.383035 (+ 740us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 22:12:25.383214 (+ 179us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 22:12:25.383307 (+ 93us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 22:12:25.383502 (+ 195us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0923 22:12:25.383586 (+ 84us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0923 22:12:25.383670 (+ 84us) server_negotiation.cc:1035] Waiting for connection context
0923 22:12:25.383720 (+ 50us) server_negotiation.cc:299] Negotiation successful
0923 22:12:25.383736 (+ 16us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":33}
I20260923 22:12:25.385670 2840 raft_consensus.cc:1261] T 81c4606146c44f60b51e88b7a5ad31ae P 88a6b9e139ed41b49b48710b7d8658de [term 1 FOLLOWER]: Refusing update from remote peer a4124d4f2ad348d09b2f8903f9f35ba5: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260923 22:12:25.385692 3120 raft_consensus.cc:1261] T 81c4606146c44f60b51e88b7a5ad31ae P e067e9390e544580989450a94afe6ca4 [term 1 FOLLOWER]: Refusing update from remote peer a4124d4f2ad348d09b2f8903f9f35ba5: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260923 22:12:25.385991 3199 consensus_queue.cc:1035] T 81c4606146c44f60b51e88b7a5ad31ae P a4124d4f2ad348d09b2f8903f9f35ba5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "88a6b9e139ed41b49b48710b7d8658de" member_type: VOTER last_known_addr { host: "127.1.135.193" port: 44317 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260923 22:12:25.386096 3196 consensus_queue.cc:1035] T 81c4606146c44f60b51e88b7a5ad31ae P a4124d4f2ad348d09b2f8903f9f35ba5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e067e9390e544580989450a94afe6ca4" member_type: VOTER last_known_addr { host: "127.1.135.195" port: 44437 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260923 22:12:25.389606 3204 mvcc.cc:205] Tried to move back new op lower bound from 7332665529897500672 to 7332665529848344576. Current Snapshot: MvccSnapshot[applied={T|T < 7332665529897500672}]
I20260923 22:12:25.391371 2683 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:56362:
table { table_name: "test-table" } modify_external_catalogs: true
I20260923 22:12:25.391446 2683 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:56362:
table { table_name: "test-table" } modify_external_catalogs: true
I20260923 22:12:25.392162 2683 catalog_manager.cc:5782] T 00000000000000000000000000000000 P f63d6b7613c94887b75f55a19fcea635: Sending DeleteTablet for 3 replicas of tablet 81c4606146c44f60b51e88b7a5ad31ae
I20260923 22:12:25.392481 2960 tablet_service.cc:1514] Processing DeleteTablet for tablet 81c4606146c44f60b51e88b7a5ad31ae with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-23 22:12:25 UTC) from {username='kudu', principal='kudu/127.1.135.254@KRBTEST.COM'} at 127.0.0.1:47168
I20260923 22:12:25.392472 2820 tablet_service.cc:1514] Processing DeleteTablet for tablet 81c4606146c44f60b51e88b7a5ad31ae with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-23 22:12:25 UTC) from {username='kudu', principal='kudu/127.1.135.254@KRBTEST.COM'} at 127.0.0.1:42500
I20260923 22:12:25.392468 3100 tablet_service.cc:1514] Processing DeleteTablet for tablet 81c4606146c44f60b51e88b7a5ad31ae with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-23 22:12:25 UTC) from {username='kudu', principal='kudu/127.1.135.254@KRBTEST.COM'} at 127.0.0.1:35644
I20260923 22:12:25.392807 3211 tablet_replica.cc:330] T 81c4606146c44f60b51e88b7a5ad31ae P 88a6b9e139ed41b49b48710b7d8658de: stopping tablet replica
I20260923 22:12:25.392900 3211 raft_consensus.cc:2229] T 81c4606146c44f60b51e88b7a5ad31ae P 88a6b9e139ed41b49b48710b7d8658de [term 1 FOLLOWER]: Raft consensus shutting down.
I20260923 22:12:25.392920 2683 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:56362:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20260923 22:12:25.393004 3211 raft_consensus.cc:2258] T 81c4606146c44f60b51e88b7a5ad31ae P 88a6b9e139ed41b49b48710b7d8658de [term 1 FOLLOWER]: Raft consensus is shut down!
W20260923 22:12:25.393101 2683 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260923 22:12:25.393473 3211 ts_tablet_manager.cc:1905] T 81c4606146c44f60b51e88b7a5ad31ae P 88a6b9e139ed41b49b48710b7d8658de: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260923 22:12:25.394299 2820 tablet_service.cc:1467] Processing CreateTablet for tablet 4ff07aeb859349d49dafb0c1c4d4330e (DEFAULT_TABLE table=test-table [id=82e30eab0daf47e2bcc79e1dc71f9e22]), partition=RANGE (key) PARTITION UNBOUNDED
I20260923 22:12:25.394299 3100 tablet_service.cc:1467] Processing CreateTablet for tablet 4ff07aeb859349d49dafb0c1c4d4330e (DEFAULT_TABLE table=test-table [id=82e30eab0daf47e2bcc79e1dc71f9e22]), partition=RANGE (key) PARTITION UNBOUNDED
I20260923 22:12:25.394399 3100 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 4ff07aeb859349d49dafb0c1c4d4330e. 1 dirs total, 0 dirs full, 0 dirs failed
I20260923 22:12:25.394412 2820 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 4ff07aeb859349d49dafb0c1c4d4330e. 1 dirs total, 0 dirs full, 0 dirs failed
I20260923 22:12:25.395429 3211 ts_tablet_manager.cc:1918] T 81c4606146c44f60b51e88b7a5ad31ae P 88a6b9e139ed41b49b48710b7d8658de: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260923 22:12:25.395499 3211 log.cc:1196] T 81c4606146c44f60b51e88b7a5ad31ae P 88a6b9e139ed41b49b48710b7d8658de: Deleting WAL directory at /tmp/dist-test-taskRiQe_a/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790201539513534-1567-0/minicluster-data/ts-0/wal/wals/81c4606146c44f60b51e88b7a5ad31ae
I20260923 22:12:25.395643 3190 tablet_bootstrap.cc:492] T 4ff07aeb859349d49dafb0c1c4d4330e P e067e9390e544580989450a94afe6ca4: Bootstrap starting.
I20260923 22:12:25.395685 3189 tablet_bootstrap.cc:492] T 4ff07aeb859349d49dafb0c1c4d4330e P 88a6b9e139ed41b49b48710b7d8658de: Bootstrap starting.
I20260923 22:12:25.395740 3211 ts_tablet_manager.cc:1939] T 81c4606146c44f60b51e88b7a5ad31ae P 88a6b9e139ed41b49b48710b7d8658de: Deleting consensus metadata
I20260923 22:12:25.395924 3213 tablet_replica.cc:330] T 81c4606146c44f60b51e88b7a5ad31ae P a4124d4f2ad348d09b2f8903f9f35ba5: stopping tablet replica
I20260923 22:12:25.396000 3213 raft_consensus.cc:2229] T 81c4606146c44f60b51e88b7a5ad31ae P a4124d4f2ad348d09b2f8903f9f35ba5 [term 1 LEADER]: Raft consensus shutting down.
I20260923 22:12:25.396099 3213 raft_consensus.cc:2258] T 81c4606146c44f60b51e88b7a5ad31ae P a4124d4f2ad348d09b2f8903f9f35ba5 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260923 22:12:25.396142 3189 tablet_bootstrap.cc:654] T 4ff07aeb859349d49dafb0c1c4d4330e P 88a6b9e139ed41b49b48710b7d8658de: Neither blocks nor log segments found. Creating new log.
I20260923 22:12:25.396275 3190 tablet_bootstrap.cc:654] T 4ff07aeb859349d49dafb0c1c4d4330e P e067e9390e544580989450a94afe6ca4: Neither blocks nor log segments found. Creating new log.
I20260923 22:12:25.396425 3212 tablet_replica.cc:330] T 81c4606146c44f60b51e88b7a5ad31ae P e067e9390e544580989450a94afe6ca4: stopping tablet replica
I20260923 22:12:25.396535 3212 raft_consensus.cc:2229] T 81c4606146c44f60b51e88b7a5ad31ae P e067e9390e544580989450a94afe6ca4 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260923 22:12:25.396626 3212 raft_consensus.cc:2258] T 81c4606146c44f60b51e88b7a5ad31ae P e067e9390e544580989450a94afe6ca4 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260923 22:12:25.396694 2670 catalog_manager.cc:4841] TS 88a6b9e139ed41b49b48710b7d8658de (127.1.135.193:44317): tablet 81c4606146c44f60b51e88b7a5ad31ae (table test-table [id=ea683e7d69a44f27919f313c1d8be96d]) successfully deleted
I20260923 22:12:25.396821 2960 tablet_service.cc:1467] Processing CreateTablet for tablet 4ff07aeb859349d49dafb0c1c4d4330e (DEFAULT_TABLE table=test-table [id=82e30eab0daf47e2bcc79e1dc71f9e22]), partition=RANGE (key) PARTITION UNBOUNDED
I20260923 22:12:25.396926 2960 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 4ff07aeb859349d49dafb0c1c4d4330e. 1 dirs total, 0 dirs full, 0 dirs failed
I20260923 22:12:25.397221 3212 ts_tablet_manager.cc:1905] T 81c4606146c44f60b51e88b7a5ad31ae P e067e9390e544580989450a94afe6ca4: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260923 22:12:25.398025 3191 tablet_bootstrap.cc:492] T 4ff07aeb859349d49dafb0c1c4d4330e P a4124d4f2ad348d09b2f8903f9f35ba5: Bootstrap starting.
I20260923 22:12:25.398423 3191 tablet_bootstrap.cc:654] T 4ff07aeb859349d49dafb0c1c4d4330e P a4124d4f2ad348d09b2f8903f9f35ba5: Neither blocks nor log segments found. Creating new log.
I20260923 22:12:25.398995 3212 ts_tablet_manager.cc:1918] T 81c4606146c44f60b51e88b7a5ad31ae P e067e9390e544580989450a94afe6ca4: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260923 22:12:25.399053 3212 log.cc:1196] T 81c4606146c44f60b51e88b7a5ad31ae P e067e9390e544580989450a94afe6ca4: Deleting WAL directory at /tmp/dist-test-taskRiQe_a/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790201539513534-1567-0/minicluster-data/ts-2/wal/wals/81c4606146c44f60b51e88b7a5ad31ae
I20260923 22:12:25.399289 3212 ts_tablet_manager.cc:1939] T 81c4606146c44f60b51e88b7a5ad31ae P e067e9390e544580989450a94afe6ca4: Deleting consensus metadata
I20260923 22:12:25.399487 3191 tablet_bootstrap.cc:492] T 4ff07aeb859349d49dafb0c1c4d4330e P a4124d4f2ad348d09b2f8903f9f35ba5: No bootstrap required, opened a new log
I20260923 22:12:25.399551 3191 ts_tablet_manager.cc:1397] T 4ff07aeb859349d49dafb0c1c4d4330e P a4124d4f2ad348d09b2f8903f9f35ba5: Time spent bootstrapping tablet: real 0.002s user 0.000s sys 0.001s
I20260923 22:12:25.399704 2667 catalog_manager.cc:4841] TS e067e9390e544580989450a94afe6ca4 (127.1.135.195:44437): tablet 81c4606146c44f60b51e88b7a5ad31ae (table test-table [id=ea683e7d69a44f27919f313c1d8be96d]) successfully deleted
I20260923 22:12:25.399722 3191 raft_consensus.cc:348] T 4ff07aeb859349d49dafb0c1c4d4330e P a4124d4f2ad348d09b2f8903f9f35ba5 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "88a6b9e139ed41b49b48710b7d8658de" member_type: VOTER last_known_addr { host: "127.1.135.193" port: 44317 } } peers { permanent_uuid: "a4124d4f2ad348d09b2f8903f9f35ba5" member_type: VOTER last_known_addr { host: "127.1.135.194" port: 36581 } } peers { permanent_uuid: "e067e9390e544580989450a94afe6ca4" member_type: VOTER last_known_addr { host: "127.1.135.195" port: 44437 } }
I20260923 22:12:25.399785 3213 ts_tablet_manager.cc:1905] T 81c4606146c44f60b51e88b7a5ad31ae P a4124d4f2ad348d09b2f8903f9f35ba5: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260923 22:12:25.399791 3191 raft_consensus.cc:374] T 4ff07aeb859349d49dafb0c1c4d4330e P a4124d4f2ad348d09b2f8903f9f35ba5 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260923 22:12:25.399858 3191 raft_consensus.cc:729] T 4ff07aeb859349d49dafb0c1c4d4330e P a4124d4f2ad348d09b2f8903f9f35ba5 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: a4124d4f2ad348d09b2f8903f9f35ba5, State: Initialized, Role: FOLLOWER
I20260923 22:12:25.399911 3191 consensus_queue.cc:260] T 4ff07aeb859349d49dafb0c1c4d4330e P a4124d4f2ad348d09b2f8903f9f35ba5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "88a6b9e139ed41b49b48710b7d8658de" member_type: VOTER last_known_addr { host: "127.1.135.193" port: 44317 } } peers { permanent_uuid: "a4124d4f2ad348d09b2f8903f9f35ba5" member_type: VOTER last_known_addr { host: "127.1.135.194" port: 36581 } } peers { permanent_uuid: "e067e9390e544580989450a94afe6ca4" member_type: VOTER last_known_addr { host: "127.1.135.195" port: 44437 } }
I20260923 22:12:25.400000 3191 ts_tablet_manager.cc:1428] T 4ff07aeb859349d49dafb0c1c4d4330e P a4124d4f2ad348d09b2f8903f9f35ba5: Time spent starting tablet: real 0.000s user 0.000s sys 0.001s
I20260923 22:12:25.401181 3213 ts_tablet_manager.cc:1918] T 81c4606146c44f60b51e88b7a5ad31ae P a4124d4f2ad348d09b2f8903f9f35ba5: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260923 22:12:25.401252 3213 log.cc:1196] T 81c4606146c44f60b51e88b7a5ad31ae P a4124d4f2ad348d09b2f8903f9f35ba5: Deleting WAL directory at /tmp/dist-test-taskRiQe_a/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790201539513534-1567-0/minicluster-data/ts-1/wal/wals/81c4606146c44f60b51e88b7a5ad31ae
I20260923 22:12:25.401505 3213 ts_tablet_manager.cc:1939] T 81c4606146c44f60b51e88b7a5ad31ae P a4124d4f2ad348d09b2f8903f9f35ba5: Deleting consensus metadata
I20260923 22:12:25.401852 2668 catalog_manager.cc:4841] TS a4124d4f2ad348d09b2f8903f9f35ba5 (127.1.135.194:36581): tablet 81c4606146c44f60b51e88b7a5ad31ae (table test-table [id=ea683e7d69a44f27919f313c1d8be96d]) successfully deleted
I20260923 22:12:25.404470 3189 tablet_bootstrap.cc:492] T 4ff07aeb859349d49dafb0c1c4d4330e P 88a6b9e139ed41b49b48710b7d8658de: No bootstrap required, opened a new log
I20260923 22:12:25.404469 3190 tablet_bootstrap.cc:492] T 4ff07aeb859349d49dafb0c1c4d4330e P e067e9390e544580989450a94afe6ca4: No bootstrap required, opened a new log
I20260923 22:12:25.404523 3189 ts_tablet_manager.cc:1397] T 4ff07aeb859349d49dafb0c1c4d4330e P 88a6b9e139ed41b49b48710b7d8658de: Time spent bootstrapping tablet: real 0.009s user 0.000s sys 0.001s
I20260923 22:12:25.404523 3190 ts_tablet_manager.cc:1397] T 4ff07aeb859349d49dafb0c1c4d4330e P e067e9390e544580989450a94afe6ca4: Time spent bootstrapping tablet: real 0.009s user 0.001s sys 0.000s
I20260923 22:12:25.404662 3190 raft_consensus.cc:348] T 4ff07aeb859349d49dafb0c1c4d4330e P e067e9390e544580989450a94afe6ca4 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "88a6b9e139ed41b49b48710b7d8658de" member_type: VOTER last_known_addr { host: "127.1.135.193" port: 44317 } } peers { permanent_uuid: "a4124d4f2ad348d09b2f8903f9f35ba5" member_type: VOTER last_known_addr { host: "127.1.135.194" port: 36581 } } peers { permanent_uuid: "e067e9390e544580989450a94afe6ca4" member_type: VOTER last_known_addr { host: "127.1.135.195" port: 44437 } }
I20260923 22:12:25.404666 3189 raft_consensus.cc:348] T 4ff07aeb859349d49dafb0c1c4d4330e P 88a6b9e139ed41b49b48710b7d8658de [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "88a6b9e139ed41b49b48710b7d8658de" member_type: VOTER last_known_addr { host: "127.1.135.193" port: 44317 } } peers { permanent_uuid: "a4124d4f2ad348d09b2f8903f9f35ba5" member_type: VOTER last_known_addr { host: "127.1.135.194" port: 36581 } } peers { permanent_uuid: "e067e9390e544580989450a94afe6ca4" member_type: VOTER last_known_addr { host: "127.1.135.195" port: 44437 } }
I20260923 22:12:25.404732 3190 raft_consensus.cc:374] T 4ff07aeb859349d49dafb0c1c4d4330e P e067e9390e544580989450a94afe6ca4 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260923 22:12:25.404732 3189 raft_consensus.cc:374] T 4ff07aeb859349d49dafb0c1c4d4330e P 88a6b9e139ed41b49b48710b7d8658de [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260923 22:12:25.404754 3190 raft_consensus.cc:729] T 4ff07aeb859349d49dafb0c1c4d4330e P e067e9390e544580989450a94afe6ca4 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e067e9390e544580989450a94afe6ca4, State: Initialized, Role: FOLLOWER
I20260923 22:12:25.404754 3189 raft_consensus.cc:729] T 4ff07aeb859349d49dafb0c1c4d4330e P 88a6b9e139ed41b49b48710b7d8658de [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 88a6b9e139ed41b49b48710b7d8658de, State: Initialized, Role: FOLLOWER
I20260923 22:12:25.404815 3190 consensus_queue.cc:260] T 4ff07aeb859349d49dafb0c1c4d4330e P e067e9390e544580989450a94afe6ca4 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "88a6b9e139ed41b49b48710b7d8658de" member_type: VOTER last_known_addr { host: "127.1.135.193" port: 44317 } } peers { permanent_uuid: "a4124d4f2ad348d09b2f8903f9f35ba5" member_type: VOTER last_known_addr { host: "127.1.135.194" port: 36581 } } peers { permanent_uuid: "e067e9390e544580989450a94afe6ca4" member_type: VOTER last_known_addr { host: "127.1.135.195" port: 44437 } }
I20260923 22:12:25.404820 3189 consensus_queue.cc:260] T 4ff07aeb859349d49dafb0c1c4d4330e P 88a6b9e139ed41b49b48710b7d8658de [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "88a6b9e139ed41b49b48710b7d8658de" member_type: VOTER last_known_addr { host: "127.1.135.193" port: 44317 } } peers { permanent_uuid: "a4124d4f2ad348d09b2f8903f9f35ba5" member_type: VOTER last_known_addr { host: "127.1.135.194" port: 36581 } } peers { permanent_uuid: "e067e9390e544580989450a94afe6ca4" member_type: VOTER last_known_addr { host: "127.1.135.195" port: 44437 } }
I20260923 22:12:25.404907 3189 ts_tablet_manager.cc:1428] T 4ff07aeb859349d49dafb0c1c4d4330e P 88a6b9e139ed41b49b48710b7d8658de: Time spent starting tablet: real 0.000s user 0.000s sys 0.001s
I20260923 22:12:25.404919 3190 ts_tablet_manager.cc:1428] T 4ff07aeb859349d49dafb0c1c4d4330e P e067e9390e544580989450a94afe6ca4: Time spent starting tablet: real 0.000s user 0.001s sys 0.000s
I20260923 22:12:25.446473 3195 raft_consensus.cc:482] T 4ff07aeb859349d49dafb0c1c4d4330e P 88a6b9e139ed41b49b48710b7d8658de [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260923 22:12:25.446556 3195 raft_consensus.cc:504] T 4ff07aeb859349d49dafb0c1c4d4330e P 88a6b9e139ed41b49b48710b7d8658de [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "88a6b9e139ed41b49b48710b7d8658de" member_type: VOTER last_known_addr { host: "127.1.135.193" port: 44317 } } peers { permanent_uuid: "a4124d4f2ad348d09b2f8903f9f35ba5" member_type: VOTER last_known_addr { host: "127.1.135.194" port: 36581 } } peers { permanent_uuid: "e067e9390e544580989450a94afe6ca4" member_type: VOTER last_known_addr { host: "127.1.135.195" port: 44437 } }
I20260923 22:12:25.446934 3195 leader_election.cc:288] T 4ff07aeb859349d49dafb0c1c4d4330e P 88a6b9e139ed41b49b48710b7d8658de [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers a4124d4f2ad348d09b2f8903f9f35ba5 (127.1.135.194:36581), e067e9390e544580989450a94afe6ca4 (127.1.135.195:44437)
I20260923 22:12:25.450289 3187 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 22:12:25.447267 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.135.193:50659 (local address 127.1.135.195:44437)
0923 22:12:25.447362 (+ 95us) server_negotiation.cc:206] Beginning negotiation
0923 22:12:25.447364 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0923 22:12:25.447372 (+ 8us) server_negotiation.cc:407] Connection header received
0923 22:12:25.447458 (+ 86us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 22:12:25.447459 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 22:12:25.447468 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 22:12:25.447497 (+ 29us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0923 22:12:25.447700 (+ 203us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 22:12:25.448401 (+ 701us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 22:12:25.449360 (+ 959us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 22:12:25.449916 (+ 556us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 22:12:25.449984 (+ 68us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 22:12:25.449994 (+ 10us) server_negotiation.cc:1035] Waiting for connection context
0923 22:12:25.450188 (+ 194us) server_negotiation.cc:299] Negotiation successful
0923 22:12:25.450222 (+ 34us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":74}
I20260923 22:12:25.450301 3217 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 22:12:25.447100 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.135.195:44437 (local address 127.1.135.193:50659)
0923 22:12:25.447318 (+ 218us) negotiation.cc:107] Waiting for socket to connect
0923 22:12:25.447329 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0923 22:12:25.447375 (+ 46us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0923 22:12:25.447557 (+ 182us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0923 22:12:25.447558 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0923 22:12:25.447559 (+ 1us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0923 22:12:25.447629 (+ 70us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 22:12:25.447630 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 22:12:25.448480 (+ 850us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 22:12:25.448481 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 22:12:25.449269 (+ 788us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 22:12:25.449271 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 22:12:25.450047 (+ 776us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 22:12:25.450048 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 22:12:25.450116 (+ 68us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 22:12:25.450121 (+ 5us) client_negotiation.cc:769] Sending connection context
0923 22:12:25.450168 (+ 47us) client_negotiation.cc:240] Negotiation successful
0923 22:12:25.450207 (+ 39us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":205,"thread_start_us":69,"threads_started":1}
I20260923 22:12:25.450453 3188 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 22:12:25.447147 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.135.193:42629 (local address 127.1.135.194:36581)
0923 22:12:25.447190 (+ 43us) server_negotiation.cc:206] Beginning negotiation
0923 22:12:25.447192 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0923 22:12:25.447225 (+ 33us) server_negotiation.cc:407] Connection header received
0923 22:12:25.447252 (+ 27us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 22:12:25.447253 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 22:12:25.447264 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 22:12:25.447307 (+ 43us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0923 22:12:25.447455 (+ 148us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 22:12:25.448229 (+ 774us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 22:12:25.449498 (+ 1269us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 22:12:25.450029 (+ 531us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 22:12:25.450091 (+ 62us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 22:12:25.450097 (+ 6us) server_negotiation.cc:1035] Waiting for connection context
0923 22:12:25.450339 (+ 242us) server_negotiation.cc:299] Negotiation successful
0923 22:12:25.450377 (+ 38us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":21}
I20260923 22:12:25.450516 3120 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "4ff07aeb859349d49dafb0c1c4d4330e" candidate_uuid: "88a6b9e139ed41b49b48710b7d8658de" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e067e9390e544580989450a94afe6ca4" is_pre_election: true
I20260923 22:12:25.450615 3120 raft_consensus.cc:2454] T 4ff07aeb859349d49dafb0c1c4d4330e P e067e9390e544580989450a94afe6ca4 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 88a6b9e139ed41b49b48710b7d8658de in term 0.
I20260923 22:12:25.450623 2779 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 22:12:25.447083 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.135.194:36581 (local address 127.1.135.193:42629)
0923 22:12:25.447136 (+ 53us) negotiation.cc:107] Waiting for socket to connect
0923 22:12:25.447147 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0923 22:12:25.447186 (+ 39us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0923 22:12:25.447323 (+ 137us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0923 22:12:25.447324 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0923 22:12:25.447325 (+ 1us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0923 22:12:25.447395 (+ 70us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 22:12:25.447397 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 22:12:25.448307 (+ 910us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 22:12:25.448309 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 22:12:25.449398 (+ 1089us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 22:12:25.449401 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 22:12:25.450121 (+ 720us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 22:12:25.450122 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 22:12:25.450201 (+ 79us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 22:12:25.450205 (+ 4us) client_negotiation.cc:769] Sending connection context
0923 22:12:25.450262 (+ 57us) client_negotiation.cc:240] Negotiation successful
0923 22:12:25.450302 (+ 40us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":35}
I20260923 22:12:25.450778 2771 leader_election.cc:302] T 4ff07aeb859349d49dafb0c1c4d4330e P 88a6b9e139ed41b49b48710b7d8658de [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 88a6b9e139ed41b49b48710b7d8658de, e067e9390e544580989450a94afe6ca4; no voters:
I20260923 22:12:25.450829 2979 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "4ff07aeb859349d49dafb0c1c4d4330e" candidate_uuid: "88a6b9e139ed41b49b48710b7d8658de" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a4124d4f2ad348d09b2f8903f9f35ba5" is_pre_election: true
I20260923 22:12:25.450881 3195 raft_consensus.cc:2781] T 4ff07aeb859349d49dafb0c1c4d4330e P 88a6b9e139ed41b49b48710b7d8658de [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260923 22:12:25.450928 3195 raft_consensus.cc:482] T 4ff07aeb859349d49dafb0c1c4d4330e P 88a6b9e139ed41b49b48710b7d8658de [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260923 22:12:25.450951 3195 raft_consensus.cc:3037] T 4ff07aeb859349d49dafb0c1c4d4330e P 88a6b9e139ed41b49b48710b7d8658de [term 0 FOLLOWER]: Advancing to term 1
I20260923 22:12:25.450935 2979 raft_consensus.cc:2454] T 4ff07aeb859349d49dafb0c1c4d4330e P a4124d4f2ad348d09b2f8903f9f35ba5 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 88a6b9e139ed41b49b48710b7d8658de in term 0.
I20260923 22:12:25.451673 3195 raft_consensus.cc:504] T 4ff07aeb859349d49dafb0c1c4d4330e P 88a6b9e139ed41b49b48710b7d8658de [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "88a6b9e139ed41b49b48710b7d8658de" member_type: VOTER last_known_addr { host: "127.1.135.193" port: 44317 } } peers { permanent_uuid: "a4124d4f2ad348d09b2f8903f9f35ba5" member_type: VOTER last_known_addr { host: "127.1.135.194" port: 36581 } } peers { permanent_uuid: "e067e9390e544580989450a94afe6ca4" member_type: VOTER last_known_addr { host: "127.1.135.195" port: 44437 } }
I20260923 22:12:25.451817 3195 leader_election.cc:288] T 4ff07aeb859349d49dafb0c1c4d4330e P 88a6b9e139ed41b49b48710b7d8658de [CANDIDATE]: Term 1 election: Requested vote from peers a4124d4f2ad348d09b2f8903f9f35ba5 (127.1.135.194:36581), e067e9390e544580989450a94afe6ca4 (127.1.135.195:44437)
I20260923 22:12:25.451960 3120 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "4ff07aeb859349d49dafb0c1c4d4330e" candidate_uuid: "88a6b9e139ed41b49b48710b7d8658de" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e067e9390e544580989450a94afe6ca4"
I20260923 22:12:25.451988 2979 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "4ff07aeb859349d49dafb0c1c4d4330e" candidate_uuid: "88a6b9e139ed41b49b48710b7d8658de" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a4124d4f2ad348d09b2f8903f9f35ba5"
I20260923 22:12:25.452030 3120 raft_consensus.cc:3037] T 4ff07aeb859349d49dafb0c1c4d4330e P e067e9390e544580989450a94afe6ca4 [term 0 FOLLOWER]: Advancing to term 1
I20260923 22:12:25.452042 2979 raft_consensus.cc:3037] T 4ff07aeb859349d49dafb0c1c4d4330e P a4124d4f2ad348d09b2f8903f9f35ba5 [term 0 FOLLOWER]: Advancing to term 1
I20260923 22:12:25.452757 2979 raft_consensus.cc:2454] T 4ff07aeb859349d49dafb0c1c4d4330e P a4124d4f2ad348d09b2f8903f9f35ba5 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 88a6b9e139ed41b49b48710b7d8658de in term 1.
I20260923 22:12:25.452757 3120 raft_consensus.cc:2454] T 4ff07aeb859349d49dafb0c1c4d4330e P e067e9390e544580989450a94afe6ca4 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 88a6b9e139ed41b49b48710b7d8658de in term 1.
I20260923 22:12:25.452941 2771 leader_election.cc:302] T 4ff07aeb859349d49dafb0c1c4d4330e P 88a6b9e139ed41b49b48710b7d8658de [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 88a6b9e139ed41b49b48710b7d8658de, e067e9390e544580989450a94afe6ca4; no voters:
I20260923 22:12:25.453012 3195 raft_consensus.cc:2781] T 4ff07aeb859349d49dafb0c1c4d4330e P 88a6b9e139ed41b49b48710b7d8658de [term 1 FOLLOWER]: Leader election won for term 1
I20260923 22:12:25.453132 3195 raft_consensus.cc:686] T 4ff07aeb859349d49dafb0c1c4d4330e P 88a6b9e139ed41b49b48710b7d8658de [term 1 LEADER]: Becoming Leader. State: Replica: 88a6b9e139ed41b49b48710b7d8658de, State: Running, Role: LEADER
I20260923 22:12:25.453202 3195 consensus_queue.cc:237] T 4ff07aeb859349d49dafb0c1c4d4330e P 88a6b9e139ed41b49b48710b7d8658de [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "88a6b9e139ed41b49b48710b7d8658de" member_type: VOTER last_known_addr { host: "127.1.135.193" port: 44317 } } peers { permanent_uuid: "a4124d4f2ad348d09b2f8903f9f35ba5" member_type: VOTER last_known_addr { host: "127.1.135.194" port: 36581 } } peers { permanent_uuid: "e067e9390e544580989450a94afe6ca4" member_type: VOTER last_known_addr { host: "127.1.135.195" port: 44437 } }
I20260923 22:12:25.453883 2681 catalog_manager.cc:5495] T 4ff07aeb859349d49dafb0c1c4d4330e P 88a6b9e139ed41b49b48710b7d8658de reported cstate change: term changed from 0 to 1, leader changed from <none> to 88a6b9e139ed41b49b48710b7d8658de (127.1.135.193). New cstate: current_term: 1 leader_uuid: "88a6b9e139ed41b49b48710b7d8658de" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "88a6b9e139ed41b49b48710b7d8658de" member_type: VOTER last_known_addr { host: "127.1.135.193" port: 44317 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "a4124d4f2ad348d09b2f8903f9f35ba5" member_type: VOTER last_known_addr { host: "127.1.135.194" port: 36581 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "e067e9390e544580989450a94afe6ca4" member_type: VOTER last_known_addr { host: "127.1.135.195" port: 44437 } health_report { overall_health: UNKNOWN } } }
W20260923 22:12:25.456233 2887 tablet.cc:2369] T 4ff07aeb859349d49dafb0c1c4d4330e P 88a6b9e139ed41b49b48710b7d8658de: Can't schedule compaction. Clean time has not been advanced past its initial value.
Sep 23 22:12:25 dist-test-slave-s2fj krb5kdc[2638](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790201544, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.135.254@KRBTEST.COM for kudu/127.1.135.254@KRBTEST.COM
I20260923 22:12:25.484645 2751 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 22:12:25.476799 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56374 (local address 127.1.135.254:40067)
0923 22:12:25.476847 (+ 48us) server_negotiation.cc:206] Beginning negotiation
0923 22:12:25.476849 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0923 22:12:25.476979 (+ 130us) server_negotiation.cc:407] Connection header received
0923 22:12:25.477046 (+ 67us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 22:12:25.477047 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 22:12:25.477055 (+ 8us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 22:12:25.477105 (+ 50us) server_negotiation.cc:226] Negotiated authn=SASL
0923 22:12:25.477664 (+ 559us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 22:12:25.478165 (+ 501us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 22:12:25.479176 (+ 1011us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 22:12:25.479345 (+ 169us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 22:12:25.479392 (+ 47us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 22:12:25.482058 (+ 2666us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0923 22:12:25.482059 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0923 22:12:25.482060 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0923 22:12:25.482067 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0923 22:12:25.483427 (+ 1360us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0923 22:12:25.483821 (+ 394us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0923 22:12:25.483822 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0923 22:12:25.483823 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0923 22:12:25.483891 (+ 68us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0923 22:12:25.484111 (+ 220us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0923 22:12:25.484112 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0923 22:12:25.484113 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0923 22:12:25.484217 (+ 104us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0923 22:12:25.484293 (+ 76us) server_negotiation.cc:1035] Waiting for connection context
0923 22:12:25.484447 (+ 154us) server_negotiation.cc:299] Negotiation successful
0923 22:12:25.484531 (+ 84us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":27}
I20260923 22:12:25.484714 3225 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 22:12:25.476714 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.135.254:40067 (local address 127.0.0.1:56374)
0923 22:12:25.476895 (+ 181us) negotiation.cc:107] Waiting for socket to connect
0923 22:12:25.476907 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0923 22:12:25.476985 (+ 78us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0923 22:12:25.477133 (+ 148us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0923 22:12:25.477133 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0923 22:12:25.477533 (+ 400us) client_negotiation.cc:189] Negotiated authn=SASL
0923 22:12:25.477588 (+ 55us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 22:12:25.477591 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 22:12:25.478243 (+ 652us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 22:12:25.478244 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 22:12:25.479094 (+ 850us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 22:12:25.479097 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 22:12:25.479412 (+ 315us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 22:12:25.479413 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 22:12:25.479481 (+ 68us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 22:12:25.480046 (+ 565us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0923 22:12:25.480052 (+ 6us) client_negotiation.cc:656] calling sasl_client_start()
0923 22:12:25.481965 (+ 1913us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0923 22:12:25.483529 (+ 1564us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0923 22:12:25.483537 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0923 22:12:25.483539 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0923 22:12:25.483763 (+ 224us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0923 22:12:25.483987 (+ 224us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0923 22:12:25.483988 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0923 22:12:25.483988 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0923 22:12:25.484048 (+ 60us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0923 22:12:25.484283 (+ 235us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0923 22:12:25.484290 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0923 22:12:25.484334 (+ 44us) client_negotiation.cc:769] Sending connection context
0923 22:12:25.484425 (+ 91us) client_negotiation.cc:240] Negotiation successful
0923 22:12:25.484598 (+ 173us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":149,"mutex_wait_us":78,"thread_start_us":93,"threads_started":1}
I20260923 22:12:25.486723 2681 catalog_manager.cc:1908] Servicing CreateTable request from {username='kudu', principal='kudu/127.1.135.254@KRBTEST.COM'} at 127.0.0.1:56374:
name: "kudu_system.kudu_transactions"
schema {
columns {
name: "txn_id"
type: INT64
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "entry_type"
type: INT8
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "identifier"
type: STRING
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "metadata"
type: STRING
is_key: false
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
rows: "<redacted>""\006\001\000\000\000\000\000\000\000\000\007\001@B\017\000\000\000\000\000"
indirect_data: "<redacted>"""
}
partition_schema {
range_schema {
columns {
name: "txn_id"
}
}
}
table_type: TXN_STATUS_TABLE
W20260923 22:12:25.486994 2681 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table kudu_system.kudu_transactions in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260923 22:12:25.488591 3100 tablet_service.cc:1467] Processing CreateTablet for tablet 1aaba0de8aea4efda9a1b998152811d1 (TXN_STATUS_TABLE table=kudu_system.kudu_transactions [id=d0f5499a3e6c40e4b8eeb561f21a76bd]), partition=RANGE (txn_id) PARTITION 0 <= VALUES < 1000000
I20260923 22:12:25.488783 3100 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 1aaba0de8aea4efda9a1b998152811d1. 1 dirs total, 0 dirs full, 0 dirs failed
I20260923 22:12:25.488525 2820 tablet_service.cc:1467] Processing CreateTablet for tablet 1aaba0de8aea4efda9a1b998152811d1 (TXN_STATUS_TABLE table=kudu_system.kudu_transactions [id=d0f5499a3e6c40e4b8eeb561f21a76bd]), partition=RANGE (txn_id) PARTITION 0 <= VALUES < 1000000
I20260923 22:12:25.489084 2820 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 1aaba0de8aea4efda9a1b998152811d1. 1 dirs total, 0 dirs full, 0 dirs failed
I20260923 22:12:25.490398 3190 tablet_bootstrap.cc:492] T 1aaba0de8aea4efda9a1b998152811d1 P e067e9390e544580989450a94afe6ca4: Bootstrap starting.
I20260923 22:12:25.490753 2960 tablet_service.cc:1467] Processing CreateTablet for tablet 1aaba0de8aea4efda9a1b998152811d1 (TXN_STATUS_TABLE table=kudu_system.kudu_transactions [id=d0f5499a3e6c40e4b8eeb561f21a76bd]), partition=RANGE (txn_id) PARTITION 0 <= VALUES < 1000000
I20260923 22:12:25.490895 2960 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 1aaba0de8aea4efda9a1b998152811d1. 1 dirs total, 0 dirs full, 0 dirs failed
I20260923 22:12:25.490926 3190 tablet_bootstrap.cc:654] T 1aaba0de8aea4efda9a1b998152811d1 P e067e9390e544580989450a94afe6ca4: Neither blocks nor log segments found. Creating new log.
I20260923 22:12:25.491168 3189 tablet_bootstrap.cc:492] T 1aaba0de8aea4efda9a1b998152811d1 P 88a6b9e139ed41b49b48710b7d8658de: Bootstrap starting.
I20260923 22:12:25.491568 3189 tablet_bootstrap.cc:654] T 1aaba0de8aea4efda9a1b998152811d1 P 88a6b9e139ed41b49b48710b7d8658de: Neither blocks nor log segments found. Creating new log.
I20260923 22:12:25.491961 3190 tablet_bootstrap.cc:492] T 1aaba0de8aea4efda9a1b998152811d1 P e067e9390e544580989450a94afe6ca4: No bootstrap required, opened a new log
I20260923 22:12:25.492014 3190 ts_tablet_manager.cc:1397] T 1aaba0de8aea4efda9a1b998152811d1 P e067e9390e544580989450a94afe6ca4: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20260923 22:12:25.492280 3190 raft_consensus.cc:348] T 1aaba0de8aea4efda9a1b998152811d1 P e067e9390e544580989450a94afe6ca4 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "88a6b9e139ed41b49b48710b7d8658de" member_type: VOTER last_known_addr { host: "127.1.135.193" port: 44317 } } peers { permanent_uuid: "a4124d4f2ad348d09b2f8903f9f35ba5" member_type: VOTER last_known_addr { host: "127.1.135.194" port: 36581 } } peers { permanent_uuid: "e067e9390e544580989450a94afe6ca4" member_type: VOTER last_known_addr { host: "127.1.135.195" port: 44437 } }
I20260923 22:12:25.492367 3190 raft_consensus.cc:374] T 1aaba0de8aea4efda9a1b998152811d1 P e067e9390e544580989450a94afe6ca4 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260923 22:12:25.492389 3190 raft_consensus.cc:729] T 1aaba0de8aea4efda9a1b998152811d1 P e067e9390e544580989450a94afe6ca4 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e067e9390e544580989450a94afe6ca4, State: Initialized, Role: FOLLOWER
I20260923 22:12:25.492435 3190 consensus_queue.cc:260] T 1aaba0de8aea4efda9a1b998152811d1 P e067e9390e544580989450a94afe6ca4 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "88a6b9e139ed41b49b48710b7d8658de" member_type: VOTER last_known_addr { host: "127.1.135.193" port: 44317 } } peers { permanent_uuid: "a4124d4f2ad348d09b2f8903f9f35ba5" member_type: VOTER last_known_addr { host: "127.1.135.194" port: 36581 } } peers { permanent_uuid: "e067e9390e544580989450a94afe6ca4" member_type: VOTER last_known_addr { host: "127.1.135.195" port: 44437 } }
I20260923 22:12:25.492524 3190 ts_tablet_manager.cc:1428] T 1aaba0de8aea4efda9a1b998152811d1 P e067e9390e544580989450a94afe6ca4: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20260923 22:12:25.492604 3189 tablet_bootstrap.cc:492] T 1aaba0de8aea4efda9a1b998152811d1 P 88a6b9e139ed41b49b48710b7d8658de: No bootstrap required, opened a new log
I20260923 22:12:25.492650 3189 ts_tablet_manager.cc:1397] T 1aaba0de8aea4efda9a1b998152811d1 P 88a6b9e139ed41b49b48710b7d8658de: Time spent bootstrapping tablet: real 0.001s user 0.000s sys 0.001s
I20260923 22:12:25.492681 3191 tablet_bootstrap.cc:492] T 1aaba0de8aea4efda9a1b998152811d1 P a4124d4f2ad348d09b2f8903f9f35ba5: Bootstrap starting.
I20260923 22:12:25.492669 3197 tablet_replica.cc:439] T 1aaba0de8aea4efda9a1b998152811d1 P e067e9390e544580989450a94afe6ca4: TxnStatusTablet state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "88a6b9e139ed41b49b48710b7d8658de" member_type: VOTER last_known_addr { host: "127.1.135.193" port: 44317 } } peers { permanent_uuid: "a4124d4f2ad348d09b2f8903f9f35ba5" member_type: VOTER last_known_addr { host: "127.1.135.194" port: 36581 } } peers { permanent_uuid: "e067e9390e544580989450a94afe6ca4" member_type: VOTER last_known_addr { host: "127.1.135.195" port: 44437 } } }
I20260923 22:12:25.492748 3197 tablet_replica.cc:442] T 1aaba0de8aea4efda9a1b998152811d1 P e067e9390e544580989450a94afe6ca4: This TxnStatusTablet replica's current role is: FOLLOWER
I20260923 22:12:25.492787 3189 raft_consensus.cc:348] T 1aaba0de8aea4efda9a1b998152811d1 P 88a6b9e139ed41b49b48710b7d8658de [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "88a6b9e139ed41b49b48710b7d8658de" member_type: VOTER last_known_addr { host: "127.1.135.193" port: 44317 } } peers { permanent_uuid: "a4124d4f2ad348d09b2f8903f9f35ba5" member_type: VOTER last_known_addr { host: "127.1.135.194" port: 36581 } } peers { permanent_uuid: "e067e9390e544580989450a94afe6ca4" member_type: VOTER last_known_addr { host: "127.1.135.195" port: 44437 } }
I20260923 22:12:25.492856 3189 raft_consensus.cc:374] T 1aaba0de8aea4efda9a1b998152811d1 P 88a6b9e139ed41b49b48710b7d8658de [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260923 22:12:25.492875 3189 raft_consensus.cc:729] T 1aaba0de8aea4efda9a1b998152811d1 P 88a6b9e139ed41b49b48710b7d8658de [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 88a6b9e139ed41b49b48710b7d8658de, State: Initialized, Role: FOLLOWER
I20260923 22:12:25.492918 3189 consensus_queue.cc:260] T 1aaba0de8aea4efda9a1b998152811d1 P 88a6b9e139ed41b49b48710b7d8658de [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "88a6b9e139ed41b49b48710b7d8658de" member_type: VOTER last_known_addr { host: "127.1.135.193" port: 44317 } } peers { permanent_uuid: "a4124d4f2ad348d09b2f8903f9f35ba5" member_type: VOTER last_known_addr { host: "127.1.135.194" port: 36581 } } peers { permanent_uuid: "e067e9390e544580989450a94afe6ca4" member_type: VOTER last_known_addr { host: "127.1.135.195" port: 44437 } }
I20260923 22:12:25.492996 3189 ts_tablet_manager.cc:1428] T 1aaba0de8aea4efda9a1b998152811d1 P 88a6b9e139ed41b49b48710b7d8658de: Time spent starting tablet: real 0.000s user 0.000s sys 0.001s
I20260923 22:12:25.493124 3195 tablet_replica.cc:439] T 1aaba0de8aea4efda9a1b998152811d1 P 88a6b9e139ed41b49b48710b7d8658de: TxnStatusTablet state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "88a6b9e139ed41b49b48710b7d8658de" member_type: VOTER last_known_addr { host: "127.1.135.193" port: 44317 } } peers { permanent_uuid: "a4124d4f2ad348d09b2f8903f9f35ba5" member_type: VOTER last_known_addr { host: "127.1.135.194" port: 36581 } } peers { permanent_uuid: "e067e9390e544580989450a94afe6ca4" member_type: VOTER last_known_addr { host: "127.1.135.195" port: 44437 } } }
I20260923 22:12:25.493211 3195 tablet_replica.cc:442] T 1aaba0de8aea4efda9a1b998152811d1 P 88a6b9e139ed41b49b48710b7d8658de: This TxnStatusTablet replica's current role is: FOLLOWER
I20260923 22:12:25.493223 3191 tablet_bootstrap.cc:654] T 1aaba0de8aea4efda9a1b998152811d1 P a4124d4f2ad348d09b2f8903f9f35ba5: Neither blocks nor log segments found. Creating new log.
I20260923 22:12:25.494148 3191 tablet_bootstrap.cc:492] T 1aaba0de8aea4efda9a1b998152811d1 P a4124d4f2ad348d09b2f8903f9f35ba5: No bootstrap required, opened a new log
I20260923 22:12:25.494223 3191 ts_tablet_manager.cc:1397] T 1aaba0de8aea4efda9a1b998152811d1 P a4124d4f2ad348d09b2f8903f9f35ba5: Time spent bootstrapping tablet: real 0.002s user 0.000s sys 0.001s
I20260923 22:12:25.494388 3191 raft_consensus.cc:348] T 1aaba0de8aea4efda9a1b998152811d1 P a4124d4f2ad348d09b2f8903f9f35ba5 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "88a6b9e139ed41b49b48710b7d8658de" member_type: VOTER last_known_addr { host: "127.1.135.193" port: 44317 } } peers { permanent_uuid: "a4124d4f2ad348d09b2f8903f9f35ba5" member_type: VOTER last_known_addr { host: "127.1.135.194" port: 36581 } } peers { permanent_uuid: "e067e9390e544580989450a94afe6ca4" member_type: VOTER last_known_addr { host: "127.1.135.195" port: 44437 } }
I20260923 22:12:25.494496 3191 raft_consensus.cc:374] T 1aaba0de8aea4efda9a1b998152811d1 P a4124d4f2ad348d09b2f8903f9f35ba5 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260923 22:12:25.494539 3191 raft_consensus.cc:729] T 1aaba0de8aea4efda9a1b998152811d1 P a4124d4f2ad348d09b2f8903f9f35ba5 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: a4124d4f2ad348d09b2f8903f9f35ba5, State: Initialized, Role: FOLLOWER
I20260923 22:12:25.494593 3191 consensus_queue.cc:260] T 1aaba0de8aea4efda9a1b998152811d1 P a4124d4f2ad348d09b2f8903f9f35ba5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "88a6b9e139ed41b49b48710b7d8658de" member_type: VOTER last_known_addr { host: "127.1.135.193" port: 44317 } } peers { permanent_uuid: "a4124d4f2ad348d09b2f8903f9f35ba5" member_type: VOTER last_known_addr { host: "127.1.135.194" port: 36581 } } peers { permanent_uuid: "e067e9390e544580989450a94afe6ca4" member_type: VOTER last_known_addr { host: "127.1.135.195" port: 44437 } }
I20260923 22:12:25.494694 3191 ts_tablet_manager.cc:1428] T 1aaba0de8aea4efda9a1b998152811d1 P a4124d4f2ad348d09b2f8903f9f35ba5: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20260923 22:12:25.494738 3196 tablet_replica.cc:439] T 1aaba0de8aea4efda9a1b998152811d1 P a4124d4f2ad348d09b2f8903f9f35ba5: TxnStatusTablet state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "88a6b9e139ed41b49b48710b7d8658de" member_type: VOTER last_known_addr { host: "127.1.135.193" port: 44317 } } peers { permanent_uuid: "a4124d4f2ad348d09b2f8903f9f35ba5" member_type: VOTER last_known_addr { host: "127.1.135.194" port: 36581 } } peers { permanent_uuid: "e067e9390e544580989450a94afe6ca4" member_type: VOTER last_known_addr { host: "127.1.135.195" port: 44437 } } }
I20260923 22:12:25.494822 3196 tablet_replica.cc:442] T 1aaba0de8aea4efda9a1b998152811d1 P a4124d4f2ad348d09b2f8903f9f35ba5: This TxnStatusTablet replica's current role is: FOLLOWER
W20260923 22:12:25.518018 3167 tablet.cc:2369] T 1aaba0de8aea4efda9a1b998152811d1 P e067e9390e544580989450a94afe6ca4: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260923 22:12:25.617524 3195 raft_consensus.cc:482] T 1aaba0de8aea4efda9a1b998152811d1 P 88a6b9e139ed41b49b48710b7d8658de [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260923 22:12:25.617611 3195 raft_consensus.cc:504] T 1aaba0de8aea4efda9a1b998152811d1 P 88a6b9e139ed41b49b48710b7d8658de [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "88a6b9e139ed41b49b48710b7d8658de" member_type: VOTER last_known_addr { host: "127.1.135.193" port: 44317 } } peers { permanent_uuid: "a4124d4f2ad348d09b2f8903f9f35ba5" member_type: VOTER last_known_addr { host: "127.1.135.194" port: 36581 } } peers { permanent_uuid: "e067e9390e544580989450a94afe6ca4" member_type: VOTER last_known_addr { host: "127.1.135.195" port: 44437 } }
I20260923 22:12:25.617755 3195 leader_election.cc:288] T 1aaba0de8aea4efda9a1b998152811d1 P 88a6b9e139ed41b49b48710b7d8658de [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers a4124d4f2ad348d09b2f8903f9f35ba5 (127.1.135.194:36581), e067e9390e544580989450a94afe6ca4 (127.1.135.195:44437)
I20260923 22:12:25.617934 2979 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "1aaba0de8aea4efda9a1b998152811d1" candidate_uuid: "88a6b9e139ed41b49b48710b7d8658de" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a4124d4f2ad348d09b2f8903f9f35ba5" is_pre_election: true
I20260923 22:12:25.618005 3120 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "1aaba0de8aea4efda9a1b998152811d1" candidate_uuid: "88a6b9e139ed41b49b48710b7d8658de" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e067e9390e544580989450a94afe6ca4" is_pre_election: true
I20260923 22:12:25.618068 2979 raft_consensus.cc:2454] T 1aaba0de8aea4efda9a1b998152811d1 P a4124d4f2ad348d09b2f8903f9f35ba5 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 88a6b9e139ed41b49b48710b7d8658de in term 0.
I20260923 22:12:25.618081 3120 raft_consensus.cc:2454] T 1aaba0de8aea4efda9a1b998152811d1 P e067e9390e544580989450a94afe6ca4 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 88a6b9e139ed41b49b48710b7d8658de in term 0.
I20260923 22:12:25.618251 2772 leader_election.cc:302] T 1aaba0de8aea4efda9a1b998152811d1 P 88a6b9e139ed41b49b48710b7d8658de [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 88a6b9e139ed41b49b48710b7d8658de, a4124d4f2ad348d09b2f8903f9f35ba5; no voters:
I20260923 22:12:25.618368 3195 raft_consensus.cc:2781] T 1aaba0de8aea4efda9a1b998152811d1 P 88a6b9e139ed41b49b48710b7d8658de [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260923 22:12:25.618448 3195 raft_consensus.cc:482] T 1aaba0de8aea4efda9a1b998152811d1 P 88a6b9e139ed41b49b48710b7d8658de [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260923 22:12:25.618480 3195 raft_consensus.cc:3037] T 1aaba0de8aea4efda9a1b998152811d1 P 88a6b9e139ed41b49b48710b7d8658de [term 0 FOLLOWER]: Advancing to term 1
I20260923 22:12:25.619125 3195 raft_consensus.cc:504] T 1aaba0de8aea4efda9a1b998152811d1 P 88a6b9e139ed41b49b48710b7d8658de [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "88a6b9e139ed41b49b48710b7d8658de" member_type: VOTER last_known_addr { host: "127.1.135.193" port: 44317 } } peers { permanent_uuid: "a4124d4f2ad348d09b2f8903f9f35ba5" member_type: VOTER last_known_addr { host: "127.1.135.194" port: 36581 } } peers { permanent_uuid: "e067e9390e544580989450a94afe6ca4" member_type: VOTER last_known_addr { host: "127.1.135.195" port: 44437 } }
I20260923 22:12:25.619266 3195 leader_election.cc:288] T 1aaba0de8aea4efda9a1b998152811d1 P 88a6b9e139ed41b49b48710b7d8658de [CANDIDATE]: Term 1 election: Requested vote from peers a4124d4f2ad348d09b2f8903f9f35ba5 (127.1.135.194:36581), e067e9390e544580989450a94afe6ca4 (127.1.135.195:44437)
I20260923 22:12:25.619431 3120 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "1aaba0de8aea4efda9a1b998152811d1" candidate_uuid: "88a6b9e139ed41b49b48710b7d8658de" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e067e9390e544580989450a94afe6ca4"
I20260923 22:12:25.619444 2979 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "1aaba0de8aea4efda9a1b998152811d1" candidate_uuid: "88a6b9e139ed41b49b48710b7d8658de" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a4124d4f2ad348d09b2f8903f9f35ba5"
I20260923 22:12:25.619504 3120 raft_consensus.cc:3037] T 1aaba0de8aea4efda9a1b998152811d1 P e067e9390e544580989450a94afe6ca4 [term 0 FOLLOWER]: Advancing to term 1
I20260923 22:12:25.619524 2979 raft_consensus.cc:3037] T 1aaba0de8aea4efda9a1b998152811d1 P a4124d4f2ad348d09b2f8903f9f35ba5 [term 0 FOLLOWER]: Advancing to term 1
I20260923 22:12:25.620117 2979 raft_consensus.cc:2454] T 1aaba0de8aea4efda9a1b998152811d1 P a4124d4f2ad348d09b2f8903f9f35ba5 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 88a6b9e139ed41b49b48710b7d8658de in term 1.
I20260923 22:12:25.620169 3120 raft_consensus.cc:2454] T 1aaba0de8aea4efda9a1b998152811d1 P e067e9390e544580989450a94afe6ca4 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 88a6b9e139ed41b49b48710b7d8658de in term 1.
I20260923 22:12:25.620296 2772 leader_election.cc:302] T 1aaba0de8aea4efda9a1b998152811d1 P 88a6b9e139ed41b49b48710b7d8658de [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 88a6b9e139ed41b49b48710b7d8658de, a4124d4f2ad348d09b2f8903f9f35ba5; no voters:
I20260923 22:12:25.620437 3195 raft_consensus.cc:2781] T 1aaba0de8aea4efda9a1b998152811d1 P 88a6b9e139ed41b49b48710b7d8658de [term 1 FOLLOWER]: Leader election won for term 1
I20260923 22:12:25.620498 3195 raft_consensus.cc:686] T 1aaba0de8aea4efda9a1b998152811d1 P 88a6b9e139ed41b49b48710b7d8658de [term 1 LEADER]: Becoming Leader. State: Replica: 88a6b9e139ed41b49b48710b7d8658de, State: Running, Role: LEADER
I20260923 22:12:25.620550 3195 consensus_queue.cc:237] T 1aaba0de8aea4efda9a1b998152811d1 P 88a6b9e139ed41b49b48710b7d8658de [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "88a6b9e139ed41b49b48710b7d8658de" member_type: VOTER last_known_addr { host: "127.1.135.193" port: 44317 } } peers { permanent_uuid: "a4124d4f2ad348d09b2f8903f9f35ba5" member_type: VOTER last_known_addr { host: "127.1.135.194" port: 36581 } } peers { permanent_uuid: "e067e9390e544580989450a94afe6ca4" member_type: VOTER last_known_addr { host: "127.1.135.195" port: 44437 } }
I20260923 22:12:25.620831 3218 tablet_replica.cc:439] T 1aaba0de8aea4efda9a1b998152811d1 P 88a6b9e139ed41b49b48710b7d8658de: TxnStatusTablet state changed. Reason: New leader 88a6b9e139ed41b49b48710b7d8658de. Latest consensus state: current_term: 1 leader_uuid: "88a6b9e139ed41b49b48710b7d8658de" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "88a6b9e139ed41b49b48710b7d8658de" member_type: VOTER last_known_addr { host: "127.1.135.193" port: 44317 } } peers { permanent_uuid: "a4124d4f2ad348d09b2f8903f9f35ba5" member_type: VOTER last_known_addr { host: "127.1.135.194" port: 36581 } } peers { permanent_uuid: "e067e9390e544580989450a94afe6ca4" member_type: VOTER last_known_addr { host: "127.1.135.195" port: 44437 } } }
I20260923 22:12:25.620945 3218 tablet_replica.cc:442] T 1aaba0de8aea4efda9a1b998152811d1 P 88a6b9e139ed41b49b48710b7d8658de: This TxnStatusTablet replica's current role is: LEADER
I20260923 22:12:25.621140 3230 txn_status_manager.cc:873] Waiting until node catch up with all replicated operations in previous term...
I20260923 22:12:25.621134 2681 catalog_manager.cc:5495] T 1aaba0de8aea4efda9a1b998152811d1 P 88a6b9e139ed41b49b48710b7d8658de reported cstate change: term changed from 0 to 1, leader changed from <none> to 88a6b9e139ed41b49b48710b7d8658de (127.1.135.193). New cstate: current_term: 1 leader_uuid: "88a6b9e139ed41b49b48710b7d8658de" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "88a6b9e139ed41b49b48710b7d8658de" member_type: VOTER last_known_addr { host: "127.1.135.193" port: 44317 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "a4124d4f2ad348d09b2f8903f9f35ba5" member_type: VOTER last_known_addr { host: "127.1.135.194" port: 36581 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "e067e9390e544580989450a94afe6ca4" member_type: VOTER last_known_addr { host: "127.1.135.195" port: 44437 } health_report { overall_health: UNKNOWN } } }
I20260923 22:12:25.621268 3230 txn_status_manager.cc:929] Loading transaction status metadata into memory...
I20260923 22:12:25.639284 3186 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 22:12:25.636814 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42512 (local address 127.1.135.193:44317)
0923 22:12:25.636859 (+ 45us) server_negotiation.cc:206] Beginning negotiation
0923 22:12:25.636862 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0923 22:12:25.636871 (+ 9us) server_negotiation.cc:407] Connection header received
0923 22:12:25.636958 (+ 87us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 22:12:25.636959 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 22:12:25.636977 (+ 18us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 22:12:25.637057 (+ 80us) server_negotiation.cc:226] Negotiated authn=TOKEN
0923 22:12:25.637174 (+ 117us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 22:12:25.637657 (+ 483us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 22:12:25.638478 (+ 821us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 22:12:25.638668 (+ 190us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 22:12:25.638740 (+ 72us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 22:12:25.638919 (+ 179us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0923 22:12:25.639022 (+ 103us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0923 22:12:25.639105 (+ 83us) server_negotiation.cc:1035] Waiting for connection context
0923 22:12:25.639164 (+ 59us) server_negotiation.cc:299] Negotiation successful
0923 22:12:25.639182 (+ 18us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":23}
I20260923 22:12:25.639282 3225 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 22:12:25.636738 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.135.193:44317 (local address 127.0.0.1:42512)
0923 22:12:25.636817 (+ 79us) negotiation.cc:107] Waiting for socket to connect
0923 22:12:25.636828 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0923 22:12:25.636877 (+ 49us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0923 22:12:25.637063 (+ 186us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0923 22:12:25.637064 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0923 22:12:25.637065 (+ 1us) client_negotiation.cc:189] Negotiated authn=TOKEN
0923 22:12:25.637110 (+ 45us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 22:12:25.637113 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 22:12:25.637745 (+ 632us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 22:12:25.637747 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 22:12:25.638396 (+ 649us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 22:12:25.638399 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 22:12:25.638741 (+ 342us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 22:12:25.638742 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 22:12:25.638807 (+ 65us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 22:12:25.638824 (+ 17us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0923 22:12:25.639093 (+ 269us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0923 22:12:25.639103 (+ 10us) client_negotiation.cc:769] Sending connection context
0923 22:12:25.639158 (+ 55us) client_negotiation.cc:240] Negotiation successful
0923 22:12:25.639202 (+ 44us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":41}
I20260923 22:12:25.640342 3120 raft_consensus.cc:1261] T 1aaba0de8aea4efda9a1b998152811d1 P e067e9390e544580989450a94afe6ca4 [term 1 FOLLOWER]: Refusing update from remote peer 88a6b9e139ed41b49b48710b7d8658de: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260923 22:12:25.640419 2979 raft_consensus.cc:1261] T 1aaba0de8aea4efda9a1b998152811d1 P a4124d4f2ad348d09b2f8903f9f35ba5 [term 1 FOLLOWER]: Refusing update from remote peer 88a6b9e139ed41b49b48710b7d8658de: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260923 22:12:25.640599 3218 consensus_queue.cc:1035] T 1aaba0de8aea4efda9a1b998152811d1 P 88a6b9e139ed41b49b48710b7d8658de [LEADER]: Connected to new peer: Peer: permanent_uuid: "e067e9390e544580989450a94afe6ca4" member_type: VOTER last_known_addr { host: "127.1.135.195" port: 44437 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260923 22:12:25.640728 3195 consensus_queue.cc:1035] T 1aaba0de8aea4efda9a1b998152811d1 P 88a6b9e139ed41b49b48710b7d8658de [LEADER]: Connected to new peer: Peer: permanent_uuid: "a4124d4f2ad348d09b2f8903f9f35ba5" member_type: VOTER last_known_addr { host: "127.1.135.194" port: 36581 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260923 22:12:25.641258 3196 tablet_replica.cc:439] T 1aaba0de8aea4efda9a1b998152811d1 P a4124d4f2ad348d09b2f8903f9f35ba5: TxnStatusTablet state changed. Reason: New leader 88a6b9e139ed41b49b48710b7d8658de. Latest consensus state: current_term: 1 leader_uuid: "88a6b9e139ed41b49b48710b7d8658de" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "88a6b9e139ed41b49b48710b7d8658de" member_type: VOTER last_known_addr { host: "127.1.135.193" port: 44317 } } peers { permanent_uuid: "a4124d4f2ad348d09b2f8903f9f35ba5" member_type: VOTER last_known_addr { host: "127.1.135.194" port: 36581 } } peers { permanent_uuid: "e067e9390e544580989450a94afe6ca4" member_type: VOTER last_known_addr { host: "127.1.135.195" port: 44437 } } }
I20260923 22:12:25.641348 3196 tablet_replica.cc:442] T 1aaba0de8aea4efda9a1b998152811d1 P a4124d4f2ad348d09b2f8903f9f35ba5: This TxnStatusTablet replica's current role is: FOLLOWER
I20260923 22:12:25.641343 3197 tablet_replica.cc:439] T 1aaba0de8aea4efda9a1b998152811d1 P e067e9390e544580989450a94afe6ca4: TxnStatusTablet state changed. Reason: New leader 88a6b9e139ed41b49b48710b7d8658de. Latest consensus state: current_term: 1 leader_uuid: "88a6b9e139ed41b49b48710b7d8658de" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "88a6b9e139ed41b49b48710b7d8658de" member_type: VOTER last_known_addr { host: "127.1.135.193" port: 44317 } } peers { permanent_uuid: "a4124d4f2ad348d09b2f8903f9f35ba5" member_type: VOTER last_known_addr { host: "127.1.135.194" port: 36581 } } peers { permanent_uuid: "e067e9390e544580989450a94afe6ca4" member_type: VOTER last_known_addr { host: "127.1.135.195" port: 44437 } } }
I20260923 22:12:25.641424 3197 tablet_replica.cc:442] T 1aaba0de8aea4efda9a1b998152811d1 P e067e9390e544580989450a94afe6ca4: This TxnStatusTablet replica's current role is: FOLLOWER
I20260923 22:12:25.642242 3195 tablet_replica.cc:439] T 1aaba0de8aea4efda9a1b998152811d1 P 88a6b9e139ed41b49b48710b7d8658de: TxnStatusTablet state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "88a6b9e139ed41b49b48710b7d8658de" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "88a6b9e139ed41b49b48710b7d8658de" member_type: VOTER last_known_addr { host: "127.1.135.193" port: 44317 } } peers { permanent_uuid: "a4124d4f2ad348d09b2f8903f9f35ba5" member_type: VOTER last_known_addr { host: "127.1.135.194" port: 36581 } } peers { permanent_uuid: "e067e9390e544580989450a94afe6ca4" member_type: VOTER last_known_addr { host: "127.1.135.195" port: 44437 } } }
I20260923 22:12:25.642318 3195 tablet_replica.cc:442] T 1aaba0de8aea4efda9a1b998152811d1 P 88a6b9e139ed41b49b48710b7d8658de: This TxnStatusTablet replica's current role is: LEADER
I20260923 22:12:25.642500 3218 tablet_replica.cc:439] T 1aaba0de8aea4efda9a1b998152811d1 P 88a6b9e139ed41b49b48710b7d8658de: TxnStatusTablet state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "88a6b9e139ed41b49b48710b7d8658de" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "88a6b9e139ed41b49b48710b7d8658de" member_type: VOTER last_known_addr { host: "127.1.135.193" port: 44317 } } peers { permanent_uuid: "a4124d4f2ad348d09b2f8903f9f35ba5" member_type: VOTER last_known_addr { host: "127.1.135.194" port: 36581 } } peers { permanent_uuid: "e067e9390e544580989450a94afe6ca4" member_type: VOTER last_known_addr { host: "127.1.135.195" port: 44437 } } }
I20260923 22:12:25.642570 3218 tablet_replica.cc:442] T 1aaba0de8aea4efda9a1b998152811d1 P 88a6b9e139ed41b49b48710b7d8658de: This TxnStatusTablet replica's current role is: LEADER
I20260923 22:12:25.642540 3203 mvcc.cc:205] Tried to move back new op lower bound from 7332665530940903424 to 7332665530862112768. Current Snapshot: MvccSnapshot[applied={T|T < 7332665530940903424 or (T in {7332665530940903424})}]
I20260923 22:12:25.642710 3197 tablet_replica.cc:439] T 1aaba0de8aea4efda9a1b998152811d1 P e067e9390e544580989450a94afe6ca4: TxnStatusTablet state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "88a6b9e139ed41b49b48710b7d8658de" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "88a6b9e139ed41b49b48710b7d8658de" member_type: VOTER last_known_addr { host: "127.1.135.193" port: 44317 } } peers { permanent_uuid: "a4124d4f2ad348d09b2f8903f9f35ba5" member_type: VOTER last_known_addr { host: "127.1.135.194" port: 36581 } } peers { permanent_uuid: "e067e9390e544580989450a94afe6ca4" member_type: VOTER last_known_addr { host: "127.1.135.195" port: 44437 } } }
I20260923 22:12:25.642794 3197 tablet_replica.cc:442] T 1aaba0de8aea4efda9a1b998152811d1 P e067e9390e544580989450a94afe6ca4: This TxnStatusTablet replica's current role is: FOLLOWER
I20260923 22:12:25.643695 3196 tablet_replica.cc:439] T 1aaba0de8aea4efda9a1b998152811d1 P a4124d4f2ad348d09b2f8903f9f35ba5: TxnStatusTablet state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "88a6b9e139ed41b49b48710b7d8658de" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "88a6b9e139ed41b49b48710b7d8658de" member_type: VOTER last_known_addr { host: "127.1.135.193" port: 44317 } } peers { permanent_uuid: "a4124d4f2ad348d09b2f8903f9f35ba5" member_type: VOTER last_known_addr { host: "127.1.135.194" port: 36581 } } peers { permanent_uuid: "e067e9390e544580989450a94afe6ca4" member_type: VOTER last_known_addr { host: "127.1.135.195" port: 44437 } } }
I20260923 22:12:25.643775 3196 tablet_replica.cc:442] T 1aaba0de8aea4efda9a1b998152811d1 P a4124d4f2ad348d09b2f8903f9f35ba5: This TxnStatusTablet replica's current role is: FOLLOWER
I20260923 22:12:25.646600 3186 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 22:12:25.644145 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42524 (local address 127.1.135.193:44317)
0923 22:12:25.644179 (+ 34us) server_negotiation.cc:206] Beginning negotiation
0923 22:12:25.644181 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0923 22:12:25.644207 (+ 26us) server_negotiation.cc:407] Connection header received
0923 22:12:25.644350 (+ 143us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 22:12:25.644351 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 22:12:25.644360 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 22:12:25.644408 (+ 48us) server_negotiation.cc:226] Negotiated authn=TOKEN
0923 22:12:25.644547 (+ 139us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 22:12:25.645276 (+ 729us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 22:12:25.645941 (+ 665us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 22:12:25.646232 (+ 291us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 22:12:25.646313 (+ 81us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 22:12:25.646405 (+ 92us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0923 22:12:25.646433 (+ 28us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0923 22:12:25.646479 (+ 46us) server_negotiation.cc:1035] Waiting for connection context
0923 22:12:25.646515 (+ 36us) server_negotiation.cc:299] Negotiation successful
0923 22:12:25.646534 (+ 19us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":17}
I20260923 22:12:25.651240 3186 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 22:12:25.649002 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.135.193:44901 (local address 127.1.135.193:44317)
0923 22:12:25.649042 (+ 40us) server_negotiation.cc:206] Beginning negotiation
0923 22:12:25.649044 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0923 22:12:25.649146 (+ 102us) server_negotiation.cc:407] Connection header received
0923 22:12:25.649213 (+ 67us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 22:12:25.649214 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 22:12:25.649221 (+ 7us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 22:12:25.649270 (+ 49us) server_negotiation.cc:226] Negotiated authn=TOKEN
0923 22:12:25.649392 (+ 122us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 22:12:25.649815 (+ 423us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 22:12:25.650559 (+ 744us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 22:12:25.650738 (+ 179us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 22:12:25.650818 (+ 80us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 22:12:25.650986 (+ 168us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0923 22:12:25.651019 (+ 33us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0923 22:12:25.651079 (+ 60us) server_negotiation.cc:1035] Waiting for connection context
0923 22:12:25.651145 (+ 66us) server_negotiation.cc:299] Negotiation successful
0923 22:12:25.651166 (+ 21us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":3,"server-negotiator.queue_time_us":22}
I20260923 22:12:25.651563 3237 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 22:12:25.648942 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.135.193:44317 (local address 127.1.135.193:44901)
0923 22:12:25.649084 (+ 142us) negotiation.cc:107] Waiting for socket to connect
0923 22:12:25.649092 (+ 8us) client_negotiation.cc:174] Beginning negotiation
0923 22:12:25.649159 (+ 67us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0923 22:12:25.649273 (+ 114us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0923 22:12:25.649274 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0923 22:12:25.649275 (+ 1us) client_negotiation.cc:189] Negotiated authn=TOKEN
0923 22:12:25.649329 (+ 54us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 22:12:25.649330 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 22:12:25.649903 (+ 573us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 22:12:25.649903 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 22:12:25.650472 (+ 569us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 22:12:25.650473 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 22:12:25.650820 (+ 347us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 22:12:25.650821 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 22:12:25.650887 (+ 66us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 22:12:25.650898 (+ 11us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0923 22:12:25.651092 (+ 194us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0923 22:12:25.651093 (+ 1us) client_negotiation.cc:769] Sending connection context
0923 22:12:25.651141 (+ 48us) client_negotiation.cc:240] Negotiation successful
0923 22:12:25.651182 (+ 41us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":104,"mutex_wait_us":1,"thread_start_us":67,"threads_started":1}
I20260923 22:12:25.653589 3120 raft_consensus.cc:1261] T 4ff07aeb859349d49dafb0c1c4d4330e P e067e9390e544580989450a94afe6ca4 [term 1 FOLLOWER]: Refusing update from remote peer 88a6b9e139ed41b49b48710b7d8658de: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260923 22:12:25.653622 2979 raft_consensus.cc:1261] T 4ff07aeb859349d49dafb0c1c4d4330e P a4124d4f2ad348d09b2f8903f9f35ba5 [term 1 FOLLOWER]: Refusing update from remote peer 88a6b9e139ed41b49b48710b7d8658de: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260923 22:12:25.653777 3238 consensus_queue.cc:1035] T 4ff07aeb859349d49dafb0c1c4d4330e P 88a6b9e139ed41b49b48710b7d8658de [LEADER]: Connected to new peer: Peer: permanent_uuid: "e067e9390e544580989450a94afe6ca4" member_type: VOTER last_known_addr { host: "127.1.135.195" port: 44437 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260923 22:12:25.653867 3218 consensus_queue.cc:1035] T 4ff07aeb859349d49dafb0c1c4d4330e P 88a6b9e139ed41b49b48710b7d8658de [LEADER]: Connected to new peer: Peer: permanent_uuid: "a4124d4f2ad348d09b2f8903f9f35ba5" member_type: VOTER last_known_addr { host: "127.1.135.194" port: 36581 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260923 22:12:25.655397 3202 mvcc.cc:205] Tried to move back new op lower bound from 7332665530995879936 to 7332665530177040384. Current Snapshot: MvccSnapshot[applied={T|T < 7332665530995879936}]
I20260923 22:12:25.667675 2681 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:56362:
table { table_name: "test-table" } modify_external_catalogs: true
I20260923 22:12:25.667769 2681 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:56362:
table { table_name: "test-table" } modify_external_catalogs: true
I20260923 22:12:25.668360 2681 catalog_manager.cc:5782] T 00000000000000000000000000000000 P f63d6b7613c94887b75f55a19fcea635: Sending DeleteTablet for 3 replicas of tablet 4ff07aeb859349d49dafb0c1c4d4330e
I20260923 22:12:25.668608 3100 tablet_service.cc:1514] Processing DeleteTablet for tablet 4ff07aeb859349d49dafb0c1c4d4330e with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-23 22:12:25 UTC) from {username='kudu', principal='kudu/127.1.135.254@KRBTEST.COM'} at 127.0.0.1:35644
I20260923 22:12:25.668706 2960 tablet_service.cc:1514] Processing DeleteTablet for tablet 4ff07aeb859349d49dafb0c1c4d4330e with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-23 22:12:25 UTC) from {username='kudu', principal='kudu/127.1.135.254@KRBTEST.COM'} at 127.0.0.1:47168
I20260923 22:12:25.668712 3212 tablet_replica.cc:330] T 4ff07aeb859349d49dafb0c1c4d4330e P e067e9390e544580989450a94afe6ca4: stopping tablet replica
I20260923 22:12:25.668774 3212 raft_consensus.cc:2229] T 4ff07aeb859349d49dafb0c1c4d4330e P e067e9390e544580989450a94afe6ca4 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260923 22:12:25.668785 3213 tablet_replica.cc:330] T 4ff07aeb859349d49dafb0c1c4d4330e P a4124d4f2ad348d09b2f8903f9f35ba5: stopping tablet replica
I20260923 22:12:25.668812 3212 raft_consensus.cc:2258] T 4ff07aeb859349d49dafb0c1c4d4330e P e067e9390e544580989450a94afe6ca4 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260923 22:12:25.668829 3213 raft_consensus.cc:2229] T 4ff07aeb859349d49dafb0c1c4d4330e P a4124d4f2ad348d09b2f8903f9f35ba5 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260923 22:12:25.668860 3213 raft_consensus.cc:2258] T 4ff07aeb859349d49dafb0c1c4d4330e P a4124d4f2ad348d09b2f8903f9f35ba5 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260923 22:12:25.669194 3213 ts_tablet_manager.cc:1905] T 4ff07aeb859349d49dafb0c1c4d4330e P a4124d4f2ad348d09b2f8903f9f35ba5: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260923 22:12:25.669574 2819 tablet_service.cc:1514] Processing DeleteTablet for tablet 4ff07aeb859349d49dafb0c1c4d4330e with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-23 22:12:25 UTC) from {username='kudu', principal='kudu/127.1.135.254@KRBTEST.COM'} at 127.0.0.1:42500
I20260923 22:12:25.669662 3211 tablet_replica.cc:330] T 4ff07aeb859349d49dafb0c1c4d4330e P 88a6b9e139ed41b49b48710b7d8658de: stopping tablet replica
I20260923 22:12:25.669706 3211 raft_consensus.cc:2229] T 4ff07aeb859349d49dafb0c1c4d4330e P 88a6b9e139ed41b49b48710b7d8658de [term 1 LEADER]: Raft consensus shutting down.
I20260923 22:12:25.669787 3211 raft_consensus.cc:2258] T 4ff07aeb859349d49dafb0c1c4d4330e P 88a6b9e139ed41b49b48710b7d8658de [term 1 FOLLOWER]: Raft consensus is shut down!
I20260923 22:12:25.670073 3211 ts_tablet_manager.cc:1905] T 4ff07aeb859349d49dafb0c1c4d4330e P 88a6b9e139ed41b49b48710b7d8658de: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260923 22:12:25.671038 3213 ts_tablet_manager.cc:1918] T 4ff07aeb859349d49dafb0c1c4d4330e P a4124d4f2ad348d09b2f8903f9f35ba5: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260923 22:12:25.671108 3213 log.cc:1196] T 4ff07aeb859349d49dafb0c1c4d4330e P a4124d4f2ad348d09b2f8903f9f35ba5: Deleting WAL directory at /tmp/dist-test-taskRiQe_a/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790201539513534-1567-0/minicluster-data/ts-1/wal/wals/4ff07aeb859349d49dafb0c1c4d4330e
I20260923 22:12:25.671314 3213 ts_tablet_manager.cc:1939] T 4ff07aeb859349d49dafb0c1c4d4330e P a4124d4f2ad348d09b2f8903f9f35ba5: Deleting consensus metadata
I20260923 22:12:25.671605 2668 catalog_manager.cc:4841] TS a4124d4f2ad348d09b2f8903f9f35ba5 (127.1.135.194:36581): tablet 4ff07aeb859349d49dafb0c1c4d4330e (table test-table [id=82e30eab0daf47e2bcc79e1dc71f9e22]) successfully deleted
I20260923 22:12:25.671996 3211 ts_tablet_manager.cc:1918] T 4ff07aeb859349d49dafb0c1c4d4330e P 88a6b9e139ed41b49b48710b7d8658de: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260923 22:12:25.672055 3211 log.cc:1196] T 4ff07aeb859349d49dafb0c1c4d4330e P 88a6b9e139ed41b49b48710b7d8658de: Deleting WAL directory at /tmp/dist-test-taskRiQe_a/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790201539513534-1567-0/minicluster-data/ts-0/wal/wals/4ff07aeb859349d49dafb0c1c4d4330e
I20260923 22:12:25.672293 3211 ts_tablet_manager.cc:1939] T 4ff07aeb859349d49dafb0c1c4d4330e P 88a6b9e139ed41b49b48710b7d8658de: Deleting consensus metadata
I20260923 22:12:25.672601 2670 catalog_manager.cc:4841] TS 88a6b9e139ed41b49b48710b7d8658de (127.1.135.193:44317): tablet 4ff07aeb859349d49dafb0c1c4d4330e (table test-table [id=82e30eab0daf47e2bcc79e1dc71f9e22]) successfully deleted
I20260923 22:12:25.672753 3212 ts_tablet_manager.cc:1905] T 4ff07aeb859349d49dafb0c1c4d4330e P e067e9390e544580989450a94afe6ca4: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260923 22:12:25.674234 3212 ts_tablet_manager.cc:1918] T 4ff07aeb859349d49dafb0c1c4d4330e P e067e9390e544580989450a94afe6ca4: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260923 22:12:25.674294 3212 log.cc:1196] T 4ff07aeb859349d49dafb0c1c4d4330e P e067e9390e544580989450a94afe6ca4: Deleting WAL directory at /tmp/dist-test-taskRiQe_a/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790201539513534-1567-0/minicluster-data/ts-2/wal/wals/4ff07aeb859349d49dafb0c1c4d4330e
I20260923 22:12:25.674492 3212 ts_tablet_manager.cc:1939] T 4ff07aeb859349d49dafb0c1c4d4330e P e067e9390e544580989450a94afe6ca4: Deleting consensus metadata
I20260923 22:12:25.674819 2667 catalog_manager.cc:4841] TS e067e9390e544580989450a94afe6ca4 (127.1.135.195:44437): tablet 4ff07aeb859349d49dafb0c1c4d4330e (table test-table [id=82e30eab0daf47e2bcc79e1dc71f9e22]) successfully deleted
Sep 23 22:12:25 dist-test-slave-s2fj krb5kdc[2638](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790201545, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for kudu/127.1.135.193@KRBTEST.COM
I20260923 22:12:25.683182 3186 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 22:12:25.672509 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42534 (local address 127.1.135.193:44317)
0923 22:12:25.672576 (+ 67us) server_negotiation.cc:206] Beginning negotiation
0923 22:12:25.672578 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0923 22:12:25.674782 (+ 2204us) server_negotiation.cc:407] Connection header received
0923 22:12:25.674868 (+ 86us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 22:12:25.674869 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 22:12:25.674880 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 22:12:25.674944 (+ 64us) server_negotiation.cc:226] Negotiated authn=SASL
0923 22:12:25.675593 (+ 649us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 22:12:25.676027 (+ 434us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 22:12:25.677084 (+ 1057us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 22:12:25.677250 (+ 166us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 22:12:25.677298 (+ 48us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 22:12:25.680450 (+ 3152us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0923 22:12:25.680458 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0923 22:12:25.680461 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0923 22:12:25.680472 (+ 11us) server_negotiation.cc:922] calling sasl_server_start()
0923 22:12:25.682055 (+ 1583us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0923 22:12:25.682410 (+ 355us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0923 22:12:25.682413 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0923 22:12:25.682413 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0923 22:12:25.682474 (+ 61us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0923 22:12:25.682668 (+ 194us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0923 22:12:25.682669 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0923 22:12:25.682669 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0923 22:12:25.682794 (+ 125us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0923 22:12:25.682929 (+ 135us) server_negotiation.cc:1035] Waiting for connection context
0923 22:12:25.683011 (+ 82us) server_negotiation.cc:299] Negotiation successful
0923 22:12:25.683081 (+ 70us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":32}
W20260923 22:12:25.683415 2880 server_base.cc:1116] Unauthorized access attempt to method kudu.server.GenericService.SetFlag from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:42534
I20260923 22:12:25.690404 2751 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 22:12:25.684344 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56384 (local address 127.1.135.254:40067)
0923 22:12:25.684384 (+ 40us) server_negotiation.cc:206] Beginning negotiation
0923 22:12:25.684386 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0923 22:12:25.684395 (+ 9us) server_negotiation.cc:407] Connection header received
0923 22:12:25.684458 (+ 63us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 22:12:25.684459 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 22:12:25.684466 (+ 7us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 22:12:25.684510 (+ 44us) server_negotiation.cc:226] Negotiated authn=SASL
0923 22:12:25.685119 (+ 609us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 22:12:25.685583 (+ 464us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 22:12:25.686581 (+ 998us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 22:12:25.686749 (+ 168us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 22:12:25.686796 (+ 47us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 22:12:25.688010 (+ 1214us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0923 22:12:25.688011 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0923 22:12:25.688012 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0923 22:12:25.688017 (+ 5us) server_negotiation.cc:922] calling sasl_server_start()
0923 22:12:25.689405 (+ 1388us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0923 22:12:25.689784 (+ 379us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0923 22:12:25.689785 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0923 22:12:25.689785 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0923 22:12:25.689830 (+ 45us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0923 22:12:25.690012 (+ 182us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0923 22:12:25.690012 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0923 22:12:25.690012 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0923 22:12:25.690099 (+ 87us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0923 22:12:25.690167 (+ 68us) server_negotiation.cc:1035] Waiting for connection context
0923 22:12:25.690302 (+ 135us) server_negotiation.cc:299] Negotiation successful
0923 22:12:25.690351 (+ 49us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":20}
W20260923 22:12:25.690723 2681 server_base.cc:1116] Unauthorized access attempt to method kudu.master.MasterService.TSHeartbeat from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:56384
I20260923 22:12:25.691396 1567 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskRiQe_a/build/release/bin/kudu with pid 2755
I20260923 22:12:25.696902 1567 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskRiQe_a/build/release/bin/kudu with pid 2895
I20260923 22:12:25.702287 1567 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskRiQe_a/build/release/bin/kudu with pid 3035
I20260923 22:12:25.706962 1567 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskRiQe_a/build/release/bin/kudu with pid 2651
2026-09-23T22:12:25Z chronyd exiting
[ OK ] SecurityITest.SmokeTestAsAuthorizedUser (2176 ms)
[ RUN ] SecurityITest.TxnSmokeWithDifferentUserTypes
Loading random data
Initializing database '/tmp/dist-test-taskRiQe_a/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790201539513534-1567-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Sep 23 22:12:25 dist-test-slave-s2fj krb5kdc[3248](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Sep 23 22:12:25 dist-test-slave-s2fj krb5kdc[3248](info): set up 2 sockets
Sep 23 22:12:25 dist-test-slave-s2fj krb5kdc[3248](info): commencing operation
krb5kdc: starting...
W20260923 22:12:26.731753 1567 mini_kdc.cc:121] Time spent starting KDC: real 1.004s user 0.000s sys 0.004s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskRiQe_a/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790201539513534-1567-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskRiQe_a/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790201539513534-1567-0/krb5kdc/test-user.keytab.
Sep 23 22:12:26 dist-test-slave-s2fj krb5kdc[3248](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790201546, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-09-23T22:12:26Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-09-23T22:12:26Z Disabled control of system clock
WARNING: no policy specified for kudu/127.1.135.254@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.1.135.254@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.1.135.254 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskRiQe_a/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790201539513534-1567-0/krb5kdc/kudu_127.1.135.254.keytab.
Entry for principal kudu/127.1.135.254 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskRiQe_a/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790201539513534-1567-0/krb5kdc/kudu_127.1.135.254.keytab.
I20260923 22:12:26.842562 1567 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskRiQe_a/build/release/bin/kudu
/tmp/dist-test-taskRiQe_a/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskRiQe_a/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790201539513534-1567-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskRiQe_a/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790201539513534-1567-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskRiQe_a/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790201539513534-1567-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskRiQe_a/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790201539513534-1567-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.1.135.254:37377
--webserver_interface=127.1.135.254
--webserver_port=0
--keytab_file=/tmp/dist-test-taskRiQe_a/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790201539513534-1567-0/krb5kdc/kudu_127.1.135.254.keytab
--principal=kudu/127.1.135.254
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.1.135.212:39525
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.1.135.254:37377
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--txn_manager_enabled=true with env {KRB5CCNAME=/tmp/dist-test-taskRiQe_a/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790201539513534-1567-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskRiQe_a/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790201539513534-1567-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskRiQe_a/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790201539513534-1567-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260923 22:12:26.919427 3261 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260923 22:12:26.919631 3261 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260923 22:12:26.919653 3261 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260923 22:12:26.920831 3261 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260923 22:12:26.920873 3261 flags.cc:425] Enabled experimental flag: --txn_manager_enabled=true
W20260923 22:12:26.920889 3261 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260923 22:12:26.920902 3261 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260923 22:12:26.920960 3261 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260923 22:12:26.920977 3261 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260923 22:12:26.922444 3261 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.1.135.212:39525
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskRiQe_a/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790201539513534-1567-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskRiQe_a/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790201539513534-1567-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.1.135.254:37377
--txn_manager_enabled=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskRiQe_a/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790201539513534-1567-0/krb5kdc/kudu_127.1.135.254.keytab
--principal=kudu/127.1.135.254
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.1.135.254:37377
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskRiQe_a/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790201539513534-1567-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.1.135.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.3261
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskRiQe_a/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790201539513534-1567-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 23 Sep 2026 20:53:37 UTC on ccycloud.quasar-rkviyx.root.comops.site
build id 4668083
I20260923 22:12:26.922688 3261 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260923 22:12:26.922912 3261 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260923 22:12:26.927724 3267 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260923 22:12:26.927788 3261 server_base.cc:1034] running on GCE node
W20260923 22:12:26.927711 3266 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260923 22:12:26.927711 3269 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260923 22:12:26.928469 3261 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260923 22:12:26.928738 3261 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260923 22:12:26.929884 3261 hybrid_clock.cc:648] HybridClock initialized: now 1790201546929869 us; error 30 us; skew 500 ppm
Sep 23 22:12:26 dist-test-slave-s2fj krb5kdc[3248](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790201546, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.135.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260923 22:12:26.932276 3261 init.cc:377] Logged in from keytab as kudu/127.1.135.254@KRBTEST.COM (short username kudu)
I20260923 22:12:26.932698 3261 webserver.cc:460] Webserver started at http://127.1.135.254:43525/ using document root <none> and password file <none>
I20260923 22:12:26.932869 3261 fs_manager.cc:359] Metadata directory not provided
I20260923 22:12:26.932915 3261 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260923 22:12:26.933000 3261 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260923 22:12:26.933983 3261 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskRiQe_a/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790201539513534-1567-0/minicluster-data/master-0/data/instance:
uuid: "24a871a68671419f89d1a0d5a2bdcd3f"
format_stamp: "Formatted at 2026-09-23 22:12:26 on dist-test-slave-s2fj"
server_key: "cd4b47a48f6a3628e450ef91d4caa4c2"
server_key_iv: "28508b36e849f2e3de15b0643cc2f411"
server_key_version: "encryptionkey@0"
I20260923 22:12:26.934265 3261 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskRiQe_a/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790201539513534-1567-0/minicluster-data/master-0/wal/instance:
uuid: "24a871a68671419f89d1a0d5a2bdcd3f"
format_stamp: "Formatted at 2026-09-23 22:12:26 on dist-test-slave-s2fj"
server_key: "cd4b47a48f6a3628e450ef91d4caa4c2"
server_key_iv: "28508b36e849f2e3de15b0643cc2f411"
server_key_version: "encryptionkey@0"
I20260923 22:12:26.935441 3261 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.001s
I20260923 22:12:26.936095 3276 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260923 22:12:26.936254 3261 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.000s
I20260923 22:12:26.936339 3261 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskRiQe_a/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790201539513534-1567-0/minicluster-data/master-0/data,/tmp/dist-test-taskRiQe_a/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790201539513534-1567-0/minicluster-data/master-0/wal
uuid: "24a871a68671419f89d1a0d5a2bdcd3f"
format_stamp: "Formatted at 2026-09-23 22:12:26 on dist-test-slave-s2fj"
server_key: "cd4b47a48f6a3628e450ef91d4caa4c2"
server_key_iv: "28508b36e849f2e3de15b0643cc2f411"
server_key_version: "encryptionkey@0"
I20260923 22:12:26.936393 3261 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskRiQe_a/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790201539513534-1567-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskRiQe_a/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790201539513534-1567-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskRiQe_a/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790201539513534-1567-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260923 22:12:26.944837 3261 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260923 22:12:26.946481 3261 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260923 22:12:26.946581 3261 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260923 22:12:26.950621 3261 rpc_server.cc:307] RPC server started. Bound to: 127.1.135.254:37377
I20260923 22:12:26.950700 3338 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.1.135.254:37377 every 8 connection(s)
I20260923 22:12:26.950989 3261 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskRiQe_a/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790201539513534-1567-0/minicluster-data/master-0/data/info.pb
I20260923 22:12:26.951637 3339 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260923 22:12:26.953851 3339 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 24a871a68671419f89d1a0d5a2bdcd3f: Bootstrap starting.
I20260923 22:12:26.954434 3339 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 24a871a68671419f89d1a0d5a2bdcd3f: Neither blocks nor log segments found. Creating new log.
I20260923 22:12:26.954715 3339 log.cc:824] T 00000000000000000000000000000000 P 24a871a68671419f89d1a0d5a2bdcd3f: Log is configured to *not* fsync() on all Append() calls
I20260923 22:12:26.955363 3339 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 24a871a68671419f89d1a0d5a2bdcd3f: No bootstrap required, opened a new log
I20260923 22:12:26.956444 1567 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskRiQe_a/build/release/bin/kudu as pid 3261
I20260923 22:12:26.956516 1567 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskRiQe_a/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790201539513534-1567-0/minicluster-data/master-0/wal/instance
I20260923 22:12:26.956678 1567 external_mini_cluster.cc:1428] Setting key e7616d8ea5401c02ce7ac5bbfee08ee8
I20260923 22:12:26.957821 3339 raft_consensus.cc:348] T 00000000000000000000000000000000 P 24a871a68671419f89d1a0d5a2bdcd3f [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "24a871a68671419f89d1a0d5a2bdcd3f" member_type: VOTER last_known_addr { host: "127.1.135.254" port: 37377 } }
I20260923 22:12:26.957978 3339 raft_consensus.cc:374] T 00000000000000000000000000000000 P 24a871a68671419f89d1a0d5a2bdcd3f [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260923 22:12:26.958040 3339 raft_consensus.cc:729] T 00000000000000000000000000000000 P 24a871a68671419f89d1a0d5a2bdcd3f [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 24a871a68671419f89d1a0d5a2bdcd3f, State: Initialized, Role: FOLLOWER
I20260923 22:12:26.958175 3339 consensus_queue.cc:260] T 00000000000000000000000000000000 P 24a871a68671419f89d1a0d5a2bdcd3f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "24a871a68671419f89d1a0d5a2bdcd3f" member_type: VOTER last_known_addr { host: "127.1.135.254" port: 37377 } }
I20260923 22:12:26.958274 3339 raft_consensus.cc:388] T 00000000000000000000000000000000 P 24a871a68671419f89d1a0d5a2bdcd3f [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260923 22:12:26.958319 3339 raft_consensus.cc:482] T 00000000000000000000000000000000 P 24a871a68671419f89d1a0d5a2bdcd3f [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260923 22:12:26.958370 3339 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 24a871a68671419f89d1a0d5a2bdcd3f [term 0 FOLLOWER]: Advancing to term 1
I20260923 22:12:26.958918 3339 raft_consensus.cc:504] T 00000000000000000000000000000000 P 24a871a68671419f89d1a0d5a2bdcd3f [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "24a871a68671419f89d1a0d5a2bdcd3f" member_type: VOTER last_known_addr { host: "127.1.135.254" port: 37377 } }
I20260923 22:12:26.959031 3339 leader_election.cc:302] T 00000000000000000000000000000000 P 24a871a68671419f89d1a0d5a2bdcd3f [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 24a871a68671419f89d1a0d5a2bdcd3f; no voters:
I20260923 22:12:26.959226 3339 leader_election.cc:288] T 00000000000000000000000000000000 P 24a871a68671419f89d1a0d5a2bdcd3f [CANDIDATE]: Term 1 election: Requested vote from peers
I20260923 22:12:26.959260 3344 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 24a871a68671419f89d1a0d5a2bdcd3f [term 1 FOLLOWER]: Leader election won for term 1
I20260923 22:12:26.959364 3344 raft_consensus.cc:686] T 00000000000000000000000000000000 P 24a871a68671419f89d1a0d5a2bdcd3f [term 1 LEADER]: Becoming Leader. State: Replica: 24a871a68671419f89d1a0d5a2bdcd3f, State: Running, Role: LEADER
I20260923 22:12:26.959517 3339 sys_catalog.cc:564] T 00000000000000000000000000000000 P 24a871a68671419f89d1a0d5a2bdcd3f [sys.catalog]: configured and running, proceeding with master startup.
I20260923 22:12:26.959563 3344 consensus_queue.cc:237] T 00000000000000000000000000000000 P 24a871a68671419f89d1a0d5a2bdcd3f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "24a871a68671419f89d1a0d5a2bdcd3f" member_type: VOTER last_known_addr { host: "127.1.135.254" port: 37377 } }
I20260923 22:12:26.959981 3345 sys_catalog.cc:455] T 00000000000000000000000000000000 P 24a871a68671419f89d1a0d5a2bdcd3f [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "24a871a68671419f89d1a0d5a2bdcd3f" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "24a871a68671419f89d1a0d5a2bdcd3f" member_type: VOTER last_known_addr { host: "127.1.135.254" port: 37377 } } }
I20260923 22:12:26.960093 3345 sys_catalog.cc:458] T 00000000000000000000000000000000 P 24a871a68671419f89d1a0d5a2bdcd3f [sys.catalog]: This master's current role is: LEADER
I20260923 22:12:26.960484 3346 sys_catalog.cc:455] T 00000000000000000000000000000000 P 24a871a68671419f89d1a0d5a2bdcd3f [sys.catalog]: SysCatalogTable state changed. Reason: New leader 24a871a68671419f89d1a0d5a2bdcd3f. Latest consensus state: current_term: 1 leader_uuid: "24a871a68671419f89d1a0d5a2bdcd3f" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "24a871a68671419f89d1a0d5a2bdcd3f" member_type: VOTER last_known_addr { host: "127.1.135.254" port: 37377 } } }
I20260923 22:12:26.960661 3346 sys_catalog.cc:458] T 00000000000000000000000000000000 P 24a871a68671419f89d1a0d5a2bdcd3f [sys.catalog]: This master's current role is: LEADER
I20260923 22:12:26.960812 3353 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260923 22:12:26.961472 3353 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260923 22:12:26.963053 3353 catalog_manager.cc:1347] Generated new cluster ID: 10d44c5f220549d0a53be265e1161258
I20260923 22:12:26.963100 3353 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
Sep 23 22:12:26 dist-test-slave-s2fj krb5kdc[3248](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790201546, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.1.135.254@KRBTEST.COM
I20260923 22:12:26.968441 3343 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 22:12:26.957183 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35730 (local address 127.1.135.254:37377)
0923 22:12:26.957379 (+ 196us) server_negotiation.cc:206] Beginning negotiation
0923 22:12:26.957387 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0923 22:12:26.957412 (+ 25us) server_negotiation.cc:407] Connection header received
0923 22:12:26.957806 (+ 394us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 22:12:26.957811 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 22:12:26.957853 (+ 42us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 22:12:26.957992 (+ 139us) server_negotiation.cc:226] Negotiated authn=SASL
0923 22:12:26.958776 (+ 784us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 22:12:26.959959 (+ 1183us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 22:12:26.961102 (+ 1143us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 22:12:26.961663 (+ 561us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 22:12:26.961765 (+ 102us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 22:12:26.964463 (+ 2698us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0923 22:12:26.964467 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0923 22:12:26.964467 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0923 22:12:26.964478 (+ 11us) server_negotiation.cc:922] calling sasl_server_start()
0923 22:12:26.966389 (+ 1911us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0923 22:12:26.966780 (+ 391us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0923 22:12:26.966783 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0923 22:12:26.966783 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0923 22:12:26.967319 (+ 536us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0923 22:12:26.967522 (+ 203us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0923 22:12:26.967523 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0923 22:12:26.967524 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0923 22:12:26.967816 (+ 292us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0923 22:12:26.967981 (+ 165us) server_negotiation.cc:1035] Waiting for connection context
0923 22:12:26.968050 (+ 69us) server_negotiation.cc:299] Negotiation successful
0923 22:12:26.968224 (+ 174us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":145,"thread_start_us":102,"threads_started":1}
I20260923 22:12:26.979104 3353 catalog_manager.cc:1370] Generated new certificate authority record
I20260923 22:12:26.979782 3353 catalog_manager.cc:1504] Loading token signing keys...
I20260923 22:12:26.983572 3353 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 24a871a68671419f89d1a0d5a2bdcd3f: Generated new TSK 0
I20260923 22:12:26.983731 3353 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.1.135.193@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.1.135.193@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.1.135.193 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskRiQe_a/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790201539513534-1567-0/krb5kdc/kudu_127.1.135.193.keytab.
Entry for principal kudu/127.1.135.193 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskRiQe_a/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790201539513534-1567-0/krb5kdc/kudu_127.1.135.193.keytab.
I20260923 22:12:27.010926 1567 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskRiQe_a/build/release/bin/kudu
/tmp/dist-test-taskRiQe_a/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskRiQe_a/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790201539513534-1567-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskRiQe_a/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790201539513534-1567-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskRiQe_a/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790201539513534-1567-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskRiQe_a/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790201539513534-1567-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.1.135.193:0
--local_ip_for_outbound_sockets=127.1.135.193
--webserver_interface=127.1.135.193
--webserver_port=0
--tserver_master_addrs=127.1.135.254:37377
--keytab_file=/tmp/dist-test-taskRiQe_a/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790201539513534-1567-0/krb5kdc/kudu_127.1.135.193.keytab
--principal=kudu/127.1.135.193
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.1.135.212:39525
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--enable_txn_system_client_init=true
--txn_keepalive_interval_ms=500 with env {KRB5CCNAME=/tmp/dist-test-taskRiQe_a/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790201539513534-1567-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskRiQe_a/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790201539513534-1567-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskRiQe_a/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790201539513534-1567-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260923 22:12:27.091224 3365 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260923 22:12:27.091456 3365 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260923 22:12:27.091496 3365 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260923 22:12:27.092618 3365 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260923 22:12:27.092681 3365 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260923 22:12:27.092718 3365 flags.cc:425] Enabled experimental flag: --txn_keepalive_interval_ms=500
W20260923 22:12:27.092768 3365 flags.cc:425] Enabled experimental flag: --enable_txn_system_client_init=true
W20260923 22:12:27.092806 3365 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.1.135.193
I20260923 22:12:27.094223 3365 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.1.135.212:39525
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskRiQe_a/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790201539513534-1567-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskRiQe_a/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790201539513534-1567-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskRiQe_a/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790201539513534-1567-0/krb5kdc/kudu_127.1.135.193.keytab
--principal=kudu/127.1.135.193
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.1.135.193:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskRiQe_a/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790201539513534-1567-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.1.135.193
--webserver_port=0
--txn_keepalive_interval_ms=500
--enable_txn_system_client_init=true
--tserver_master_addrs=127.1.135.254:37377
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.3365
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.1.135.193
--log_dir=/tmp/dist-test-taskRiQe_a/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790201539513534-1567-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 23 Sep 2026 20:53:37 UTC on ccycloud.quasar-rkviyx.root.comops.site
build id 4668083
I20260923 22:12:27.094543 3365 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260923 22:12:27.094837 3365 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260923 22:12:27.099536 3373 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260923 22:12:27.099551 3370 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260923 22:12:27.099612 3371 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260923 22:12:27.099742 3365 server_base.cc:1034] running on GCE node
I20260923 22:12:27.100021 3365 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260923 22:12:27.100308 3365 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260923 22:12:27.101472 3365 hybrid_clock.cc:648] HybridClock initialized: now 1790201547101454 us; error 30 us; skew 500 ppm
Sep 23 22:12:27 dist-test-slave-s2fj krb5kdc[3248](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790201547, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.135.193@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260923 22:12:27.103894 3365 init.cc:377] Logged in from keytab as kudu/127.1.135.193@KRBTEST.COM (short username kudu)
I20260923 22:12:27.104355 3365 webserver.cc:460] Webserver started at http://127.1.135.193:41349/ using document root <none> and password file <none>
I20260923 22:12:27.104575 3365 fs_manager.cc:359] Metadata directory not provided
I20260923 22:12:27.104674 3365 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260923 22:12:27.104841 3365 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260923 22:12:27.105830 3365 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskRiQe_a/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790201539513534-1567-0/minicluster-data/ts-0/data/instance:
uuid: "04978c60cca64c6783ac03221353dbe6"
format_stamp: "Formatted at 2026-09-23 22:12:27 on dist-test-slave-s2fj"
server_key: "f4c7e4f8349eb292d1492e037319808b"
server_key_iv: "ef0352578afb2f7ed189c93384fb3a6f"
server_key_version: "encryptionkey@0"
I20260923 22:12:27.106235 3365 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskRiQe_a/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790201539513534-1567-0/minicluster-data/ts-0/wal/instance:
uuid: "04978c60cca64c6783ac03221353dbe6"
format_stamp: "Formatted at 2026-09-23 22:12:27 on dist-test-slave-s2fj"
server_key: "f4c7e4f8349eb292d1492e037319808b"
server_key_iv: "ef0352578afb2f7ed189c93384fb3a6f"
server_key_version: "encryptionkey@0"
I20260923 22:12:27.107491 3365 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.003s
I20260923 22:12:27.108194 3380 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260923 22:12:27.108362 3365 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.001s
I20260923 22:12:27.108462 3365 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskRiQe_a/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790201539513534-1567-0/minicluster-data/ts-0/data,/tmp/dist-test-taskRiQe_a/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790201539513534-1567-0/minicluster-data/ts-0/wal
uuid: "04978c60cca64c6783ac03221353dbe6"
format_stamp: "Formatted at 2026-09-23 22:12:27 on dist-test-slave-s2fj"
server_key: "f4c7e4f8349eb292d1492e037319808b"
server_key_iv: "ef0352578afb2f7ed189c93384fb3a6f"
server_key_version: "encryptionkey@0"
I20260923 22:12:27.108546 3365 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskRiQe_a/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790201539513534-1567-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskRiQe_a/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790201539513534-1567-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskRiQe_a/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790201539513534-1567-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260923 22:12:27.151332 3365 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260923 22:12:27.153721 3365 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260923 22:12:27.153885 3365 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260923 22:12:27.154400 3365 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260923 22:12:27.154460 3365 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260923 22:12:27.154506 3365 ts_tablet_manager.cc:610] Registered 0 tablets
I20260923 22:12:27.154531 3365 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
Sep 23 22:12:27 dist-test-slave-s2fj krb5kdc[3248](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790201547, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.135.193@KRBTEST.COM for kudu/127.1.135.254@KRBTEST.COM
I20260923 22:12:27.161960 3365 rpc_server.cc:307] RPC server started. Bound to: 127.1.135.193:42577
I20260923 22:12:27.162326 3365 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskRiQe_a/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790201539513534-1567-0/minicluster-data/ts-0/data/info.pb
I20260923 22:12:27.162329 3495 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.1.135.193:42577 every 8 connection(s)
I20260923 22:12:27.164904 3343 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 22:12:27.154960 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.135.193:53927 (local address 127.1.135.254:37377)
0923 22:12:27.154999 (+ 39us) server_negotiation.cc:206] Beginning negotiation
0923 22:12:27.155001 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0923 22:12:27.155217 (+ 216us) server_negotiation.cc:407] Connection header received
0923 22:12:27.155743 (+ 526us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 22:12:27.155744 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 22:12:27.155756 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 22:12:27.155805 (+ 49us) server_negotiation.cc:226] Negotiated authn=SASL
0923 22:12:27.156474 (+ 669us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 22:12:27.156980 (+ 506us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 22:12:27.157909 (+ 929us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 22:12:27.158123 (+ 214us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 22:12:27.158210 (+ 87us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 22:12:27.161764 (+ 3554us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0923 22:12:27.161765 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0923 22:12:27.161766 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0923 22:12:27.161774 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0923 22:12:27.163366 (+ 1592us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0923 22:12:27.163840 (+ 474us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0923 22:12:27.163843 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0923 22:12:27.163843 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0923 22:12:27.163899 (+ 56us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0923 22:12:27.164216 (+ 317us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0923 22:12:27.164217 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0923 22:12:27.164217 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0923 22:12:27.164337 (+ 120us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0923 22:12:27.164413 (+ 76us) server_negotiation.cc:1035] Waiting for connection context
0923 22:12:27.164685 (+ 272us) server_negotiation.cc:299] Negotiation successful
0923 22:12:27.164807 (+ 122us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":18}
I20260923 22:12:27.165022 3389 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 22:12:27.154949 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.135.254:37377 (local address 127.1.135.193:53927)
0923 22:12:27.155099 (+ 150us) negotiation.cc:107] Waiting for socket to connect
0923 22:12:27.155133 (+ 34us) client_negotiation.cc:174] Beginning negotiation
0923 22:12:27.155632 (+ 499us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0923 22:12:27.155892 (+ 260us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0923 22:12:27.155897 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0923 22:12:27.156209 (+ 312us) client_negotiation.cc:189] Negotiated authn=SASL
0923 22:12:27.156378 (+ 169us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 22:12:27.156382 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 22:12:27.157068 (+ 686us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 22:12:27.157069 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 22:12:27.157813 (+ 744us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 22:12:27.157815 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 22:12:27.158221 (+ 406us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 22:12:27.158221 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 22:12:27.158415 (+ 194us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 22:12:27.158971 (+ 556us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0923 22:12:27.158981 (+ 10us) client_negotiation.cc:656] calling sasl_client_start()
0923 22:12:27.161653 (+ 2672us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0923 22:12:27.163467 (+ 1814us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0923 22:12:27.163473 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0923 22:12:27.163475 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0923 22:12:27.163701 (+ 226us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0923 22:12:27.163979 (+ 278us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0923 22:12:27.163980 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0923 22:12:27.163980 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0923 22:12:27.164140 (+ 160us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0923 22:12:27.164415 (+ 275us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0923 22:12:27.164422 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0923 22:12:27.164552 (+ 130us) client_negotiation.cc:769] Sending connection context
0923 22:12:27.164650 (+ 98us) client_negotiation.cc:240] Negotiation successful
0923 22:12:27.164814 (+ 164us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":114,"thread_start_us":92,"threads_started":1}
I20260923 22:12:27.165469 3496 heartbeater.cc:347] Connected to a master server at 127.1.135.254:37377
I20260923 22:12:27.165552 3496 heartbeater.cc:464] Registering TS with master...
I20260923 22:12:27.165726 3496 heartbeater.cc:511] Master 127.1.135.254:37377 requested a full tablet report, sending...
I20260923 22:12:27.166091 1567 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskRiQe_a/build/release/bin/kudu as pid 3365
I20260923 22:12:27.166184 1567 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskRiQe_a/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790201539513534-1567-0/minicluster-data/ts-0/wal/instance
I20260923 22:12:27.166268 3292 ts_manager.cc:194] Registered new tserver with Master: 04978c60cca64c6783ac03221353dbe6 (127.1.135.193:42577)
I20260923 22:12:27.166365 1567 external_mini_cluster.cc:1428] Setting key deedced21eb498b8fb6304295933aaa1
I20260923 22:12:27.167114 3292 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.1.135.193@KRBTEST.COM'} at 127.1.135.193:53927
I20260923 22:12:27.172187 3343 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 22:12:27.166534 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.135.193:51867 (local address 127.1.135.254:37377)
0923 22:12:27.166566 (+ 32us) server_negotiation.cc:206] Beginning negotiation
0923 22:12:27.166567 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0923 22:12:27.166679 (+ 112us) server_negotiation.cc:407] Connection header received
0923 22:12:27.166700 (+ 21us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 22:12:27.166701 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 22:12:27.166709 (+ 8us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 22:12:27.166771 (+ 62us) server_negotiation.cc:226] Negotiated authn=SASL
0923 22:12:27.167020 (+ 249us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 22:12:27.167698 (+ 678us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 22:12:27.168502 (+ 804us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 22:12:27.168719 (+ 217us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 22:12:27.168796 (+ 77us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 22:12:27.169671 (+ 875us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0923 22:12:27.169672 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0923 22:12:27.169672 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0923 22:12:27.169679 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0923 22:12:27.171080 (+ 1401us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0923 22:12:27.171416 (+ 336us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0923 22:12:27.171417 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0923 22:12:27.171417 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0923 22:12:27.171470 (+ 53us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0923 22:12:27.171662 (+ 192us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0923 22:12:27.171663 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0923 22:12:27.171663 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0923 22:12:27.171767 (+ 104us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0923 22:12:27.171835 (+ 68us) server_negotiation.cc:1035] Waiting for connection context
0923 22:12:27.172003 (+ 168us) server_negotiation.cc:299] Negotiation successful
0923 22:12:27.172104 (+ 101us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":17}
I20260923 22:12:27.172196 3502 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 22:12:27.166492 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.135.254:37377 (local address 127.1.135.193:51867)
0923 22:12:27.166604 (+ 112us) negotiation.cc:107] Waiting for socket to connect
0923 22:12:27.166614 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0923 22:12:27.166669 (+ 55us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0923 22:12:27.166772 (+ 103us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0923 22:12:27.166773 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0923 22:12:27.166902 (+ 129us) client_negotiation.cc:189] Negotiated authn=SASL
0923 22:12:27.166963 (+ 61us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 22:12:27.166965 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 22:12:27.167782 (+ 817us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 22:12:27.167783 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 22:12:27.168407 (+ 624us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 22:12:27.168410 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 22:12:27.168794 (+ 384us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 22:12:27.168795 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 22:12:27.168870 (+ 75us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 22:12:27.169282 (+ 412us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0923 22:12:27.169294 (+ 12us) client_negotiation.cc:656] calling sasl_client_start()
0923 22:12:27.169567 (+ 273us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0923 22:12:27.171155 (+ 1588us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0923 22:12:27.171155 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0923 22:12:27.171156 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0923 22:12:27.171322 (+ 166us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0923 22:12:27.171533 (+ 211us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0923 22:12:27.171534 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0923 22:12:27.171534 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0923 22:12:27.171590 (+ 56us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0923 22:12:27.171839 (+ 249us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0923 22:12:27.171840 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0923 22:12:27.171893 (+ 53us) client_negotiation.cc:769] Sending connection context
0923 22:12:27.171979 (+ 86us) client_negotiation.cc:240] Negotiation successful
0923 22:12:27.172096 (+ 117us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":90,"thread_start_us":65,"threads_started":1}
WARNING: no policy specified for kudu/127.1.135.194@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.1.135.194@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.1.135.194 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskRiQe_a/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790201539513534-1567-0/krb5kdc/kudu_127.1.135.194.keytab.
Entry for principal kudu/127.1.135.194 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskRiQe_a/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790201539513534-1567-0/krb5kdc/kudu_127.1.135.194.keytab.
I20260923 22:12:27.192956 1567 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskRiQe_a/build/release/bin/kudu
/tmp/dist-test-taskRiQe_a/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskRiQe_a/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790201539513534-1567-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskRiQe_a/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790201539513534-1567-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskRiQe_a/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790201539513534-1567-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskRiQe_a/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790201539513534-1567-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.1.135.194:0
--local_ip_for_outbound_sockets=127.1.135.194
--webserver_interface=127.1.135.194
--webserver_port=0
--tserver_master_addrs=127.1.135.254:37377
--keytab_file=/tmp/dist-test-taskRiQe_a/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790201539513534-1567-0/krb5kdc/kudu_127.1.135.194.keytab
--principal=kudu/127.1.135.194
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.1.135.212:39525
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--enable_txn_system_client_init=true
--txn_keepalive_interval_ms=500 with env {KRB5CCNAME=/tmp/dist-test-taskRiQe_a/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790201539513534-1567-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskRiQe_a/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790201539513534-1567-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskRiQe_a/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790201539513534-1567-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260923 22:12:27.272002 3505 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260923 22:12:27.272281 3505 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260923 22:12:27.272320 3505 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260923 22:12:27.273429 3505 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260923 22:12:27.273494 3505 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260923 22:12:27.273530 3505 flags.cc:425] Enabled experimental flag: --txn_keepalive_interval_ms=500
W20260923 22:12:27.273561 3505 flags.cc:425] Enabled experimental flag: --enable_txn_system_client_init=true
W20260923 22:12:27.273610 3505 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.1.135.194
I20260923 22:12:27.275002 3505 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.1.135.212:39525
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskRiQe_a/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790201539513534-1567-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskRiQe_a/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790201539513534-1567-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskRiQe_a/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790201539513534-1567-0/krb5kdc/kudu_127.1.135.194.keytab
--principal=kudu/127.1.135.194
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.1.135.194:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskRiQe_a/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790201539513534-1567-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.1.135.194
--webserver_port=0
--txn_keepalive_interval_ms=500
--enable_txn_system_client_init=true
--tserver_master_addrs=127.1.135.254:37377
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.3505
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.1.135.194
--log_dir=/tmp/dist-test-taskRiQe_a/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790201539513534-1567-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 23 Sep 2026 20:53:37 UTC on ccycloud.quasar-rkviyx.root.comops.site
build id 4668083
I20260923 22:12:27.275251 3505 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260923 22:12:27.275481 3505 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260923 22:12:27.280236 3513 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260923 22:12:27.280244 3511 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260923 22:12:27.280303 3505 server_base.cc:1034] running on GCE node
W20260923 22:12:27.280303 3510 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260923 22:12:27.280699 3505 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260923 22:12:27.280937 3505 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260923 22:12:27.282099 3505 hybrid_clock.cc:648] HybridClock initialized: now 1790201547282094 us; error 35 us; skew 500 ppm
Sep 23 22:12:27 dist-test-slave-s2fj krb5kdc[3248](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790201547, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.135.194@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260923 22:12:27.284523 3505 init.cc:377] Logged in from keytab as kudu/127.1.135.194@KRBTEST.COM (short username kudu)
I20260923 22:12:27.284961 3505 webserver.cc:460] Webserver started at http://127.1.135.194:38879/ using document root <none> and password file <none>
I20260923 22:12:27.285130 3505 fs_manager.cc:359] Metadata directory not provided
I20260923 22:12:27.285169 3505 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260923 22:12:27.285251 3505 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260923 22:12:27.286170 3505 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskRiQe_a/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790201539513534-1567-0/minicluster-data/ts-1/data/instance:
uuid: "d269cfa9b6874c6fb8b07b0da9f00ea5"
format_stamp: "Formatted at 2026-09-23 22:12:27 on dist-test-slave-s2fj"
server_key: "2b8a34ad33804a28b56d25556c06ed7a"
server_key_iv: "2303ad4c675d1179596bbb0ab3c2c433"
server_key_version: "encryptionkey@0"
I20260923 22:12:27.286468 3505 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskRiQe_a/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790201539513534-1567-0/minicluster-data/ts-1/wal/instance:
uuid: "d269cfa9b6874c6fb8b07b0da9f00ea5"
format_stamp: "Formatted at 2026-09-23 22:12:27 on dist-test-slave-s2fj"
server_key: "2b8a34ad33804a28b56d25556c06ed7a"
server_key_iv: "2303ad4c675d1179596bbb0ab3c2c433"
server_key_version: "encryptionkey@0"
I20260923 22:12:27.287621 3505 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.001s
I20260923 22:12:27.288291 3520 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260923 22:12:27.288487 3505 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.000s
I20260923 22:12:27.288573 3505 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskRiQe_a/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790201539513534-1567-0/minicluster-data/ts-1/data,/tmp/dist-test-taskRiQe_a/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790201539513534-1567-0/minicluster-data/ts-1/wal
uuid: "d269cfa9b6874c6fb8b07b0da9f00ea5"
format_stamp: "Formatted at 2026-09-23 22:12:27 on dist-test-slave-s2fj"
server_key: "2b8a34ad33804a28b56d25556c06ed7a"
server_key_iv: "2303ad4c675d1179596bbb0ab3c2c433"
server_key_version: "encryptionkey@0"
I20260923 22:12:27.288623 3505 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskRiQe_a/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790201539513534-1567-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskRiQe_a/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790201539513534-1567-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskRiQe_a/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790201539513534-1567-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260923 22:12:27.304433 3505 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260923 22:12:27.306134 3505 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260923 22:12:27.306248 3505 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260923 22:12:27.306757 3505 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260923 22:12:27.306798 3505 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260923 22:12:27.306821 3505 ts_tablet_manager.cc:610] Registered 0 tablets
I20260923 22:12:27.306834 3505 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
Sep 23 22:12:27 dist-test-slave-s2fj krb5kdc[3248](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790201547, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.135.194@KRBTEST.COM for kudu/127.1.135.254@KRBTEST.COM
I20260923 22:12:27.313855 3505 rpc_server.cc:307] RPC server started. Bound to: 127.1.135.194:36299
I20260923 22:12:27.313930 3635 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.1.135.194:36299 every 8 connection(s)
I20260923 22:12:27.314258 3505 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskRiQe_a/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790201539513534-1567-0/minicluster-data/ts-1/data/info.pb
I20260923 22:12:27.315977 3343 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 22:12:27.307180 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.135.194:57297 (local address 127.1.135.254:37377)
0923 22:12:27.307233 (+ 53us) server_negotiation.cc:206] Beginning negotiation
0923 22:12:27.307236 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0923 22:12:27.307418 (+ 182us) server_negotiation.cc:407] Connection header received
0923 22:12:27.307916 (+ 498us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 22:12:27.307917 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 22:12:27.307930 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 22:12:27.307975 (+ 45us) server_negotiation.cc:226] Negotiated authn=SASL
0923 22:12:27.308640 (+ 665us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 22:12:27.309125 (+ 485us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 22:12:27.310115 (+ 990us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 22:12:27.310368 (+ 253us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 22:12:27.310464 (+ 96us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 22:12:27.313319 (+ 2855us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0923 22:12:27.313320 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0923 22:12:27.313321 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0923 22:12:27.313327 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0923 22:12:27.314911 (+ 1584us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0923 22:12:27.315251 (+ 340us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0923 22:12:27.315251 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0923 22:12:27.315252 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0923 22:12:27.315301 (+ 49us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0923 22:12:27.315477 (+ 176us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0923 22:12:27.315477 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0923 22:12:27.315478 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0923 22:12:27.315572 (+ 94us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0923 22:12:27.315622 (+ 50us) server_negotiation.cc:1035] Waiting for connection context
0923 22:12:27.315847 (+ 225us) server_negotiation.cc:299] Negotiation successful
0923 22:12:27.315917 (+ 70us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":29}
I20260923 22:12:27.316174 3529 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 22:12:27.307138 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.135.254:37377 (local address 127.1.135.194:57297)
0923 22:12:27.307286 (+ 148us) negotiation.cc:107] Waiting for socket to connect
0923 22:12:27.307331 (+ 45us) client_negotiation.cc:174] Beginning negotiation
0923 22:12:27.307815 (+ 484us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0923 22:12:27.308038 (+ 223us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0923 22:12:27.308041 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0923 22:12:27.308364 (+ 323us) client_negotiation.cc:189] Negotiated authn=SASL
0923 22:12:27.308547 (+ 183us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 22:12:27.308551 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 22:12:27.309216 (+ 665us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 22:12:27.309217 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 22:12:27.310027 (+ 810us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 22:12:27.310028 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 22:12:27.310474 (+ 446us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 22:12:27.310475 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 22:12:27.310618 (+ 143us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 22:12:27.311097 (+ 479us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0923 22:12:27.311106 (+ 9us) client_negotiation.cc:656] calling sasl_client_start()
0923 22:12:27.313211 (+ 2105us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0923 22:12:27.314981 (+ 1770us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0923 22:12:27.314984 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0923 22:12:27.314986 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0923 22:12:27.315182 (+ 196us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0923 22:12:27.315341 (+ 159us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0923 22:12:27.315341 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0923 22:12:27.315342 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0923 22:12:27.315426 (+ 84us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0923 22:12:27.315629 (+ 203us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0923 22:12:27.315633 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0923 22:12:27.315735 (+ 102us) client_negotiation.cc:769] Sending connection context
0923 22:12:27.315812 (+ 77us) client_negotiation.cc:240] Negotiation successful
0923 22:12:27.315931 (+ 119us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":110,"thread_start_us":81,"threads_started":1}
I20260923 22:12:27.316601 3636 heartbeater.cc:347] Connected to a master server at 127.1.135.254:37377
I20260923 22:12:27.316679 3636 heartbeater.cc:464] Registering TS with master...
I20260923 22:12:27.316854 3636 heartbeater.cc:511] Master 127.1.135.254:37377 requested a full tablet report, sending...
I20260923 22:12:27.317183 3292 ts_manager.cc:194] Registered new tserver with Master: d269cfa9b6874c6fb8b07b0da9f00ea5 (127.1.135.194:36299)
I20260923 22:12:27.317345 1567 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskRiQe_a/build/release/bin/kudu as pid 3505
I20260923 22:12:27.317427 1567 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskRiQe_a/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790201539513534-1567-0/minicluster-data/ts-1/wal/instance
I20260923 22:12:27.317601 1567 external_mini_cluster.cc:1428] Setting key 01a01e8719aa60029f470f7f462cc750
I20260923 22:12:27.317674 3292 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.1.135.194@KRBTEST.COM'} at 127.1.135.194:57297
I20260923 22:12:27.323444 3642 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 22:12:27.317582 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.135.254:37377 (local address 127.1.135.194:45427)
0923 22:12:27.317794 (+ 212us) negotiation.cc:107] Waiting for socket to connect
0923 22:12:27.317804 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0923 22:12:27.317873 (+ 69us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0923 22:12:27.318004 (+ 131us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0923 22:12:27.318004 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0923 22:12:27.318196 (+ 192us) client_negotiation.cc:189] Negotiated authn=SASL
0923 22:12:27.318263 (+ 67us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 22:12:27.318264 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 22:12:27.318927 (+ 663us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 22:12:27.318928 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 22:12:27.319611 (+ 683us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 22:12:27.319613 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 22:12:27.319954 (+ 341us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 22:12:27.319956 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 22:12:27.320036 (+ 80us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 22:12:27.320427 (+ 391us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0923 22:12:27.320432 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0923 22:12:27.320752 (+ 320us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0923 22:12:27.322475 (+ 1723us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0923 22:12:27.322476 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0923 22:12:27.322476 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0923 22:12:27.322625 (+ 149us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0923 22:12:27.322831 (+ 206us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0923 22:12:27.322832 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0923 22:12:27.322832 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0923 22:12:27.322877 (+ 45us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0923 22:12:27.323208 (+ 331us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0923 22:12:27.323209 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0923 22:12:27.323252 (+ 43us) client_negotiation.cc:769] Sending connection context
0923 22:12:27.323301 (+ 49us) client_negotiation.cc:240] Negotiation successful
0923 22:12:27.323369 (+ 68us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":186,"thread_start_us":59,"threads_started":1}
I20260923 22:12:27.323549 3343 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 22:12:27.317628 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.135.194:45427 (local address 127.1.135.254:37377)
0923 22:12:27.317664 (+ 36us) server_negotiation.cc:206] Beginning negotiation
0923 22:12:27.317666 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0923 22:12:27.317850 (+ 184us) server_negotiation.cc:407] Connection header received
0923 22:12:27.317936 (+ 86us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 22:12:27.317937 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 22:12:27.317946 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 22:12:27.317998 (+ 52us) server_negotiation.cc:226] Negotiated authn=SASL
0923 22:12:27.318380 (+ 382us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 22:12:27.318829 (+ 449us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 22:12:27.319694 (+ 865us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 22:12:27.319881 (+ 187us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 22:12:27.319937 (+ 56us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 22:12:27.320835 (+ 898us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0923 22:12:27.320836 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0923 22:12:27.320836 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0923 22:12:27.320843 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0923 22:12:27.322321 (+ 1478us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0923 22:12:27.322697 (+ 376us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0923 22:12:27.322699 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0923 22:12:27.322699 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0923 22:12:27.322762 (+ 63us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0923 22:12:27.322936 (+ 174us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0923 22:12:27.322936 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0923 22:12:27.322937 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0923 22:12:27.323138 (+ 201us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0923 22:12:27.323217 (+ 79us) server_negotiation.cc:1035] Waiting for connection context
0923 22:12:27.323330 (+ 113us) server_negotiation.cc:299] Negotiation successful
0923 22:12:27.323417 (+ 87us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":20}
WARNING: no policy specified for kudu/127.1.135.195@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.1.135.195@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.1.135.195 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskRiQe_a/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790201539513534-1567-0/krb5kdc/kudu_127.1.135.195.keytab.
Entry for principal kudu/127.1.135.195 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskRiQe_a/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790201539513534-1567-0/krb5kdc/kudu_127.1.135.195.keytab.
I20260923 22:12:27.340620 1567 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskRiQe_a/build/release/bin/kudu
/tmp/dist-test-taskRiQe_a/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskRiQe_a/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790201539513534-1567-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskRiQe_a/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790201539513534-1567-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskRiQe_a/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790201539513534-1567-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskRiQe_a/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790201539513534-1567-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.1.135.195:0
--local_ip_for_outbound_sockets=127.1.135.195
--webserver_interface=127.1.135.195
--webserver_port=0
--tserver_master_addrs=127.1.135.254:37377
--keytab_file=/tmp/dist-test-taskRiQe_a/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790201539513534-1567-0/krb5kdc/kudu_127.1.135.195.keytab
--principal=kudu/127.1.135.195
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.1.135.212:39525
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--enable_txn_system_client_init=true
--txn_keepalive_interval_ms=500 with env {KRB5CCNAME=/tmp/dist-test-taskRiQe_a/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790201539513534-1567-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskRiQe_a/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790201539513534-1567-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskRiQe_a/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790201539513534-1567-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260923 22:12:27.419040 3645 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260923 22:12:27.419229 3645 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260923 22:12:27.419253 3645 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260923 22:12:27.420323 3645 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260923 22:12:27.420356 3645 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260923 22:12:27.420375 3645 flags.cc:425] Enabled experimental flag: --txn_keepalive_interval_ms=500
W20260923 22:12:27.420387 3645 flags.cc:425] Enabled experimental flag: --enable_txn_system_client_init=true
W20260923 22:12:27.420403 3645 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.1.135.195
I20260923 22:12:27.421687 3645 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.1.135.212:39525
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskRiQe_a/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790201539513534-1567-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskRiQe_a/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790201539513534-1567-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskRiQe_a/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790201539513534-1567-0/krb5kdc/kudu_127.1.135.195.keytab
--principal=kudu/127.1.135.195
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.1.135.195:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskRiQe_a/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790201539513534-1567-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.1.135.195
--webserver_port=0
--txn_keepalive_interval_ms=500
--enable_txn_system_client_init=true
--tserver_master_addrs=127.1.135.254:37377
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.3645
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.1.135.195
--log_dir=/tmp/dist-test-taskRiQe_a/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790201539513534-1567-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 23 Sep 2026 20:53:37 UTC on ccycloud.quasar-rkviyx.root.comops.site
build id 4668083
I20260923 22:12:27.421856 3645 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260923 22:12:27.422016 3645 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260923 22:12:27.426716 3651 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260923 22:12:27.426764 3645 server_base.cc:1034] running on GCE node
W20260923 22:12:27.426935 3650 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260923 22:12:27.427069 3653 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260923 22:12:27.427281 3645 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260923 22:12:27.427592 3645 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260923 22:12:27.428741 3645 hybrid_clock.cc:648] HybridClock initialized: now 1790201547428732 us; error 26 us; skew 500 ppm
Sep 23 22:12:27 dist-test-slave-s2fj krb5kdc[3248](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790201547, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.135.195@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260923 22:12:27.431069 3645 init.cc:377] Logged in from keytab as kudu/127.1.135.195@KRBTEST.COM (short username kudu)
I20260923 22:12:27.431490 3645 webserver.cc:460] Webserver started at http://127.1.135.195:34039/ using document root <none> and password file <none>
I20260923 22:12:27.431670 3645 fs_manager.cc:359] Metadata directory not provided
I20260923 22:12:27.431708 3645 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260923 22:12:27.431792 3645 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260923 22:12:27.432765 3645 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskRiQe_a/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790201539513534-1567-0/minicluster-data/ts-2/data/instance:
uuid: "ece2a20b74874eb7acf32b82a1691f97"
format_stamp: "Formatted at 2026-09-23 22:12:27 on dist-test-slave-s2fj"
server_key: "5cb85ba4bb1e518685964419263098c0"
server_key_iv: "eac56ea4387c1055707922bea311d65c"
server_key_version: "encryptionkey@0"
I20260923 22:12:27.433048 3645 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskRiQe_a/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790201539513534-1567-0/minicluster-data/ts-2/wal/instance:
uuid: "ece2a20b74874eb7acf32b82a1691f97"
format_stamp: "Formatted at 2026-09-23 22:12:27 on dist-test-slave-s2fj"
server_key: "5cb85ba4bb1e518685964419263098c0"
server_key_iv: "eac56ea4387c1055707922bea311d65c"
server_key_version: "encryptionkey@0"
I20260923 22:12:27.434185 3645 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.002s
I20260923 22:12:27.434834 3660 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260923 22:12:27.435022 3645 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.001s
I20260923 22:12:27.435098 3645 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskRiQe_a/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790201539513534-1567-0/minicluster-data/ts-2/data,/tmp/dist-test-taskRiQe_a/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790201539513534-1567-0/minicluster-data/ts-2/wal
uuid: "ece2a20b74874eb7acf32b82a1691f97"
format_stamp: "Formatted at 2026-09-23 22:12:27 on dist-test-slave-s2fj"
server_key: "5cb85ba4bb1e518685964419263098c0"
server_key_iv: "eac56ea4387c1055707922bea311d65c"
server_key_version: "encryptionkey@0"
I20260923 22:12:27.435149 3645 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskRiQe_a/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790201539513534-1567-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskRiQe_a/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790201539513534-1567-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskRiQe_a/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790201539513534-1567-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260923 22:12:27.442375 3645 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260923 22:12:27.443936 3645 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260923 22:12:27.444034 3645 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260923 22:12:27.444589 3645 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260923 22:12:27.444636 3645 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260923 22:12:27.444656 3645 ts_tablet_manager.cc:610] Registered 0 tablets
I20260923 22:12:27.444669 3645 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
Sep 23 22:12:27 dist-test-slave-s2fj krb5kdc[3248](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790201547, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.135.195@KRBTEST.COM for kudu/127.1.135.254@KRBTEST.COM
I20260923 22:12:27.451700 3645 rpc_server.cc:307] RPC server started. Bound to: 127.1.135.195:35699
I20260923 22:12:27.451779 3775 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.1.135.195:35699 every 8 connection(s)
I20260923 22:12:27.452142 3645 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskRiQe_a/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790201539513534-1567-0/minicluster-data/ts-2/data/info.pb
I20260923 22:12:27.454401 3343 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 22:12:27.445062 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.135.195:49885 (local address 127.1.135.254:37377)
0923 22:12:27.445132 (+ 70us) server_negotiation.cc:206] Beginning negotiation
0923 22:12:27.445135 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0923 22:12:27.445297 (+ 162us) server_negotiation.cc:407] Connection header received
0923 22:12:27.445775 (+ 478us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 22:12:27.445777 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 22:12:27.445789 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 22:12:27.445845 (+ 56us) server_negotiation.cc:226] Negotiated authn=SASL
0923 22:12:27.446749 (+ 904us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 22:12:27.447599 (+ 850us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 22:12:27.449279 (+ 1680us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 22:12:27.449494 (+ 215us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 22:12:27.449590 (+ 96us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 22:12:27.451772 (+ 2182us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0923 22:12:27.451774 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0923 22:12:27.451774 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0923 22:12:27.451782 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0923 22:12:27.453223 (+ 1441us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0923 22:12:27.453610 (+ 387us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0923 22:12:27.453610 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0923 22:12:27.453611 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0923 22:12:27.453659 (+ 48us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0923 22:12:27.453888 (+ 229us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0923 22:12:27.453889 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0923 22:12:27.453889 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0923 22:12:27.453984 (+ 95us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0923 22:12:27.454035 (+ 51us) server_negotiation.cc:1035] Waiting for connection context
0923 22:12:27.454273 (+ 238us) server_negotiation.cc:299] Negotiation successful
0923 22:12:27.454337 (+ 64us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":42}
I20260923 22:12:27.454550 3669 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 22:12:27.445052 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.135.254:37377 (local address 127.1.135.195:49885)
0923 22:12:27.445193 (+ 141us) negotiation.cc:107] Waiting for socket to connect
0923 22:12:27.445229 (+ 36us) client_negotiation.cc:174] Beginning negotiation
0923 22:12:27.445666 (+ 437us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0923 22:12:27.445971 (+ 305us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0923 22:12:27.445976 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0923 22:12:27.446357 (+ 381us) client_negotiation.cc:189] Negotiated authn=SASL
0923 22:12:27.446645 (+ 288us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 22:12:27.446651 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 22:12:27.447708 (+ 1057us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 22:12:27.447708 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 22:12:27.449182 (+ 1474us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 22:12:27.449185 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 22:12:27.449610 (+ 425us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 22:12:27.449611 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 22:12:27.449725 (+ 114us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 22:12:27.450231 (+ 506us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0923 22:12:27.450239 (+ 8us) client_negotiation.cc:656] calling sasl_client_start()
0923 22:12:27.451650 (+ 1411us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0923 22:12:27.453320 (+ 1670us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0923 22:12:27.453325 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0923 22:12:27.453327 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0923 22:12:27.453540 (+ 213us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0923 22:12:27.453708 (+ 168us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0923 22:12:27.453708 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0923 22:12:27.453709 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0923 22:12:27.453815 (+ 106us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0923 22:12:27.454044 (+ 229us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0923 22:12:27.454049 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0923 22:12:27.454163 (+ 114us) client_negotiation.cc:769] Sending connection context
0923 22:12:27.454244 (+ 81us) client_negotiation.cc:240] Negotiation successful
0923 22:12:27.454358 (+ 114us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":106,"thread_start_us":81,"threads_started":1}
I20260923 22:12:27.455037 3776 heartbeater.cc:347] Connected to a master server at 127.1.135.254:37377
I20260923 22:12:27.455127 3776 heartbeater.cc:464] Registering TS with master...
I20260923 22:12:27.455204 1567 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskRiQe_a/build/release/bin/kudu as pid 3645
I20260923 22:12:27.455303 1567 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskRiQe_a/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790201539513534-1567-0/minicluster-data/ts-2/wal/instance
I20260923 22:12:27.455435 3776 heartbeater.cc:511] Master 127.1.135.254:37377 requested a full tablet report, sending...
I20260923 22:12:27.455516 1567 external_mini_cluster.cc:1428] Setting key 7692718e91347bacafbc6e330c1ab2ea
I20260923 22:12:27.455793 3292 ts_manager.cc:194] Registered new tserver with Master: ece2a20b74874eb7acf32b82a1691f97 (127.1.135.195:35699)
I20260923 22:12:27.456310 3292 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.1.135.195@KRBTEST.COM'} at 127.1.135.195:49885
I20260923 22:12:27.456382 1567 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260923 22:12:27.462182 3782 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 22:12:27.456144 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.135.254:37377 (local address 127.1.135.195:60949)
0923 22:12:27.456311 (+ 167us) negotiation.cc:107] Waiting for socket to connect
0923 22:12:27.456323 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0923 22:12:27.456385 (+ 62us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0923 22:12:27.456529 (+ 144us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0923 22:12:27.456529 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0923 22:12:27.456760 (+ 231us) client_negotiation.cc:189] Negotiated authn=SASL
0923 22:12:27.456833 (+ 73us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 22:12:27.456835 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 22:12:27.457689 (+ 854us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 22:12:27.457690 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 22:12:27.458316 (+ 626us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 22:12:27.458318 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 22:12:27.458646 (+ 328us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 22:12:27.458647 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 22:12:27.458726 (+ 79us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 22:12:27.459124 (+ 398us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0923 22:12:27.459129 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0923 22:12:27.459552 (+ 423us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0923 22:12:27.461238 (+ 1686us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0923 22:12:27.461239 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0923 22:12:27.461240 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0923 22:12:27.461465 (+ 225us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0923 22:12:27.461669 (+ 204us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0923 22:12:27.461669 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0923 22:12:27.461669 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0923 22:12:27.461715 (+ 46us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0923 22:12:27.461975 (+ 260us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0923 22:12:27.461976 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0923 22:12:27.462006 (+ 30us) client_negotiation.cc:769] Sending connection context
0923 22:12:27.462051 (+ 45us) client_negotiation.cc:240] Negotiation successful
0923 22:12:27.462117 (+ 66us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":143,"mutex_wait_us":4,"thread_start_us":57,"threads_started":1}
I20260923 22:12:27.462271 3343 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 22:12:27.456167 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.135.195:60949 (local address 127.1.135.254:37377)
0923 22:12:27.456199 (+ 32us) server_negotiation.cc:206] Beginning negotiation
0923 22:12:27.456200 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0923 22:12:27.456365 (+ 165us) server_negotiation.cc:407] Connection header received
0923 22:12:27.456442 (+ 77us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 22:12:27.456443 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 22:12:27.456452 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 22:12:27.456483 (+ 31us) server_negotiation.cc:226] Negotiated authn=SASL
0923 22:12:27.456919 (+ 436us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 22:12:27.457596 (+ 677us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 22:12:27.458407 (+ 811us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 22:12:27.458573 (+ 166us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 22:12:27.458626 (+ 53us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 22:12:27.459665 (+ 1039us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0923 22:12:27.459666 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0923 22:12:27.459666 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0923 22:12:27.459673 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0923 22:12:27.461182 (+ 1509us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0923 22:12:27.461544 (+ 362us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0923 22:12:27.461545 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0923 22:12:27.461546 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0923 22:12:27.461602 (+ 56us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0923 22:12:27.461774 (+ 172us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0923 22:12:27.461775 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0923 22:12:27.461775 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0923 22:12:27.461914 (+ 139us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0923 22:12:27.461989 (+ 75us) server_negotiation.cc:1035] Waiting for connection context
0923 22:12:27.462076 (+ 87us) server_negotiation.cc:299] Negotiation successful
0923 22:12:27.462153 (+ 77us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":14}
Sep 23 22:12:27 dist-test-slave-s2fj krb5kdc[3248](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790201547, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
Sep 23 22:12:27 dist-test-slave-s2fj krb5kdc[3248](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790201547, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.1.135.254@KRBTEST.COM
I20260923 22:12:27.479730 3343 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 22:12:27.473016 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35744 (local address 127.1.135.254:37377)
0923 22:12:27.473078 (+ 62us) server_negotiation.cc:206] Beginning negotiation
0923 22:12:27.473081 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0923 22:12:27.473088 (+ 7us) server_negotiation.cc:407] Connection header received
0923 22:12:27.473115 (+ 27us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 22:12:27.473115 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 22:12:27.473123 (+ 8us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 22:12:27.473219 (+ 96us) server_negotiation.cc:226] Negotiated authn=SASL
0923 22:12:27.473738 (+ 519us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 22:12:27.474194 (+ 456us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 22:12:27.474924 (+ 730us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 22:12:27.475104 (+ 180us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 22:12:27.475189 (+ 85us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 22:12:27.477360 (+ 2171us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0923 22:12:27.477361 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0923 22:12:27.477361 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0923 22:12:27.477367 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0923 22:12:27.478641 (+ 1274us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0923 22:12:27.479003 (+ 362us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0923 22:12:27.479004 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0923 22:12:27.479005 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0923 22:12:27.479058 (+ 53us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0923 22:12:27.479251 (+ 193us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0923 22:12:27.479253 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0923 22:12:27.479253 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0923 22:12:27.479367 (+ 114us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0923 22:12:27.479466 (+ 99us) server_negotiation.cc:1035] Waiting for connection context
0923 22:12:27.479565 (+ 99us) server_negotiation.cc:299] Negotiation successful
0923 22:12:27.479636 (+ 71us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":37}
I20260923 22:12:27.481360 3292 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:35744:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260923 22:12:27.482797 3292 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260923 22:12:27.488885 3794 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 22:12:27.485307 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41660 (local address 127.1.135.195:35699)
0923 22:12:27.485472 (+ 165us) server_negotiation.cc:206] Beginning negotiation
0923 22:12:27.485476 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0923 22:12:27.485489 (+ 13us) server_negotiation.cc:407] Connection header received
0923 22:12:27.485537 (+ 48us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 22:12:27.485539 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 22:12:27.485556 (+ 17us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 22:12:27.485634 (+ 78us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0923 22:12:27.485864 (+ 230us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 22:12:27.486569 (+ 705us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 22:12:27.487751 (+ 1182us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 22:12:27.488466 (+ 715us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 22:12:27.488552 (+ 86us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 22:12:27.488607 (+ 55us) server_negotiation.cc:1035] Waiting for connection context
0923 22:12:27.488761 (+ 154us) server_negotiation.cc:299] Negotiation successful
0923 22:12:27.488795 (+ 34us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":129,"thread_start_us":77,"threads_started":1}
I20260923 22:12:27.488879 3790 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 22:12:27.484975 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.135.195:35699 (local address 127.0.0.1:41660)
0923 22:12:27.485065 (+ 90us) negotiation.cc:107] Waiting for socket to connect
0923 22:12:27.485299 (+ 234us) client_negotiation.cc:174] Beginning negotiation
0923 22:12:27.485379 (+ 80us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0923 22:12:27.485713 (+ 334us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0923 22:12:27.485716 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0923 22:12:27.485718 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0923 22:12:27.485787 (+ 69us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 22:12:27.485789 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 22:12:27.486654 (+ 865us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 22:12:27.486655 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 22:12:27.487661 (+ 1006us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 22:12:27.487663 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 22:12:27.488550 (+ 887us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 22:12:27.488551 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 22:12:27.488632 (+ 81us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 22:12:27.488644 (+ 12us) client_negotiation.cc:769] Sending connection context
0923 22:12:27.488746 (+ 102us) client_negotiation.cc:240] Negotiation successful
0923 22:12:27.488807 (+ 61us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":68,"thread_start_us":50,"threads_started":1}
I20260923 22:12:27.489298 3791 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 22:12:27.485182 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.135.193:42577 (local address 127.0.0.1:37922)
0923 22:12:27.485544 (+ 362us) negotiation.cc:107] Waiting for socket to connect
0923 22:12:27.485554 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0923 22:12:27.485613 (+ 59us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0923 22:12:27.485864 (+ 251us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0923 22:12:27.485864 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0923 22:12:27.485866 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0923 22:12:27.485907 (+ 41us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 22:12:27.485909 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 22:12:27.487192 (+ 1283us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 22:12:27.487193 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 22:12:27.488169 (+ 976us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 22:12:27.488172 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 22:12:27.489091 (+ 919us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 22:12:27.489092 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 22:12:27.489149 (+ 57us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 22:12:27.489153 (+ 4us) client_negotiation.cc:769] Sending connection context
0923 22:12:27.489196 (+ 43us) client_negotiation.cc:240] Negotiation successful
0923 22:12:27.489232 (+ 36us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":348,"mutex_wait_us":34,"thread_start_us":75,"threads_started":1}
I20260923 22:12:27.489548 3792 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 22:12:27.485159 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37922 (local address 127.1.135.193:42577)
0923 22:12:27.485486 (+ 327us) server_negotiation.cc:206] Beginning negotiation
0923 22:12:27.485491 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0923 22:12:27.485671 (+ 180us) server_negotiation.cc:407] Connection header received
0923 22:12:27.485715 (+ 44us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 22:12:27.485717 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 22:12:27.485731 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 22:12:27.485800 (+ 69us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0923 22:12:27.485970 (+ 170us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 22:12:27.487091 (+ 1121us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 22:12:27.488246 (+ 1155us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 22:12:27.488927 (+ 681us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 22:12:27.489002 (+ 75us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 22:12:27.489059 (+ 57us) server_negotiation.cc:1035] Waiting for connection context
0923 22:12:27.489415 (+ 356us) server_negotiation.cc:299] Negotiation successful
0923 22:12:27.489461 (+ 46us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":296,"thread_start_us":74,"threads_started":1}
I20260923 22:12:27.489523 3710 tablet_service.cc:1467] Processing CreateTablet for tablet 9fd0c8c6b7b541798f3da5d6dab48ddd (DEFAULT_TABLE table=test-table [id=e9898a4ea2424a458f64a73481e76db7]), partition=RANGE (key) PARTITION UNBOUNDED
I20260923 22:12:27.489753 3793 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 22:12:27.485159 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.135.194:36299 (local address 127.0.0.1:48168)
0923 22:12:27.485532 (+ 373us) negotiation.cc:107] Waiting for socket to connect
0923 22:12:27.485540 (+ 8us) client_negotiation.cc:174] Beginning negotiation
0923 22:12:27.485611 (+ 71us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0923 22:12:27.485887 (+ 276us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0923 22:12:27.485888 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0923 22:12:27.485890 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0923 22:12:27.485966 (+ 76us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 22:12:27.485968 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 22:12:27.487190 (+ 1222us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 22:12:27.487191 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 22:12:27.488331 (+ 1140us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 22:12:27.488334 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 22:12:27.489530 (+ 1196us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 22:12:27.489530 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 22:12:27.489593 (+ 63us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 22:12:27.489596 (+ 3us) client_negotiation.cc:769] Sending connection context
0923 22:12:27.489648 (+ 52us) client_negotiation.cc:240] Negotiation successful
0923 22:12:27.489682 (+ 34us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":363,"mutex_wait_us":39,"thread_start_us":38,"threads_started":1}
I20260923 22:12:27.489894 3710 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 9fd0c8c6b7b541798f3da5d6dab48ddd. 1 dirs total, 0 dirs full, 0 dirs failed
I20260923 22:12:27.490001 3795 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 22:12:27.485312 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48168 (local address 127.1.135.194:36299)
0923 22:12:27.485500 (+ 188us) server_negotiation.cc:206] Beginning negotiation
0923 22:12:27.485503 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0923 22:12:27.485610 (+ 107us) server_negotiation.cc:407] Connection header received
0923 22:12:27.485723 (+ 113us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 22:12:27.485726 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 22:12:27.485744 (+ 18us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 22:12:27.485815 (+ 71us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0923 22:12:27.486051 (+ 236us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 22:12:27.487110 (+ 1059us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 22:12:27.488411 (+ 1301us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 22:12:27.489384 (+ 973us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 22:12:27.489449 (+ 65us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 22:12:27.489497 (+ 48us) server_negotiation.cc:1035] Waiting for connection context
0923 22:12:27.489848 (+ 351us) server_negotiation.cc:299] Negotiation successful
0923 22:12:27.489901 (+ 53us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":161,"thread_start_us":89,"threads_started":1}
I20260923 22:12:27.490546 3570 tablet_service.cc:1467] Processing CreateTablet for tablet 9fd0c8c6b7b541798f3da5d6dab48ddd (DEFAULT_TABLE table=test-table [id=e9898a4ea2424a458f64a73481e76db7]), partition=RANGE (key) PARTITION UNBOUNDED
I20260923 22:12:27.490867 3570 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 9fd0c8c6b7b541798f3da5d6dab48ddd. 1 dirs total, 0 dirs full, 0 dirs failed
I20260923 22:12:27.492871 3430 tablet_service.cc:1467] Processing CreateTablet for tablet 9fd0c8c6b7b541798f3da5d6dab48ddd (DEFAULT_TABLE table=test-table [id=e9898a4ea2424a458f64a73481e76db7]), partition=RANGE (key) PARTITION UNBOUNDED
I20260923 22:12:27.493111 3796 tablet_bootstrap.cc:492] T 9fd0c8c6b7b541798f3da5d6dab48ddd P ece2a20b74874eb7acf32b82a1691f97: Bootstrap starting.
I20260923 22:12:27.493178 3430 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 9fd0c8c6b7b541798f3da5d6dab48ddd. 1 dirs total, 0 dirs full, 0 dirs failed
I20260923 22:12:27.493794 3796 tablet_bootstrap.cc:654] T 9fd0c8c6b7b541798f3da5d6dab48ddd P ece2a20b74874eb7acf32b82a1691f97: Neither blocks nor log segments found. Creating new log.
I20260923 22:12:27.494035 3797 tablet_bootstrap.cc:492] T 9fd0c8c6b7b541798f3da5d6dab48ddd P d269cfa9b6874c6fb8b07b0da9f00ea5: Bootstrap starting.
I20260923 22:12:27.494213 3796 log.cc:824] T 9fd0c8c6b7b541798f3da5d6dab48ddd P ece2a20b74874eb7acf32b82a1691f97: Log is configured to *not* fsync() on all Append() calls
I20260923 22:12:27.494590 3797 tablet_bootstrap.cc:654] T 9fd0c8c6b7b541798f3da5d6dab48ddd P d269cfa9b6874c6fb8b07b0da9f00ea5: Neither blocks nor log segments found. Creating new log.
I20260923 22:12:27.494906 3797 log.cc:824] T 9fd0c8c6b7b541798f3da5d6dab48ddd P d269cfa9b6874c6fb8b07b0da9f00ea5: Log is configured to *not* fsync() on all Append() calls
I20260923 22:12:27.495036 3796 tablet_bootstrap.cc:492] T 9fd0c8c6b7b541798f3da5d6dab48ddd P ece2a20b74874eb7acf32b82a1691f97: No bootstrap required, opened a new log
I20260923 22:12:27.495095 3796 ts_tablet_manager.cc:1397] T 9fd0c8c6b7b541798f3da5d6dab48ddd P ece2a20b74874eb7acf32b82a1691f97: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20260923 22:12:27.495708 3797 tablet_bootstrap.cc:492] T 9fd0c8c6b7b541798f3da5d6dab48ddd P d269cfa9b6874c6fb8b07b0da9f00ea5: No bootstrap required, opened a new log
I20260923 22:12:27.495764 3797 ts_tablet_manager.cc:1397] T 9fd0c8c6b7b541798f3da5d6dab48ddd P d269cfa9b6874c6fb8b07b0da9f00ea5: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20260923 22:12:27.496230 3800 tablet_bootstrap.cc:492] T 9fd0c8c6b7b541798f3da5d6dab48ddd P 04978c60cca64c6783ac03221353dbe6: Bootstrap starting.
I20260923 22:12:27.496762 3800 tablet_bootstrap.cc:654] T 9fd0c8c6b7b541798f3da5d6dab48ddd P 04978c60cca64c6783ac03221353dbe6: Neither blocks nor log segments found. Creating new log.
I20260923 22:12:27.497044 3800 log.cc:824] T 9fd0c8c6b7b541798f3da5d6dab48ddd P 04978c60cca64c6783ac03221353dbe6: Log is configured to *not* fsync() on all Append() calls
I20260923 22:12:27.497285 3796 raft_consensus.cc:348] T 9fd0c8c6b7b541798f3da5d6dab48ddd P ece2a20b74874eb7acf32b82a1691f97 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ece2a20b74874eb7acf32b82a1691f97" member_type: VOTER last_known_addr { host: "127.1.135.195" port: 35699 } } peers { permanent_uuid: "d269cfa9b6874c6fb8b07b0da9f00ea5" member_type: VOTER last_known_addr { host: "127.1.135.194" port: 36299 } } peers { permanent_uuid: "04978c60cca64c6783ac03221353dbe6" member_type: VOTER last_known_addr { host: "127.1.135.193" port: 42577 } }
I20260923 22:12:27.497450 3796 raft_consensus.cc:374] T 9fd0c8c6b7b541798f3da5d6dab48ddd P ece2a20b74874eb7acf32b82a1691f97 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260923 22:12:27.497489 3796 raft_consensus.cc:729] T 9fd0c8c6b7b541798f3da5d6dab48ddd P ece2a20b74874eb7acf32b82a1691f97 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: ece2a20b74874eb7acf32b82a1691f97, State: Initialized, Role: FOLLOWER
I20260923 22:12:27.497592 3796 consensus_queue.cc:260] T 9fd0c8c6b7b541798f3da5d6dab48ddd P ece2a20b74874eb7acf32b82a1691f97 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ece2a20b74874eb7acf32b82a1691f97" member_type: VOTER last_known_addr { host: "127.1.135.195" port: 35699 } } peers { permanent_uuid: "d269cfa9b6874c6fb8b07b0da9f00ea5" member_type: VOTER last_known_addr { host: "127.1.135.194" port: 36299 } } peers { permanent_uuid: "04978c60cca64c6783ac03221353dbe6" member_type: VOTER last_known_addr { host: "127.1.135.193" port: 42577 } }
I20260923 22:12:27.497805 3800 tablet_bootstrap.cc:492] T 9fd0c8c6b7b541798f3da5d6dab48ddd P 04978c60cca64c6783ac03221353dbe6: No bootstrap required, opened a new log
I20260923 22:12:27.497869 3800 ts_tablet_manager.cc:1397] T 9fd0c8c6b7b541798f3da5d6dab48ddd P 04978c60cca64c6783ac03221353dbe6: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20260923 22:12:27.497864 3796 ts_tablet_manager.cc:1428] T 9fd0c8c6b7b541798f3da5d6dab48ddd P ece2a20b74874eb7acf32b82a1691f97: Time spent starting tablet: real 0.003s user 0.003s sys 0.000s
I20260923 22:12:27.497988 3776 heartbeater.cc:503] Master 127.1.135.254:37377 was elected leader, sending a full tablet report...
I20260923 22:12:27.497967 3797 raft_consensus.cc:348] T 9fd0c8c6b7b541798f3da5d6dab48ddd P d269cfa9b6874c6fb8b07b0da9f00ea5 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ece2a20b74874eb7acf32b82a1691f97" member_type: VOTER last_known_addr { host: "127.1.135.195" port: 35699 } } peers { permanent_uuid: "d269cfa9b6874c6fb8b07b0da9f00ea5" member_type: VOTER last_known_addr { host: "127.1.135.194" port: 36299 } } peers { permanent_uuid: "04978c60cca64c6783ac03221353dbe6" member_type: VOTER last_known_addr { host: "127.1.135.193" port: 42577 } }
I20260923 22:12:27.498131 3797 raft_consensus.cc:374] T 9fd0c8c6b7b541798f3da5d6dab48ddd P d269cfa9b6874c6fb8b07b0da9f00ea5 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260923 22:12:27.498170 3797 raft_consensus.cc:729] T 9fd0c8c6b7b541798f3da5d6dab48ddd P d269cfa9b6874c6fb8b07b0da9f00ea5 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d269cfa9b6874c6fb8b07b0da9f00ea5, State: Initialized, Role: FOLLOWER
I20260923 22:12:27.498281 3797 consensus_queue.cc:260] T 9fd0c8c6b7b541798f3da5d6dab48ddd P d269cfa9b6874c6fb8b07b0da9f00ea5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ece2a20b74874eb7acf32b82a1691f97" member_type: VOTER last_known_addr { host: "127.1.135.195" port: 35699 } } peers { permanent_uuid: "d269cfa9b6874c6fb8b07b0da9f00ea5" member_type: VOTER last_known_addr { host: "127.1.135.194" port: 36299 } } peers { permanent_uuid: "04978c60cca64c6783ac03221353dbe6" member_type: VOTER last_known_addr { host: "127.1.135.193" port: 42577 } }
I20260923 22:12:27.498510 3797 ts_tablet_manager.cc:1428] T 9fd0c8c6b7b541798f3da5d6dab48ddd P d269cfa9b6874c6fb8b07b0da9f00ea5: Time spent starting tablet: real 0.003s user 0.000s sys 0.003s
I20260923 22:12:27.498554 3636 heartbeater.cc:503] Master 127.1.135.254:37377 was elected leader, sending a full tablet report...
I20260923 22:12:27.499603 3800 raft_consensus.cc:348] T 9fd0c8c6b7b541798f3da5d6dab48ddd P 04978c60cca64c6783ac03221353dbe6 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ece2a20b74874eb7acf32b82a1691f97" member_type: VOTER last_known_addr { host: "127.1.135.195" port: 35699 } } peers { permanent_uuid: "d269cfa9b6874c6fb8b07b0da9f00ea5" member_type: VOTER last_known_addr { host: "127.1.135.194" port: 36299 } } peers { permanent_uuid: "04978c60cca64c6783ac03221353dbe6" member_type: VOTER last_known_addr { host: "127.1.135.193" port: 42577 } }
I20260923 22:12:27.499714 3800 raft_consensus.cc:374] T 9fd0c8c6b7b541798f3da5d6dab48ddd P 04978c60cca64c6783ac03221353dbe6 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260923 22:12:27.499735 3800 raft_consensus.cc:729] T 9fd0c8c6b7b541798f3da5d6dab48ddd P 04978c60cca64c6783ac03221353dbe6 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 04978c60cca64c6783ac03221353dbe6, State: Initialized, Role: FOLLOWER
I20260923 22:12:27.499858 3800 consensus_queue.cc:260] T 9fd0c8c6b7b541798f3da5d6dab48ddd P 04978c60cca64c6783ac03221353dbe6 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ece2a20b74874eb7acf32b82a1691f97" member_type: VOTER last_known_addr { host: "127.1.135.195" port: 35699 } } peers { permanent_uuid: "d269cfa9b6874c6fb8b07b0da9f00ea5" member_type: VOTER last_known_addr { host: "127.1.135.194" port: 36299 } } peers { permanent_uuid: "04978c60cca64c6783ac03221353dbe6" member_type: VOTER last_known_addr { host: "127.1.135.193" port: 42577 } }
I20260923 22:12:27.500134 3800 ts_tablet_manager.cc:1428] T 9fd0c8c6b7b541798f3da5d6dab48ddd P 04978c60cca64c6783ac03221353dbe6: Time spent starting tablet: real 0.002s user 0.002s sys 0.000s
I20260923 22:12:27.500174 3496 heartbeater.cc:503] Master 127.1.135.254:37377 was elected leader, sending a full tablet report...
W20260923 22:12:27.564891 3637 tablet.cc:2369] T 9fd0c8c6b7b541798f3da5d6dab48ddd P d269cfa9b6874c6fb8b07b0da9f00ea5: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260923 22:12:27.663273 3497 tablet.cc:2369] T 9fd0c8c6b7b541798f3da5d6dab48ddd P 04978c60cca64c6783ac03221353dbe6: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260923 22:12:27.673302 3802 raft_consensus.cc:482] T 9fd0c8c6b7b541798f3da5d6dab48ddd P ece2a20b74874eb7acf32b82a1691f97 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260923 22:12:27.673430 3802 raft_consensus.cc:504] T 9fd0c8c6b7b541798f3da5d6dab48ddd P ece2a20b74874eb7acf32b82a1691f97 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ece2a20b74874eb7acf32b82a1691f97" member_type: VOTER last_known_addr { host: "127.1.135.195" port: 35699 } } peers { permanent_uuid: "d269cfa9b6874c6fb8b07b0da9f00ea5" member_type: VOTER last_known_addr { host: "127.1.135.194" port: 36299 } } peers { permanent_uuid: "04978c60cca64c6783ac03221353dbe6" member_type: VOTER last_known_addr { host: "127.1.135.193" port: 42577 } }
I20260923 22:12:27.673761 3802 leader_election.cc:288] T 9fd0c8c6b7b541798f3da5d6dab48ddd P ece2a20b74874eb7acf32b82a1691f97 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers d269cfa9b6874c6fb8b07b0da9f00ea5 (127.1.135.194:36299), 04978c60cca64c6783ac03221353dbe6 (127.1.135.193:42577)
I20260923 22:12:27.677601 3792 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 22:12:27.674082 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.135.195:50211 (local address 127.1.135.193:42577)
0923 22:12:27.674120 (+ 38us) server_negotiation.cc:206] Beginning negotiation
0923 22:12:27.674122 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0923 22:12:27.674233 (+ 111us) server_negotiation.cc:407] Connection header received
0923 22:12:27.674271 (+ 38us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 22:12:27.674272 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 22:12:27.674280 (+ 8us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 22:12:27.674321 (+ 41us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0923 22:12:27.674474 (+ 153us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 22:12:27.675155 (+ 681us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 22:12:27.676697 (+ 1542us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 22:12:27.677225 (+ 528us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 22:12:27.677301 (+ 76us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 22:12:27.677309 (+ 8us) server_negotiation.cc:1035] Waiting for connection context
0923 22:12:27.677477 (+ 168us) server_negotiation.cc:299] Negotiation successful
0923 22:12:27.677513 (+ 36us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":20}
I20260923 22:12:27.677601 3795 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 22:12:27.674021 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.135.195:57263 (local address 127.1.135.194:36299)
0923 22:12:27.674066 (+ 45us) server_negotiation.cc:206] Beginning negotiation
0923 22:12:27.674069 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0923 22:12:27.674081 (+ 12us) server_negotiation.cc:407] Connection header received
0923 22:12:27.674113 (+ 32us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 22:12:27.674114 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 22:12:27.674124 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 22:12:27.674201 (+ 77us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0923 22:12:27.674382 (+ 181us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 22:12:27.675151 (+ 769us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 22:12:27.676698 (+ 1547us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 22:12:27.677230 (+ 532us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 22:12:27.677301 (+ 71us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 22:12:27.677309 (+ 8us) server_negotiation.cc:1035] Waiting for connection context
0923 22:12:27.677477 (+ 168us) server_negotiation.cc:299] Negotiation successful
0923 22:12:27.677515 (+ 38us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":23}
I20260923 22:12:27.677620 3669 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 22:12:27.673964 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.135.194:36299 (local address 127.1.135.195:57263)
0923 22:12:27.674002 (+ 38us) negotiation.cc:107] Waiting for socket to connect
0923 22:12:27.674014 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0923 22:12:27.674057 (+ 43us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0923 22:12:27.674216 (+ 159us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0923 22:12:27.674217 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0923 22:12:27.674219 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0923 22:12:27.674290 (+ 71us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 22:12:27.674296 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 22:12:27.675234 (+ 938us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 22:12:27.675234 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 22:12:27.676607 (+ 1373us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 22:12:27.676610 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 22:12:27.677308 (+ 698us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 22:12:27.677310 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 22:12:27.677403 (+ 93us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 22:12:27.677406 (+ 3us) client_negotiation.cc:769] Sending connection context
0923 22:12:27.677460 (+ 54us) client_negotiation.cc:240] Negotiation successful
0923 22:12:27.677506 (+ 46us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":21,"mutex_wait_us":54}
I20260923 22:12:27.677629 3805 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 22:12:27.673964 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.135.193:42577 (local address 127.1.135.195:50211)
0923 22:12:27.674155 (+ 191us) negotiation.cc:107] Waiting for socket to connect
0923 22:12:27.674162 (+ 7us) client_negotiation.cc:174] Beginning negotiation
0923 22:12:27.674222 (+ 60us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0923 22:12:27.674346 (+ 124us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0923 22:12:27.674347 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0923 22:12:27.674348 (+ 1us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0923 22:12:27.674411 (+ 63us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 22:12:27.674416 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 22:12:27.675234 (+ 818us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 22:12:27.675234 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 22:12:27.676607 (+ 1373us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 22:12:27.676610 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 22:12:27.677309 (+ 699us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 22:12:27.677310 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 22:12:27.677398 (+ 88us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 22:12:27.677403 (+ 5us) client_negotiation.cc:769] Sending connection context
0923 22:12:27.677460 (+ 57us) client_negotiation.cc:240] Negotiation successful
0923 22:12:27.677506 (+ 46us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":170,"mutex_wait_us":164,"thread_start_us":83,"threads_started":1}
I20260923 22:12:27.678006 3590 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "9fd0c8c6b7b541798f3da5d6dab48ddd" candidate_uuid: "ece2a20b74874eb7acf32b82a1691f97" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d269cfa9b6874c6fb8b07b0da9f00ea5" is_pre_election: true
I20260923 22:12:27.678171 3590 raft_consensus.cc:2454] T 9fd0c8c6b7b541798f3da5d6dab48ddd P d269cfa9b6874c6fb8b07b0da9f00ea5 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ece2a20b74874eb7acf32b82a1691f97 in term 0.
I20260923 22:12:27.678236 3450 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "9fd0c8c6b7b541798f3da5d6dab48ddd" candidate_uuid: "ece2a20b74874eb7acf32b82a1691f97" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "04978c60cca64c6783ac03221353dbe6" is_pre_election: true
I20260923 22:12:27.678404 3450 raft_consensus.cc:2454] T 9fd0c8c6b7b541798f3da5d6dab48ddd P 04978c60cca64c6783ac03221353dbe6 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ece2a20b74874eb7acf32b82a1691f97 in term 0.
I20260923 22:12:27.678412 3661 leader_election.cc:302] T 9fd0c8c6b7b541798f3da5d6dab48ddd P ece2a20b74874eb7acf32b82a1691f97 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: d269cfa9b6874c6fb8b07b0da9f00ea5, ece2a20b74874eb7acf32b82a1691f97; no voters:
I20260923 22:12:27.678550 3802 raft_consensus.cc:2781] T 9fd0c8c6b7b541798f3da5d6dab48ddd P ece2a20b74874eb7acf32b82a1691f97 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260923 22:12:27.678620 3802 raft_consensus.cc:482] T 9fd0c8c6b7b541798f3da5d6dab48ddd P ece2a20b74874eb7acf32b82a1691f97 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260923 22:12:27.678655 3802 raft_consensus.cc:3037] T 9fd0c8c6b7b541798f3da5d6dab48ddd P ece2a20b74874eb7acf32b82a1691f97 [term 0 FOLLOWER]: Advancing to term 1
I20260923 22:12:27.679318 3802 raft_consensus.cc:504] T 9fd0c8c6b7b541798f3da5d6dab48ddd P ece2a20b74874eb7acf32b82a1691f97 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ece2a20b74874eb7acf32b82a1691f97" member_type: VOTER last_known_addr { host: "127.1.135.195" port: 35699 } } peers { permanent_uuid: "d269cfa9b6874c6fb8b07b0da9f00ea5" member_type: VOTER last_known_addr { host: "127.1.135.194" port: 36299 } } peers { permanent_uuid: "04978c60cca64c6783ac03221353dbe6" member_type: VOTER last_known_addr { host: "127.1.135.193" port: 42577 } }
I20260923 22:12:27.679484 3802 leader_election.cc:288] T 9fd0c8c6b7b541798f3da5d6dab48ddd P ece2a20b74874eb7acf32b82a1691f97 [CANDIDATE]: Term 1 election: Requested vote from peers d269cfa9b6874c6fb8b07b0da9f00ea5 (127.1.135.194:36299), 04978c60cca64c6783ac03221353dbe6 (127.1.135.193:42577)
I20260923 22:12:27.679618 3450 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "9fd0c8c6b7b541798f3da5d6dab48ddd" candidate_uuid: "ece2a20b74874eb7acf32b82a1691f97" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "04978c60cca64c6783ac03221353dbe6"
I20260923 22:12:27.679611 3590 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "9fd0c8c6b7b541798f3da5d6dab48ddd" candidate_uuid: "ece2a20b74874eb7acf32b82a1691f97" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d269cfa9b6874c6fb8b07b0da9f00ea5"
I20260923 22:12:27.679705 3450 raft_consensus.cc:3037] T 9fd0c8c6b7b541798f3da5d6dab48ddd P 04978c60cca64c6783ac03221353dbe6 [term 0 FOLLOWER]: Advancing to term 1
I20260923 22:12:27.679715 3590 raft_consensus.cc:3037] T 9fd0c8c6b7b541798f3da5d6dab48ddd P d269cfa9b6874c6fb8b07b0da9f00ea5 [term 0 FOLLOWER]: Advancing to term 1
I20260923 22:12:27.680424 3450 raft_consensus.cc:2454] T 9fd0c8c6b7b541798f3da5d6dab48ddd P 04978c60cca64c6783ac03221353dbe6 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ece2a20b74874eb7acf32b82a1691f97 in term 1.
I20260923 22:12:27.680493 3590 raft_consensus.cc:2454] T 9fd0c8c6b7b541798f3da5d6dab48ddd P d269cfa9b6874c6fb8b07b0da9f00ea5 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ece2a20b74874eb7acf32b82a1691f97 in term 1.
I20260923 22:12:27.680951 3662 leader_election.cc:302] T 9fd0c8c6b7b541798f3da5d6dab48ddd P ece2a20b74874eb7acf32b82a1691f97 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 04978c60cca64c6783ac03221353dbe6, ece2a20b74874eb7acf32b82a1691f97; no voters:
I20260923 22:12:27.681092 3802 raft_consensus.cc:2781] T 9fd0c8c6b7b541798f3da5d6dab48ddd P ece2a20b74874eb7acf32b82a1691f97 [term 1 FOLLOWER]: Leader election won for term 1
I20260923 22:12:27.681252 3802 raft_consensus.cc:686] T 9fd0c8c6b7b541798f3da5d6dab48ddd P ece2a20b74874eb7acf32b82a1691f97 [term 1 LEADER]: Becoming Leader. State: Replica: ece2a20b74874eb7acf32b82a1691f97, State: Running, Role: LEADER
I20260923 22:12:27.681340 3802 consensus_queue.cc:237] T 9fd0c8c6b7b541798f3da5d6dab48ddd P ece2a20b74874eb7acf32b82a1691f97 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ece2a20b74874eb7acf32b82a1691f97" member_type: VOTER last_known_addr { host: "127.1.135.195" port: 35699 } } peers { permanent_uuid: "d269cfa9b6874c6fb8b07b0da9f00ea5" member_type: VOTER last_known_addr { host: "127.1.135.194" port: 36299 } } peers { permanent_uuid: "04978c60cca64c6783ac03221353dbe6" member_type: VOTER last_known_addr { host: "127.1.135.193" port: 42577 } }
I20260923 22:12:27.682153 3293 catalog_manager.cc:5495] T 9fd0c8c6b7b541798f3da5d6dab48ddd P ece2a20b74874eb7acf32b82a1691f97 reported cstate change: term changed from 0 to 1, leader changed from <none> to ece2a20b74874eb7acf32b82a1691f97 (127.1.135.195). New cstate: current_term: 1 leader_uuid: "ece2a20b74874eb7acf32b82a1691f97" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ece2a20b74874eb7acf32b82a1691f97" member_type: VOTER last_known_addr { host: "127.1.135.195" port: 35699 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "d269cfa9b6874c6fb8b07b0da9f00ea5" member_type: VOTER last_known_addr { host: "127.1.135.194" port: 36299 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "04978c60cca64c6783ac03221353dbe6" member_type: VOTER last_known_addr { host: "127.1.135.193" port: 42577 } health_report { overall_health: UNKNOWN } } }
W20260923 22:12:27.702829 3777 tablet.cc:2369] T 9fd0c8c6b7b541798f3da5d6dab48ddd P ece2a20b74874eb7acf32b82a1691f97: Can't schedule compaction. Clean time has not been advanced past its initial value.
Sep 23 22:12:27 dist-test-slave-s2fj krb5kdc[3248](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790201546, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.135.254@KRBTEST.COM for kudu/127.1.135.254@KRBTEST.COM
I20260923 22:12:27.734901 3813 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 22:12:27.728852 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.135.254:37377 (local address 127.0.0.1:35756)
0923 22:12:27.729003 (+ 151us) negotiation.cc:107] Waiting for socket to connect
0923 22:12:27.729013 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0923 22:12:27.729076 (+ 63us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0923 22:12:27.729192 (+ 116us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0923 22:12:27.729193 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0923 22:12:27.729468 (+ 275us) client_negotiation.cc:189] Negotiated authn=SASL
0923 22:12:27.729511 (+ 43us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 22:12:27.729513 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 22:12:27.730108 (+ 595us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 22:12:27.730109 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 22:12:27.730610 (+ 501us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 22:12:27.730611 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 22:12:27.730930 (+ 319us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 22:12:27.730931 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 22:12:27.730969 (+ 38us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 22:12:27.731399 (+ 430us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0923 22:12:27.731403 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0923 22:12:27.732770 (+ 1367us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0923 22:12:27.734009 (+ 1239us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0923 22:12:27.734015 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0923 22:12:27.734022 (+ 7us) client_negotiation.cc:760] calling sasl_client_step()
0923 22:12:27.734205 (+ 183us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0923 22:12:27.734366 (+ 161us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0923 22:12:27.734367 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0923 22:12:27.734367 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0923 22:12:27.734421 (+ 54us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0923 22:12:27.734623 (+ 202us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0923 22:12:27.734628 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0923 22:12:27.734666 (+ 38us) client_negotiation.cc:769] Sending connection context
0923 22:12:27.734715 (+ 49us) client_negotiation.cc:240] Negotiation successful
0923 22:12:27.734798 (+ 83us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":127,"thread_start_us":91,"threads_started":1}
I20260923 22:12:27.734941 3343 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 22:12:27.728898 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35756 (local address 127.1.135.254:37377)
0923 22:12:27.728957 (+ 59us) server_negotiation.cc:206] Beginning negotiation
0923 22:12:27.728960 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0923 22:12:27.729076 (+ 116us) server_negotiation.cc:407] Connection header received
0923 22:12:27.729116 (+ 40us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 22:12:27.729117 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 22:12:27.729126 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 22:12:27.729175 (+ 49us) server_negotiation.cc:226] Negotiated authn=SASL
0923 22:12:27.729572 (+ 397us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 22:12:27.730045 (+ 473us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 22:12:27.730704 (+ 659us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 22:12:27.730866 (+ 162us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 22:12:27.730916 (+ 50us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 22:12:27.732845 (+ 1929us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0923 22:12:27.732846 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0923 22:12:27.732846 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0923 22:12:27.732852 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0923 22:12:27.733921 (+ 1069us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0923 22:12:27.734265 (+ 344us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0923 22:12:27.734266 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0923 22:12:27.734266 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0923 22:12:27.734312 (+ 46us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0923 22:12:27.734471 (+ 159us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0923 22:12:27.734471 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0923 22:12:27.734471 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0923 22:12:27.734572 (+ 101us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0923 22:12:27.734619 (+ 47us) server_negotiation.cc:1035] Waiting for connection context
0923 22:12:27.734729 (+ 110us) server_negotiation.cc:299] Negotiation successful
0923 22:12:27.734839 (+ 110us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":50,"server-negotiator.queue_time_us":33}
I20260923 22:12:27.736757 3293 catalog_manager.cc:1908] Servicing CreateTable request from {username='kudu', principal='kudu/127.1.135.254@KRBTEST.COM'} at 127.0.0.1:35756:
name: "kudu_system.kudu_transactions"
schema {
columns {
name: "txn_id"
type: INT64
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "entry_type"
type: INT8
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "identifier"
type: STRING
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "metadata"
type: STRING
is_key: false
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
rows: "<redacted>""\006\001\000\000\000\000\000\000\000\000\007\001@B\017\000\000\000\000\000"
indirect_data: "<redacted>"""
}
partition_schema {
range_schema {
columns {
name: "txn_id"
}
}
}
table_type: TXN_STATUS_TABLE
W20260923 22:12:27.736975 3293 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table kudu_system.kudu_transactions in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260923 22:12:27.738477 3430 tablet_service.cc:1467] Processing CreateTablet for tablet 78a64945a16444c5ad549235dbc52309 (TXN_STATUS_TABLE table=kudu_system.kudu_transactions [id=d69d432f759447bc8f6c93882a1bdef1]), partition=RANGE (txn_id) PARTITION 0 <= VALUES < 1000000
I20260923 22:12:27.738516 3570 tablet_service.cc:1467] Processing CreateTablet for tablet 78a64945a16444c5ad549235dbc52309 (TXN_STATUS_TABLE table=kudu_system.kudu_transactions [id=d69d432f759447bc8f6c93882a1bdef1]), partition=RANGE (txn_id) PARTITION 0 <= VALUES < 1000000
I20260923 22:12:27.738545 3710 tablet_service.cc:1467] Processing CreateTablet for tablet 78a64945a16444c5ad549235dbc52309 (TXN_STATUS_TABLE table=kudu_system.kudu_transactions [id=d69d432f759447bc8f6c93882a1bdef1]), partition=RANGE (txn_id) PARTITION 0 <= VALUES < 1000000
I20260923 22:12:27.738667 3430 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 78a64945a16444c5ad549235dbc52309. 1 dirs total, 0 dirs full, 0 dirs failed
I20260923 22:12:27.738670 3710 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 78a64945a16444c5ad549235dbc52309. 1 dirs total, 0 dirs full, 0 dirs failed
I20260923 22:12:27.738670 3570 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 78a64945a16444c5ad549235dbc52309. 1 dirs total, 0 dirs full, 0 dirs failed
I20260923 22:12:27.739964 3800 tablet_bootstrap.cc:492] T 78a64945a16444c5ad549235dbc52309 P 04978c60cca64c6783ac03221353dbe6: Bootstrap starting.
I20260923 22:12:27.740139 3796 tablet_bootstrap.cc:492] T 78a64945a16444c5ad549235dbc52309 P ece2a20b74874eb7acf32b82a1691f97: Bootstrap starting.
I20260923 22:12:27.740154 3797 tablet_bootstrap.cc:492] T 78a64945a16444c5ad549235dbc52309 P d269cfa9b6874c6fb8b07b0da9f00ea5: Bootstrap starting.
I20260923 22:12:27.740401 3800 tablet_bootstrap.cc:654] T 78a64945a16444c5ad549235dbc52309 P 04978c60cca64c6783ac03221353dbe6: Neither blocks nor log segments found. Creating new log.
I20260923 22:12:27.740603 3796 tablet_bootstrap.cc:654] T 78a64945a16444c5ad549235dbc52309 P ece2a20b74874eb7acf32b82a1691f97: Neither blocks nor log segments found. Creating new log.
I20260923 22:12:27.740783 3797 tablet_bootstrap.cc:654] T 78a64945a16444c5ad549235dbc52309 P d269cfa9b6874c6fb8b07b0da9f00ea5: Neither blocks nor log segments found. Creating new log.
I20260923 22:12:27.741281 3796 tablet_bootstrap.cc:492] T 78a64945a16444c5ad549235dbc52309 P ece2a20b74874eb7acf32b82a1691f97: No bootstrap required, opened a new log
I20260923 22:12:27.741338 3796 ts_tablet_manager.cc:1397] T 78a64945a16444c5ad549235dbc52309 P ece2a20b74874eb7acf32b82a1691f97: Time spent bootstrapping tablet: real 0.001s user 0.001s sys 0.000s
I20260923 22:12:27.741473 3797 tablet_bootstrap.cc:492] T 78a64945a16444c5ad549235dbc52309 P d269cfa9b6874c6fb8b07b0da9f00ea5: No bootstrap required, opened a new log
I20260923 22:12:27.741519 3797 ts_tablet_manager.cc:1397] T 78a64945a16444c5ad549235dbc52309 P d269cfa9b6874c6fb8b07b0da9f00ea5: Time spent bootstrapping tablet: real 0.001s user 0.000s sys 0.001s
I20260923 22:12:27.741478 3796 raft_consensus.cc:348] T 78a64945a16444c5ad549235dbc52309 P ece2a20b74874eb7acf32b82a1691f97 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "04978c60cca64c6783ac03221353dbe6" member_type: VOTER last_known_addr { host: "127.1.135.193" port: 42577 } } peers { permanent_uuid: "d269cfa9b6874c6fb8b07b0da9f00ea5" member_type: VOTER last_known_addr { host: "127.1.135.194" port: 36299 } } peers { permanent_uuid: "ece2a20b74874eb7acf32b82a1691f97" member_type: VOTER last_known_addr { host: "127.1.135.195" port: 35699 } }
I20260923 22:12:27.741555 3796 raft_consensus.cc:374] T 78a64945a16444c5ad549235dbc52309 P ece2a20b74874eb7acf32b82a1691f97 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260923 22:12:27.741580 3796 raft_consensus.cc:729] T 78a64945a16444c5ad549235dbc52309 P ece2a20b74874eb7acf32b82a1691f97 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: ece2a20b74874eb7acf32b82a1691f97, State: Initialized, Role: FOLLOWER
I20260923 22:12:27.741632 3796 consensus_queue.cc:260] T 78a64945a16444c5ad549235dbc52309 P ece2a20b74874eb7acf32b82a1691f97 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "04978c60cca64c6783ac03221353dbe6" member_type: VOTER last_known_addr { host: "127.1.135.193" port: 42577 } } peers { permanent_uuid: "d269cfa9b6874c6fb8b07b0da9f00ea5" member_type: VOTER last_known_addr { host: "127.1.135.194" port: 36299 } } peers { permanent_uuid: "ece2a20b74874eb7acf32b82a1691f97" member_type: VOTER last_known_addr { host: "127.1.135.195" port: 35699 } }
I20260923 22:12:27.741689 3800 tablet_bootstrap.cc:492] T 78a64945a16444c5ad549235dbc52309 P 04978c60cca64c6783ac03221353dbe6: No bootstrap required, opened a new log
I20260923 22:12:27.741674 3797 raft_consensus.cc:348] T 78a64945a16444c5ad549235dbc52309 P d269cfa9b6874c6fb8b07b0da9f00ea5 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "04978c60cca64c6783ac03221353dbe6" member_type: VOTER last_known_addr { host: "127.1.135.193" port: 42577 } } peers { permanent_uuid: "d269cfa9b6874c6fb8b07b0da9f00ea5" member_type: VOTER last_known_addr { host: "127.1.135.194" port: 36299 } } peers { permanent_uuid: "ece2a20b74874eb7acf32b82a1691f97" member_type: VOTER last_known_addr { host: "127.1.135.195" port: 35699 } }
I20260923 22:12:27.741727 3800 ts_tablet_manager.cc:1397] T 78a64945a16444c5ad549235dbc52309 P 04978c60cca64c6783ac03221353dbe6: Time spent bootstrapping tablet: real 0.002s user 0.000s sys 0.001s
I20260923 22:12:27.741737 3797 raft_consensus.cc:374] T 78a64945a16444c5ad549235dbc52309 P d269cfa9b6874c6fb8b07b0da9f00ea5 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260923 22:12:27.741735 3796 ts_tablet_manager.cc:1428] T 78a64945a16444c5ad549235dbc52309 P ece2a20b74874eb7acf32b82a1691f97: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20260923 22:12:27.741757 3797 raft_consensus.cc:729] T 78a64945a16444c5ad549235dbc52309 P d269cfa9b6874c6fb8b07b0da9f00ea5 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d269cfa9b6874c6fb8b07b0da9f00ea5, State: Initialized, Role: FOLLOWER
I20260923 22:12:27.741765 3802 tablet_replica.cc:439] T 78a64945a16444c5ad549235dbc52309 P ece2a20b74874eb7acf32b82a1691f97: TxnStatusTablet state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "04978c60cca64c6783ac03221353dbe6" member_type: VOTER last_known_addr { host: "127.1.135.193" port: 42577 } } peers { permanent_uuid: "d269cfa9b6874c6fb8b07b0da9f00ea5" member_type: VOTER last_known_addr { host: "127.1.135.194" port: 36299 } } peers { permanent_uuid: "ece2a20b74874eb7acf32b82a1691f97" member_type: VOTER last_known_addr { host: "127.1.135.195" port: 35699 } } }
I20260923 22:12:27.741799 3797 consensus_queue.cc:260] T 78a64945a16444c5ad549235dbc52309 P d269cfa9b6874c6fb8b07b0da9f00ea5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "04978c60cca64c6783ac03221353dbe6" member_type: VOTER last_known_addr { host: "127.1.135.193" port: 42577 } } peers { permanent_uuid: "d269cfa9b6874c6fb8b07b0da9f00ea5" member_type: VOTER last_known_addr { host: "127.1.135.194" port: 36299 } } peers { permanent_uuid: "ece2a20b74874eb7acf32b82a1691f97" member_type: VOTER last_known_addr { host: "127.1.135.195" port: 35699 } }
I20260923 22:12:27.741840 3802 tablet_replica.cc:442] T 78a64945a16444c5ad549235dbc52309 P ece2a20b74874eb7acf32b82a1691f97: This TxnStatusTablet replica's current role is: FOLLOWER
I20260923 22:12:27.741879 3797 ts_tablet_manager.cc:1428] T 78a64945a16444c5ad549235dbc52309 P d269cfa9b6874c6fb8b07b0da9f00ea5: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20260923 22:12:27.741884 3800 raft_consensus.cc:348] T 78a64945a16444c5ad549235dbc52309 P 04978c60cca64c6783ac03221353dbe6 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "04978c60cca64c6783ac03221353dbe6" member_type: VOTER last_known_addr { host: "127.1.135.193" port: 42577 } } peers { permanent_uuid: "d269cfa9b6874c6fb8b07b0da9f00ea5" member_type: VOTER last_known_addr { host: "127.1.135.194" port: 36299 } } peers { permanent_uuid: "ece2a20b74874eb7acf32b82a1691f97" member_type: VOTER last_known_addr { host: "127.1.135.195" port: 35699 } }
I20260923 22:12:27.741941 3800 raft_consensus.cc:374] T 78a64945a16444c5ad549235dbc52309 P 04978c60cca64c6783ac03221353dbe6 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260923 22:12:27.741961 3800 raft_consensus.cc:729] T 78a64945a16444c5ad549235dbc52309 P 04978c60cca64c6783ac03221353dbe6 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 04978c60cca64c6783ac03221353dbe6, State: Initialized, Role: FOLLOWER
I20260923 22:12:27.741957 3803 tablet_replica.cc:439] T 78a64945a16444c5ad549235dbc52309 P d269cfa9b6874c6fb8b07b0da9f00ea5: TxnStatusTablet state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "04978c60cca64c6783ac03221353dbe6" member_type: VOTER last_known_addr { host: "127.1.135.193" port: 42577 } } peers { permanent_uuid: "d269cfa9b6874c6fb8b07b0da9f00ea5" member_type: VOTER last_known_addr { host: "127.1.135.194" port: 36299 } } peers { permanent_uuid: "ece2a20b74874eb7acf32b82a1691f97" member_type: VOTER last_known_addr { host: "127.1.135.195" port: 35699 } } }
I20260923 22:12:27.742005 3800 consensus_queue.cc:260] T 78a64945a16444c5ad549235dbc52309 P 04978c60cca64c6783ac03221353dbe6 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "04978c60cca64c6783ac03221353dbe6" member_type: VOTER last_known_addr { host: "127.1.135.193" port: 42577 } } peers { permanent_uuid: "d269cfa9b6874c6fb8b07b0da9f00ea5" member_type: VOTER last_known_addr { host: "127.1.135.194" port: 36299 } } peers { permanent_uuid: "ece2a20b74874eb7acf32b82a1691f97" member_type: VOTER last_known_addr { host: "127.1.135.195" port: 35699 } }
I20260923 22:12:27.742051 3803 tablet_replica.cc:442] T 78a64945a16444c5ad549235dbc52309 P d269cfa9b6874c6fb8b07b0da9f00ea5: This TxnStatusTablet replica's current role is: FOLLOWER
I20260923 22:12:27.742117 3800 ts_tablet_manager.cc:1428] T 78a64945a16444c5ad549235dbc52309 P 04978c60cca64c6783ac03221353dbe6: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20260923 22:12:27.742259 3804 tablet_replica.cc:439] T 78a64945a16444c5ad549235dbc52309 P 04978c60cca64c6783ac03221353dbe6: TxnStatusTablet state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "04978c60cca64c6783ac03221353dbe6" member_type: VOTER last_known_addr { host: "127.1.135.193" port: 42577 } } peers { permanent_uuid: "d269cfa9b6874c6fb8b07b0da9f00ea5" member_type: VOTER last_known_addr { host: "127.1.135.194" port: 36299 } } peers { permanent_uuid: "ece2a20b74874eb7acf32b82a1691f97" member_type: VOTER last_known_addr { host: "127.1.135.195" port: 35699 } } }
I20260923 22:12:27.742339 3804 tablet_replica.cc:442] T 78a64945a16444c5ad549235dbc52309 P 04978c60cca64c6783ac03221353dbe6: This TxnStatusTablet replica's current role is: FOLLOWER
I20260923 22:12:27.975337 3802 raft_consensus.cc:482] T 78a64945a16444c5ad549235dbc52309 P ece2a20b74874eb7acf32b82a1691f97 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260923 22:12:27.975454 3802 raft_consensus.cc:504] T 78a64945a16444c5ad549235dbc52309 P ece2a20b74874eb7acf32b82a1691f97 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "04978c60cca64c6783ac03221353dbe6" member_type: VOTER last_known_addr { host: "127.1.135.193" port: 42577 } } peers { permanent_uuid: "d269cfa9b6874c6fb8b07b0da9f00ea5" member_type: VOTER last_known_addr { host: "127.1.135.194" port: 36299 } } peers { permanent_uuid: "ece2a20b74874eb7acf32b82a1691f97" member_type: VOTER last_known_addr { host: "127.1.135.195" port: 35699 } }
I20260923 22:12:27.975646 3802 leader_election.cc:288] T 78a64945a16444c5ad549235dbc52309 P ece2a20b74874eb7acf32b82a1691f97 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 04978c60cca64c6783ac03221353dbe6 (127.1.135.193:42577), d269cfa9b6874c6fb8b07b0da9f00ea5 (127.1.135.194:36299)
I20260923 22:12:27.975857 3590 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "78a64945a16444c5ad549235dbc52309" candidate_uuid: "ece2a20b74874eb7acf32b82a1691f97" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d269cfa9b6874c6fb8b07b0da9f00ea5" is_pre_election: true
I20260923 22:12:27.975886 3450 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "78a64945a16444c5ad549235dbc52309" candidate_uuid: "ece2a20b74874eb7acf32b82a1691f97" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "04978c60cca64c6783ac03221353dbe6" is_pre_election: true
I20260923 22:12:27.975978 3590 raft_consensus.cc:2454] T 78a64945a16444c5ad549235dbc52309 P d269cfa9b6874c6fb8b07b0da9f00ea5 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ece2a20b74874eb7acf32b82a1691f97 in term 0.
I20260923 22:12:27.975987 3450 raft_consensus.cc:2454] T 78a64945a16444c5ad549235dbc52309 P 04978c60cca64c6783ac03221353dbe6 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ece2a20b74874eb7acf32b82a1691f97 in term 0.
I20260923 22:12:27.976157 3662 leader_election.cc:302] T 78a64945a16444c5ad549235dbc52309 P ece2a20b74874eb7acf32b82a1691f97 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 04978c60cca64c6783ac03221353dbe6, ece2a20b74874eb7acf32b82a1691f97; no voters:
I20260923 22:12:27.976282 3802 raft_consensus.cc:2781] T 78a64945a16444c5ad549235dbc52309 P ece2a20b74874eb7acf32b82a1691f97 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260923 22:12:27.976320 3802 raft_consensus.cc:482] T 78a64945a16444c5ad549235dbc52309 P ece2a20b74874eb7acf32b82a1691f97 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260923 22:12:27.976347 3802 raft_consensus.cc:3037] T 78a64945a16444c5ad549235dbc52309 P ece2a20b74874eb7acf32b82a1691f97 [term 0 FOLLOWER]: Advancing to term 1
I20260923 22:12:27.976956 3802 raft_consensus.cc:504] T 78a64945a16444c5ad549235dbc52309 P ece2a20b74874eb7acf32b82a1691f97 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "04978c60cca64c6783ac03221353dbe6" member_type: VOTER last_known_addr { host: "127.1.135.193" port: 42577 } } peers { permanent_uuid: "d269cfa9b6874c6fb8b07b0da9f00ea5" member_type: VOTER last_known_addr { host: "127.1.135.194" port: 36299 } } peers { permanent_uuid: "ece2a20b74874eb7acf32b82a1691f97" member_type: VOTER last_known_addr { host: "127.1.135.195" port: 35699 } }
I20260923 22:12:27.977087 3802 leader_election.cc:288] T 78a64945a16444c5ad549235dbc52309 P ece2a20b74874eb7acf32b82a1691f97 [CANDIDATE]: Term 1 election: Requested vote from peers 04978c60cca64c6783ac03221353dbe6 (127.1.135.193:42577), d269cfa9b6874c6fb8b07b0da9f00ea5 (127.1.135.194:36299)
I20260923 22:12:27.977239 3450 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "78a64945a16444c5ad549235dbc52309" candidate_uuid: "ece2a20b74874eb7acf32b82a1691f97" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "04978c60cca64c6783ac03221353dbe6"
I20260923 22:12:27.977301 3450 raft_consensus.cc:3037] T 78a64945a16444c5ad549235dbc52309 P 04978c60cca64c6783ac03221353dbe6 [term 0 FOLLOWER]: Advancing to term 1
I20260923 22:12:27.977246 3590 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "78a64945a16444c5ad549235dbc52309" candidate_uuid: "ece2a20b74874eb7acf32b82a1691f97" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d269cfa9b6874c6fb8b07b0da9f00ea5"
I20260923 22:12:27.977336 3590 raft_consensus.cc:3037] T 78a64945a16444c5ad549235dbc52309 P d269cfa9b6874c6fb8b07b0da9f00ea5 [term 0 FOLLOWER]: Advancing to term 1
I20260923 22:12:27.977885 3450 raft_consensus.cc:2454] T 78a64945a16444c5ad549235dbc52309 P 04978c60cca64c6783ac03221353dbe6 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ece2a20b74874eb7acf32b82a1691f97 in term 1.
I20260923 22:12:27.978030 3662 leader_election.cc:302] T 78a64945a16444c5ad549235dbc52309 P ece2a20b74874eb7acf32b82a1691f97 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 04978c60cca64c6783ac03221353dbe6, ece2a20b74874eb7acf32b82a1691f97; no voters:
I20260923 22:12:27.978029 3590 raft_consensus.cc:2454] T 78a64945a16444c5ad549235dbc52309 P d269cfa9b6874c6fb8b07b0da9f00ea5 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ece2a20b74874eb7acf32b82a1691f97 in term 1.
I20260923 22:12:27.978123 3802 raft_consensus.cc:2781] T 78a64945a16444c5ad549235dbc52309 P ece2a20b74874eb7acf32b82a1691f97 [term 1 FOLLOWER]: Leader election won for term 1
I20260923 22:12:27.978168 3802 raft_consensus.cc:686] T 78a64945a16444c5ad549235dbc52309 P ece2a20b74874eb7acf32b82a1691f97 [term 1 LEADER]: Becoming Leader. State: Replica: ece2a20b74874eb7acf32b82a1691f97, State: Running, Role: LEADER
I20260923 22:12:27.978214 3802 consensus_queue.cc:237] T 78a64945a16444c5ad549235dbc52309 P ece2a20b74874eb7acf32b82a1691f97 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "04978c60cca64c6783ac03221353dbe6" member_type: VOTER last_known_addr { host: "127.1.135.193" port: 42577 } } peers { permanent_uuid: "d269cfa9b6874c6fb8b07b0da9f00ea5" member_type: VOTER last_known_addr { host: "127.1.135.194" port: 36299 } } peers { permanent_uuid: "ece2a20b74874eb7acf32b82a1691f97" member_type: VOTER last_known_addr { host: "127.1.135.195" port: 35699 } }
I20260923 22:12:27.978479 3802 tablet_replica.cc:439] T 78a64945a16444c5ad549235dbc52309 P ece2a20b74874eb7acf32b82a1691f97: TxnStatusTablet state changed. Reason: New leader ece2a20b74874eb7acf32b82a1691f97. Latest consensus state: current_term: 1 leader_uuid: "ece2a20b74874eb7acf32b82a1691f97" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "04978c60cca64c6783ac03221353dbe6" member_type: VOTER last_known_addr { host: "127.1.135.193" port: 42577 } } peers { permanent_uuid: "d269cfa9b6874c6fb8b07b0da9f00ea5" member_type: VOTER last_known_addr { host: "127.1.135.194" port: 36299 } } peers { permanent_uuid: "ece2a20b74874eb7acf32b82a1691f97" member_type: VOTER last_known_addr { host: "127.1.135.195" port: 35699 } } }
I20260923 22:12:27.978536 3802 tablet_replica.cc:442] T 78a64945a16444c5ad549235dbc52309 P ece2a20b74874eb7acf32b82a1691f97: This TxnStatusTablet replica's current role is: LEADER
I20260923 22:12:27.978705 3818 txn_status_manager.cc:873] Waiting until node catch up with all replicated operations in previous term...
I20260923 22:12:27.978850 3818 txn_status_manager.cc:929] Loading transaction status metadata into memory...
I20260923 22:12:27.978824 3292 catalog_manager.cc:5495] T 78a64945a16444c5ad549235dbc52309 P ece2a20b74874eb7acf32b82a1691f97 reported cstate change: term changed from 0 to 1, leader changed from <none> to ece2a20b74874eb7acf32b82a1691f97 (127.1.135.195). New cstate: current_term: 1 leader_uuid: "ece2a20b74874eb7acf32b82a1691f97" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "04978c60cca64c6783ac03221353dbe6" member_type: VOTER last_known_addr { host: "127.1.135.193" port: 42577 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "d269cfa9b6874c6fb8b07b0da9f00ea5" member_type: VOTER last_known_addr { host: "127.1.135.194" port: 36299 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "ece2a20b74874eb7acf32b82a1691f97" member_type: VOTER last_known_addr { host: "127.1.135.195" port: 35699 } health_report { overall_health: HEALTHY } } }
I20260923 22:12:28.025499 3821 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 22:12:28.022925 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41670 (local address 127.1.135.195:35699)
0923 22:12:28.023056 (+ 131us) server_negotiation.cc:206] Beginning negotiation
0923 22:12:28.023058 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0923 22:12:28.023064 (+ 6us) server_negotiation.cc:407] Connection header received
0923 22:12:28.023106 (+ 42us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 22:12:28.023107 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 22:12:28.023122 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 22:12:28.023199 (+ 77us) server_negotiation.cc:226] Negotiated authn=TOKEN
0923 22:12:28.023312 (+ 113us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 22:12:28.023825 (+ 513us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 22:12:28.024647 (+ 822us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 22:12:28.024897 (+ 250us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 22:12:28.024975 (+ 78us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 22:12:28.025165 (+ 190us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0923 22:12:28.025269 (+ 104us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0923 22:12:28.025352 (+ 83us) server_negotiation.cc:1035] Waiting for connection context
0923 22:12:28.025394 (+ 42us) server_negotiation.cc:299] Negotiation successful
0923 22:12:28.025416 (+ 22us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":95,"thread_start_us":52,"threads_started":1}
I20260923 22:12:28.025511 3813 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 22:12:28.022846 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.135.195:35699 (local address 127.0.0.1:41670)
0923 22:12:28.022914 (+ 68us) negotiation.cc:107] Waiting for socket to connect
0923 22:12:28.022926 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0923 22:12:28.022969 (+ 43us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0923 22:12:28.023204 (+ 235us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0923 22:12:28.023205 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0923 22:12:28.023206 (+ 1us) client_negotiation.cc:189] Negotiated authn=TOKEN
0923 22:12:28.023257 (+ 51us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 22:12:28.023260 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 22:12:28.023924 (+ 664us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 22:12:28.023925 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 22:12:28.024562 (+ 637us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 22:12:28.024565 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 22:12:28.024975 (+ 410us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 22:12:28.024976 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 22:12:28.025040 (+ 64us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 22:12:28.025057 (+ 17us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0923 22:12:28.025341 (+ 284us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0923 22:12:28.025349 (+ 8us) client_negotiation.cc:769] Sending connection context
0923 22:12:28.025401 (+ 52us) client_negotiation.cc:240] Negotiation successful
0923 22:12:28.025443 (+ 42us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":31}
I20260923 22:12:28.027092 3590 raft_consensus.cc:1261] T 78a64945a16444c5ad549235dbc52309 P d269cfa9b6874c6fb8b07b0da9f00ea5 [term 1 FOLLOWER]: Refusing update from remote peer ece2a20b74874eb7acf32b82a1691f97: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260923 22:12:28.027139 3450 raft_consensus.cc:1261] T 78a64945a16444c5ad549235dbc52309 P 04978c60cca64c6783ac03221353dbe6 [term 1 FOLLOWER]: Refusing update from remote peer ece2a20b74874eb7acf32b82a1691f97: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260923 22:12:28.027473 3802 consensus_queue.cc:1035] T 78a64945a16444c5ad549235dbc52309 P ece2a20b74874eb7acf32b82a1691f97 [LEADER]: Connected to new peer: Peer: permanent_uuid: "d269cfa9b6874c6fb8b07b0da9f00ea5" member_type: VOTER last_known_addr { host: "127.1.135.194" port: 36299 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260923 22:12:28.027611 3806 consensus_queue.cc:1035] T 78a64945a16444c5ad549235dbc52309 P ece2a20b74874eb7acf32b82a1691f97 [LEADER]: Connected to new peer: Peer: permanent_uuid: "04978c60cca64c6783ac03221353dbe6" member_type: VOTER last_known_addr { host: "127.1.135.193" port: 42577 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260923 22:12:28.029122 3803 tablet_replica.cc:439] T 78a64945a16444c5ad549235dbc52309 P d269cfa9b6874c6fb8b07b0da9f00ea5: TxnStatusTablet state changed. Reason: New leader ece2a20b74874eb7acf32b82a1691f97. Latest consensus state: current_term: 1 leader_uuid: "ece2a20b74874eb7acf32b82a1691f97" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "04978c60cca64c6783ac03221353dbe6" member_type: VOTER last_known_addr { host: "127.1.135.193" port: 42577 } } peers { permanent_uuid: "d269cfa9b6874c6fb8b07b0da9f00ea5" member_type: VOTER last_known_addr { host: "127.1.135.194" port: 36299 } } peers { permanent_uuid: "ece2a20b74874eb7acf32b82a1691f97" member_type: VOTER last_known_addr { host: "127.1.135.195" port: 35699 } } }
I20260923 22:12:28.029211 3803 tablet_replica.cc:442] T 78a64945a16444c5ad549235dbc52309 P d269cfa9b6874c6fb8b07b0da9f00ea5: This TxnStatusTablet replica's current role is: FOLLOWER
I20260923 22:12:28.029536 3822 mvcc.cc:205] Tried to move back new op lower bound from 7332665540716556288 to 7332665540519317504. Current Snapshot: MvccSnapshot[applied={T|T < 7332665540716556288}]
I20260923 22:12:28.029597 3806 tablet_replica.cc:439] T 78a64945a16444c5ad549235dbc52309 P ece2a20b74874eb7acf32b82a1691f97: TxnStatusTablet state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "ece2a20b74874eb7acf32b82a1691f97" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "04978c60cca64c6783ac03221353dbe6" member_type: VOTER last_known_addr { host: "127.1.135.193" port: 42577 } } peers { permanent_uuid: "d269cfa9b6874c6fb8b07b0da9f00ea5" member_type: VOTER last_known_addr { host: "127.1.135.194" port: 36299 } } peers { permanent_uuid: "ece2a20b74874eb7acf32b82a1691f97" member_type: VOTER last_known_addr { host: "127.1.135.195" port: 35699 } } }
I20260923 22:12:28.029676 3806 tablet_replica.cc:442] T 78a64945a16444c5ad549235dbc52309 P ece2a20b74874eb7acf32b82a1691f97: This TxnStatusTablet replica's current role is: LEADER
I20260923 22:12:28.030249 3804 tablet_replica.cc:439] T 78a64945a16444c5ad549235dbc52309 P 04978c60cca64c6783ac03221353dbe6: TxnStatusTablet state changed. Reason: New leader ece2a20b74874eb7acf32b82a1691f97. Latest consensus state: current_term: 1 leader_uuid: "ece2a20b74874eb7acf32b82a1691f97" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "04978c60cca64c6783ac03221353dbe6" member_type: VOTER last_known_addr { host: "127.1.135.193" port: 42577 } } peers { permanent_uuid: "d269cfa9b6874c6fb8b07b0da9f00ea5" member_type: VOTER last_known_addr { host: "127.1.135.194" port: 36299 } } peers { permanent_uuid: "ece2a20b74874eb7acf32b82a1691f97" member_type: VOTER last_known_addr { host: "127.1.135.195" port: 35699 } } }
I20260923 22:12:28.030334 3804 tablet_replica.cc:442] T 78a64945a16444c5ad549235dbc52309 P 04978c60cca64c6783ac03221353dbe6: This TxnStatusTablet replica's current role is: FOLLOWER
I20260923 22:12:28.030318 3806 tablet_replica.cc:439] T 78a64945a16444c5ad549235dbc52309 P ece2a20b74874eb7acf32b82a1691f97: TxnStatusTablet state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "ece2a20b74874eb7acf32b82a1691f97" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "04978c60cca64c6783ac03221353dbe6" member_type: VOTER last_known_addr { host: "127.1.135.193" port: 42577 } } peers { permanent_uuid: "d269cfa9b6874c6fb8b07b0da9f00ea5" member_type: VOTER last_known_addr { host: "127.1.135.194" port: 36299 } } peers { permanent_uuid: "ece2a20b74874eb7acf32b82a1691f97" member_type: VOTER last_known_addr { host: "127.1.135.195" port: 35699 } } }
I20260923 22:12:28.030377 3804 tablet_replica.cc:439] T 78a64945a16444c5ad549235dbc52309 P 04978c60cca64c6783ac03221353dbe6: TxnStatusTablet state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "ece2a20b74874eb7acf32b82a1691f97" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "04978c60cca64c6783ac03221353dbe6" member_type: VOTER last_known_addr { host: "127.1.135.193" port: 42577 } } peers { permanent_uuid: "d269cfa9b6874c6fb8b07b0da9f00ea5" member_type: VOTER last_known_addr { host: "127.1.135.194" port: 36299 } } peers { permanent_uuid: "ece2a20b74874eb7acf32b82a1691f97" member_type: VOTER last_known_addr { host: "127.1.135.195" port: 35699 } } }
I20260923 22:12:28.030385 3806 tablet_replica.cc:442] T 78a64945a16444c5ad549235dbc52309 P ece2a20b74874eb7acf32b82a1691f97: This TxnStatusTablet replica's current role is: LEADER
I20260923 22:12:28.030409 3804 tablet_replica.cc:442] T 78a64945a16444c5ad549235dbc52309 P 04978c60cca64c6783ac03221353dbe6: This TxnStatusTablet replica's current role is: FOLLOWER
I20260923 22:12:28.031139 3803 tablet_replica.cc:439] T 78a64945a16444c5ad549235dbc52309 P d269cfa9b6874c6fb8b07b0da9f00ea5: TxnStatusTablet state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "ece2a20b74874eb7acf32b82a1691f97" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "04978c60cca64c6783ac03221353dbe6" member_type: VOTER last_known_addr { host: "127.1.135.193" port: 42577 } } peers { permanent_uuid: "d269cfa9b6874c6fb8b07b0da9f00ea5" member_type: VOTER last_known_addr { host: "127.1.135.194" port: 36299 } } peers { permanent_uuid: "ece2a20b74874eb7acf32b82a1691f97" member_type: VOTER last_known_addr { host: "127.1.135.195" port: 35699 } } }
I20260923 22:12:28.031224 3803 tablet_replica.cc:442] T 78a64945a16444c5ad549235dbc52309 P d269cfa9b6874c6fb8b07b0da9f00ea5: This TxnStatusTablet replica's current role is: FOLLOWER
I20260923 22:12:28.032253 3824 mvcc.cc:205] Tried to move back new op lower bound from 7332665540716556288 to 7332665540519317504. Current Snapshot: MvccSnapshot[applied={T|T < 7332665540716556288}]
I20260923 22:12:28.034916 3821 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 22:12:28.032263 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41678 (local address 127.1.135.195:35699)
0923 22:12:28.032298 (+ 35us) server_negotiation.cc:206] Beginning negotiation
0923 22:12:28.032300 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0923 22:12:28.032449 (+ 149us) server_negotiation.cc:407] Connection header received
0923 22:12:28.032539 (+ 90us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 22:12:28.032540 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 22:12:28.032550 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 22:12:28.032576 (+ 26us) server_negotiation.cc:226] Negotiated authn=TOKEN
0923 22:12:28.032747 (+ 171us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 22:12:28.033458 (+ 711us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 22:12:28.034214 (+ 756us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 22:12:28.034508 (+ 294us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 22:12:28.034584 (+ 76us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 22:12:28.034697 (+ 113us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0923 22:12:28.034726 (+ 29us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0923 22:12:28.034779 (+ 53us) server_negotiation.cc:1035] Waiting for connection context
0923 22:12:28.034834 (+ 55us) server_negotiation.cc:299] Negotiation successful
0923 22:12:28.034853 (+ 19us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":17}
I20260923 22:12:28.039470 3821 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 22:12:28.037182 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.135.195:33601 (local address 127.1.135.195:35699)
0923 22:12:28.037216 (+ 34us) server_negotiation.cc:206] Beginning negotiation
0923 22:12:28.037218 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0923 22:12:28.037321 (+ 103us) server_negotiation.cc:407] Connection header received
0923 22:12:28.037378 (+ 57us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 22:12:28.037379 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 22:12:28.037387 (+ 8us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 22:12:28.037447 (+ 60us) server_negotiation.cc:226] Negotiated authn=TOKEN
0923 22:12:28.037526 (+ 79us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 22:12:28.037939 (+ 413us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 22:12:28.038801 (+ 862us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 22:12:28.038982 (+ 181us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 22:12:28.039051 (+ 69us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 22:12:28.039209 (+ 158us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0923 22:12:28.039236 (+ 27us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0923 22:12:28.039300 (+ 64us) server_negotiation.cc:1035] Waiting for connection context
0923 22:12:28.039356 (+ 56us) server_negotiation.cc:299] Negotiation successful
0923 22:12:28.039376 (+ 20us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":2,"server-negotiator.queue_time_us":17}
I20260923 22:12:28.039472 3836 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 22:12:28.037111 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.135.195:35699 (local address 127.1.135.195:33601)
0923 22:12:28.037252 (+ 141us) negotiation.cc:107] Waiting for socket to connect
0923 22:12:28.037260 (+ 8us) client_negotiation.cc:174] Beginning negotiation
0923 22:12:28.037335 (+ 75us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0923 22:12:28.037449 (+ 114us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0923 22:12:28.037450 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0923 22:12:28.037451 (+ 1us) client_negotiation.cc:189] Negotiated authn=TOKEN
0923 22:12:28.037483 (+ 32us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 22:12:28.037484 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 22:12:28.038044 (+ 560us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 22:12:28.038045 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 22:12:28.038710 (+ 665us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 22:12:28.038712 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 22:12:28.039048 (+ 336us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 22:12:28.039049 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 22:12:28.039096 (+ 47us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 22:12:28.039108 (+ 12us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0923 22:12:28.039300 (+ 192us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0923 22:12:28.039302 (+ 2us) client_negotiation.cc:769] Sending connection context
0923 22:12:28.039352 (+ 50us) client_negotiation.cc:240] Negotiation successful
0923 22:12:28.039395 (+ 43us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":110,"mutex_wait_us":4,"thread_start_us":78,"threads_started":1}
I20260923 22:12:28.041512 3590 raft_consensus.cc:1261] T 9fd0c8c6b7b541798f3da5d6dab48ddd P d269cfa9b6874c6fb8b07b0da9f00ea5 [term 1 FOLLOWER]: Refusing update from remote peer ece2a20b74874eb7acf32b82a1691f97: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260923 22:12:28.041512 3450 raft_consensus.cc:1261] T 9fd0c8c6b7b541798f3da5d6dab48ddd P 04978c60cca64c6783ac03221353dbe6 [term 1 FOLLOWER]: Refusing update from remote peer ece2a20b74874eb7acf32b82a1691f97: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260923 22:12:28.041697 3830 consensus_queue.cc:1035] T 9fd0c8c6b7b541798f3da5d6dab48ddd P ece2a20b74874eb7acf32b82a1691f97 [LEADER]: Connected to new peer: Peer: permanent_uuid: "d269cfa9b6874c6fb8b07b0da9f00ea5" member_type: VOTER last_known_addr { host: "127.1.135.194" port: 36299 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260923 22:12:28.041791 3806 consensus_queue.cc:1035] T 9fd0c8c6b7b541798f3da5d6dab48ddd P ece2a20b74874eb7acf32b82a1691f97 [LEADER]: Connected to new peer: Peer: permanent_uuid: "04978c60cca64c6783ac03221353dbe6" member_type: VOTER last_known_addr { host: "127.1.135.193" port: 42577 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260923 22:12:28.054854 3293 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:35744:
table { table_name: "test-table" } modify_external_catalogs: true
I20260923 22:12:28.054961 3293 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:35744:
table { table_name: "test-table" } modify_external_catalogs: true
I20260923 22:12:28.055670 3293 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 24a871a68671419f89d1a0d5a2bdcd3f: Sending DeleteTablet for 3 replicas of tablet 9fd0c8c6b7b541798f3da5d6dab48ddd
I20260923 22:12:28.056006 3430 tablet_service.cc:1514] Processing DeleteTablet for tablet 9fd0c8c6b7b541798f3da5d6dab48ddd with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-23 22:12:28 UTC) from {username='kudu', principal='kudu/127.1.135.254@KRBTEST.COM'} at 127.0.0.1:37922
I20260923 22:12:28.056088 3570 tablet_service.cc:1514] Processing DeleteTablet for tablet 9fd0c8c6b7b541798f3da5d6dab48ddd with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-23 22:12:28 UTC) from {username='kudu', principal='kudu/127.1.135.254@KRBTEST.COM'} at 127.0.0.1:48168
I20260923 22:12:28.056175 3293 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:35744:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260923 22:12:28.056303 3293 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260923 22:12:28.056367 3842 tablet_replica.cc:330] T 9fd0c8c6b7b541798f3da5d6dab48ddd P d269cfa9b6874c6fb8b07b0da9f00ea5: stopping tablet replica
I20260923 22:12:28.056355 3710 tablet_service.cc:1514] Processing DeleteTablet for tablet 9fd0c8c6b7b541798f3da5d6dab48ddd with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-23 22:12:28 UTC) from {username='kudu', principal='kudu/127.1.135.254@KRBTEST.COM'} at 127.0.0.1:41660
I20260923 22:12:28.056457 3842 raft_consensus.cc:2229] T 9fd0c8c6b7b541798f3da5d6dab48ddd P d269cfa9b6874c6fb8b07b0da9f00ea5 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260923 22:12:28.056507 3842 raft_consensus.cc:2258] T 9fd0c8c6b7b541798f3da5d6dab48ddd P d269cfa9b6874c6fb8b07b0da9f00ea5 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260923 22:12:28.056859 3842 ts_tablet_manager.cc:1905] T 9fd0c8c6b7b541798f3da5d6dab48ddd P d269cfa9b6874c6fb8b07b0da9f00ea5: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260923 22:12:28.057304 3843 tablet_replica.cc:330] T 9fd0c8c6b7b541798f3da5d6dab48ddd P ece2a20b74874eb7acf32b82a1691f97: stopping tablet replica
I20260923 22:12:28.057431 3843 raft_consensus.cc:2229] T 9fd0c8c6b7b541798f3da5d6dab48ddd P ece2a20b74874eb7acf32b82a1691f97 [term 1 LEADER]: Raft consensus shutting down.
I20260923 22:12:28.057463 3710 tablet_service.cc:1467] Processing CreateTablet for tablet 3d2bade3ede8413fbc76a068a1844cc7 (DEFAULT_TABLE table=test-table [id=f44f81eaa8d64f9a8def07c3a588a570]), partition=RANGE (key) PARTITION UNBOUNDED
I20260923 22:12:28.057482 3570 tablet_service.cc:1467] Processing CreateTablet for tablet 3d2bade3ede8413fbc76a068a1844cc7 (DEFAULT_TABLE table=test-table [id=f44f81eaa8d64f9a8def07c3a588a570]), partition=RANGE (key) PARTITION UNBOUNDED
I20260923 22:12:28.057583 3570 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 3d2bade3ede8413fbc76a068a1844cc7. 1 dirs total, 0 dirs full, 0 dirs failed
I20260923 22:12:28.057581 3710 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 3d2bade3ede8413fbc76a068a1844cc7. 1 dirs total, 0 dirs full, 0 dirs failed
I20260923 22:12:28.058516 3796 tablet_bootstrap.cc:492] T 3d2bade3ede8413fbc76a068a1844cc7 P ece2a20b74874eb7acf32b82a1691f97: Bootstrap starting.
I20260923 22:12:28.057570 3843 raft_consensus.cc:2258] T 9fd0c8c6b7b541798f3da5d6dab48ddd P ece2a20b74874eb7acf32b82a1691f97 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260923 22:12:28.058847 3797 tablet_bootstrap.cc:492] T 3d2bade3ede8413fbc76a068a1844cc7 P d269cfa9b6874c6fb8b07b0da9f00ea5: Bootstrap starting.
I20260923 22:12:28.058944 3796 tablet_bootstrap.cc:654] T 3d2bade3ede8413fbc76a068a1844cc7 P ece2a20b74874eb7acf32b82a1691f97: Neither blocks nor log segments found. Creating new log.
I20260923 22:12:28.059176 3843 ts_tablet_manager.cc:1905] T 9fd0c8c6b7b541798f3da5d6dab48ddd P ece2a20b74874eb7acf32b82a1691f97: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260923 22:12:28.059207 3797 tablet_bootstrap.cc:654] T 3d2bade3ede8413fbc76a068a1844cc7 P d269cfa9b6874c6fb8b07b0da9f00ea5: Neither blocks nor log segments found. Creating new log.
I20260923 22:12:28.059643 3430 tablet_service.cc:1467] Processing CreateTablet for tablet 3d2bade3ede8413fbc76a068a1844cc7 (DEFAULT_TABLE table=test-table [id=f44f81eaa8d64f9a8def07c3a588a570]), partition=RANGE (key) PARTITION UNBOUNDED
I20260923 22:12:28.059770 3430 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 3d2bade3ede8413fbc76a068a1844cc7. 1 dirs total, 0 dirs full, 0 dirs failed
I20260923 22:12:28.061102 3843 ts_tablet_manager.cc:1918] T 9fd0c8c6b7b541798f3da5d6dab48ddd P ece2a20b74874eb7acf32b82a1691f97: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260923 22:12:28.061194 3800 tablet_bootstrap.cc:492] T 3d2bade3ede8413fbc76a068a1844cc7 P 04978c60cca64c6783ac03221353dbe6: Bootstrap starting.
I20260923 22:12:28.061211 3843 log.cc:1196] T 9fd0c8c6b7b541798f3da5d6dab48ddd P ece2a20b74874eb7acf32b82a1691f97: Deleting WAL directory at /tmp/dist-test-taskRiQe_a/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790201539513534-1567-0/minicluster-data/ts-2/wal/wals/9fd0c8c6b7b541798f3da5d6dab48ddd
I20260923 22:12:28.061501 3843 ts_tablet_manager.cc:1939] T 9fd0c8c6b7b541798f3da5d6dab48ddd P ece2a20b74874eb7acf32b82a1691f97: Deleting consensus metadata
I20260923 22:12:28.061599 3842 ts_tablet_manager.cc:1918] T 9fd0c8c6b7b541798f3da5d6dab48ddd P d269cfa9b6874c6fb8b07b0da9f00ea5: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260923 22:12:28.061652 3842 log.cc:1196] T 9fd0c8c6b7b541798f3da5d6dab48ddd P d269cfa9b6874c6fb8b07b0da9f00ea5: Deleting WAL directory at /tmp/dist-test-taskRiQe_a/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790201539513534-1567-0/minicluster-data/ts-1/wal/wals/9fd0c8c6b7b541798f3da5d6dab48ddd
I20260923 22:12:28.061841 3800 tablet_bootstrap.cc:654] T 3d2bade3ede8413fbc76a068a1844cc7 P 04978c60cca64c6783ac03221353dbe6: Neither blocks nor log segments found. Creating new log.
I20260923 22:12:28.061888 3842 ts_tablet_manager.cc:1939] T 9fd0c8c6b7b541798f3da5d6dab48ddd P d269cfa9b6874c6fb8b07b0da9f00ea5: Deleting consensus metadata
I20260923 22:12:28.061893 3277 catalog_manager.cc:4841] TS ece2a20b74874eb7acf32b82a1691f97 (127.1.135.195:35699): tablet 9fd0c8c6b7b541798f3da5d6dab48ddd (table test-table [id=e9898a4ea2424a458f64a73481e76db7]) successfully deleted
I20260923 22:12:28.062027 3841 tablet_replica.cc:330] T 9fd0c8c6b7b541798f3da5d6dab48ddd P 04978c60cca64c6783ac03221353dbe6: stopping tablet replica
I20260923 22:12:28.062093 3841 raft_consensus.cc:2229] T 9fd0c8c6b7b541798f3da5d6dab48ddd P 04978c60cca64c6783ac03221353dbe6 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260923 22:12:28.062137 3841 raft_consensus.cc:2258] T 9fd0c8c6b7b541798f3da5d6dab48ddd P 04978c60cca64c6783ac03221353dbe6 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260923 22:12:28.062515 3841 ts_tablet_manager.cc:1905] T 9fd0c8c6b7b541798f3da5d6dab48ddd P 04978c60cca64c6783ac03221353dbe6: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260923 22:12:28.062567 3797 tablet_bootstrap.cc:492] T 3d2bade3ede8413fbc76a068a1844cc7 P d269cfa9b6874c6fb8b07b0da9f00ea5: No bootstrap required, opened a new log
I20260923 22:12:28.062619 3797 ts_tablet_manager.cc:1397] T 3d2bade3ede8413fbc76a068a1844cc7 P d269cfa9b6874c6fb8b07b0da9f00ea5: Time spent bootstrapping tablet: real 0.004s user 0.000s sys 0.001s
I20260923 22:12:28.062690 3796 tablet_bootstrap.cc:492] T 3d2bade3ede8413fbc76a068a1844cc7 P ece2a20b74874eb7acf32b82a1691f97: No bootstrap required, opened a new log
I20260923 22:12:28.062723 3796 ts_tablet_manager.cc:1397] T 3d2bade3ede8413fbc76a068a1844cc7 P ece2a20b74874eb7acf32b82a1691f97: Time spent bootstrapping tablet: real 0.004s user 0.001s sys 0.000s
I20260923 22:12:28.062776 3797 raft_consensus.cc:348] T 3d2bade3ede8413fbc76a068a1844cc7 P d269cfa9b6874c6fb8b07b0da9f00ea5 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "04978c60cca64c6783ac03221353dbe6" member_type: VOTER last_known_addr { host: "127.1.135.193" port: 42577 } } peers { permanent_uuid: "d269cfa9b6874c6fb8b07b0da9f00ea5" member_type: VOTER last_known_addr { host: "127.1.135.194" port: 36299 } } peers { permanent_uuid: "ece2a20b74874eb7acf32b82a1691f97" member_type: VOTER last_known_addr { host: "127.1.135.195" port: 35699 } }
I20260923 22:12:28.062840 3797 raft_consensus.cc:374] T 3d2bade3ede8413fbc76a068a1844cc7 P d269cfa9b6874c6fb8b07b0da9f00ea5 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260923 22:12:28.062865 3797 raft_consensus.cc:729] T 3d2bade3ede8413fbc76a068a1844cc7 P d269cfa9b6874c6fb8b07b0da9f00ea5 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d269cfa9b6874c6fb8b07b0da9f00ea5, State: Initialized, Role: FOLLOWER
I20260923 22:12:28.062847 3796 raft_consensus.cc:348] T 3d2bade3ede8413fbc76a068a1844cc7 P ece2a20b74874eb7acf32b82a1691f97 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "04978c60cca64c6783ac03221353dbe6" member_type: VOTER last_known_addr { host: "127.1.135.193" port: 42577 } } peers { permanent_uuid: "d269cfa9b6874c6fb8b07b0da9f00ea5" member_type: VOTER last_known_addr { host: "127.1.135.194" port: 36299 } } peers { permanent_uuid: "ece2a20b74874eb7acf32b82a1691f97" member_type: VOTER last_known_addr { host: "127.1.135.195" port: 35699 } }
I20260923 22:12:28.062901 3796 raft_consensus.cc:374] T 3d2bade3ede8413fbc76a068a1844cc7 P ece2a20b74874eb7acf32b82a1691f97 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260923 22:12:28.062922 3796 raft_consensus.cc:729] T 3d2bade3ede8413fbc76a068a1844cc7 P ece2a20b74874eb7acf32b82a1691f97 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: ece2a20b74874eb7acf32b82a1691f97, State: Initialized, Role: FOLLOWER
I20260923 22:12:28.062916 3797 consensus_queue.cc:260] T 3d2bade3ede8413fbc76a068a1844cc7 P d269cfa9b6874c6fb8b07b0da9f00ea5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "04978c60cca64c6783ac03221353dbe6" member_type: VOTER last_known_addr { host: "127.1.135.193" port: 42577 } } peers { permanent_uuid: "d269cfa9b6874c6fb8b07b0da9f00ea5" member_type: VOTER last_known_addr { host: "127.1.135.194" port: 36299 } } peers { permanent_uuid: "ece2a20b74874eb7acf32b82a1691f97" member_type: VOTER last_known_addr { host: "127.1.135.195" port: 35699 } }
I20260923 22:12:28.062956 3796 consensus_queue.cc:260] T 3d2bade3ede8413fbc76a068a1844cc7 P ece2a20b74874eb7acf32b82a1691f97 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "04978c60cca64c6783ac03221353dbe6" member_type: VOTER last_known_addr { host: "127.1.135.193" port: 42577 } } peers { permanent_uuid: "d269cfa9b6874c6fb8b07b0da9f00ea5" member_type: VOTER last_known_addr { host: "127.1.135.194" port: 36299 } } peers { permanent_uuid: "ece2a20b74874eb7acf32b82a1691f97" member_type: VOTER last_known_addr { host: "127.1.135.195" port: 35699 } }
I20260923 22:12:28.063001 3797 ts_tablet_manager.cc:1428] T 3d2bade3ede8413fbc76a068a1844cc7 P d269cfa9b6874c6fb8b07b0da9f00ea5: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20260923 22:12:28.063032 3796 ts_tablet_manager.cc:1428] T 3d2bade3ede8413fbc76a068a1844cc7 P ece2a20b74874eb7acf32b82a1691f97: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20260923 22:12:28.063314 3277 catalog_manager.cc:4841] TS d269cfa9b6874c6fb8b07b0da9f00ea5 (127.1.135.194:36299): tablet 9fd0c8c6b7b541798f3da5d6dab48ddd (table test-table [id=e9898a4ea2424a458f64a73481e76db7]) successfully deleted
I20260923 22:12:28.064405 3800 tablet_bootstrap.cc:492] T 3d2bade3ede8413fbc76a068a1844cc7 P 04978c60cca64c6783ac03221353dbe6: No bootstrap required, opened a new log
I20260923 22:12:28.064466 3800 ts_tablet_manager.cc:1397] T 3d2bade3ede8413fbc76a068a1844cc7 P 04978c60cca64c6783ac03221353dbe6: Time spent bootstrapping tablet: real 0.003s user 0.000s sys 0.001s
I20260923 22:12:28.064563 3841 ts_tablet_manager.cc:1918] T 9fd0c8c6b7b541798f3da5d6dab48ddd P 04978c60cca64c6783ac03221353dbe6: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260923 22:12:28.064625 3841 log.cc:1196] T 9fd0c8c6b7b541798f3da5d6dab48ddd P 04978c60cca64c6783ac03221353dbe6: Deleting WAL directory at /tmp/dist-test-taskRiQe_a/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790201539513534-1567-0/minicluster-data/ts-0/wal/wals/9fd0c8c6b7b541798f3da5d6dab48ddd
I20260923 22:12:28.064599 3800 raft_consensus.cc:348] T 3d2bade3ede8413fbc76a068a1844cc7 P 04978c60cca64c6783ac03221353dbe6 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "04978c60cca64c6783ac03221353dbe6" member_type: VOTER last_known_addr { host: "127.1.135.193" port: 42577 } } peers { permanent_uuid: "d269cfa9b6874c6fb8b07b0da9f00ea5" member_type: VOTER last_known_addr { host: "127.1.135.194" port: 36299 } } peers { permanent_uuid: "ece2a20b74874eb7acf32b82a1691f97" member_type: VOTER last_known_addr { host: "127.1.135.195" port: 35699 } }
I20260923 22:12:28.064687 3800 raft_consensus.cc:374] T 3d2bade3ede8413fbc76a068a1844cc7 P 04978c60cca64c6783ac03221353dbe6 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260923 22:12:28.064723 3800 raft_consensus.cc:729] T 3d2bade3ede8413fbc76a068a1844cc7 P 04978c60cca64c6783ac03221353dbe6 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 04978c60cca64c6783ac03221353dbe6, State: Initialized, Role: FOLLOWER
I20260923 22:12:28.064855 3800 consensus_queue.cc:260] T 3d2bade3ede8413fbc76a068a1844cc7 P 04978c60cca64c6783ac03221353dbe6 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "04978c60cca64c6783ac03221353dbe6" member_type: VOTER last_known_addr { host: "127.1.135.193" port: 42577 } } peers { permanent_uuid: "d269cfa9b6874c6fb8b07b0da9f00ea5" member_type: VOTER last_known_addr { host: "127.1.135.194" port: 36299 } } peers { permanent_uuid: "ece2a20b74874eb7acf32b82a1691f97" member_type: VOTER last_known_addr { host: "127.1.135.195" port: 35699 } }
I20260923 22:12:28.064960 3800 ts_tablet_manager.cc:1428] T 3d2bade3ede8413fbc76a068a1844cc7 P 04978c60cca64c6783ac03221353dbe6: Time spent starting tablet: real 0.000s user 0.000s sys 0.001s
I20260923 22:12:28.064975 3841 ts_tablet_manager.cc:1939] T 9fd0c8c6b7b541798f3da5d6dab48ddd P 04978c60cca64c6783ac03221353dbe6: Deleting consensus metadata
I20260923 22:12:28.065341 3278 catalog_manager.cc:4841] TS 04978c60cca64c6783ac03221353dbe6 (127.1.135.193:42577): tablet 9fd0c8c6b7b541798f3da5d6dab48ddd (table test-table [id=e9898a4ea2424a458f64a73481e76db7]) successfully deleted
I20260923 22:12:28.154335 3828 raft_consensus.cc:482] T 3d2bade3ede8413fbc76a068a1844cc7 P 04978c60cca64c6783ac03221353dbe6 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260923 22:12:28.154421 3828 raft_consensus.cc:504] T 3d2bade3ede8413fbc76a068a1844cc7 P 04978c60cca64c6783ac03221353dbe6 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "04978c60cca64c6783ac03221353dbe6" member_type: VOTER last_known_addr { host: "127.1.135.193" port: 42577 } } peers { permanent_uuid: "d269cfa9b6874c6fb8b07b0da9f00ea5" member_type: VOTER last_known_addr { host: "127.1.135.194" port: 36299 } } peers { permanent_uuid: "ece2a20b74874eb7acf32b82a1691f97" member_type: VOTER last_known_addr { host: "127.1.135.195" port: 35699 } }
I20260923 22:12:28.154807 3828 leader_election.cc:288] T 3d2bade3ede8413fbc76a068a1844cc7 P 04978c60cca64c6783ac03221353dbe6 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers d269cfa9b6874c6fb8b07b0da9f00ea5 (127.1.135.194:36299), ece2a20b74874eb7acf32b82a1691f97 (127.1.135.195:35699)
I20260923 22:12:28.158161 3821 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 22:12:28.155326 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.135.193:39973 (local address 127.1.135.195:35699)
0923 22:12:28.155372 (+ 46us) server_negotiation.cc:206] Beginning negotiation
0923 22:12:28.155375 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0923 22:12:28.155418 (+ 43us) server_negotiation.cc:407] Connection header received
0923 22:12:28.155533 (+ 115us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 22:12:28.155534 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 22:12:28.155543 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 22:12:28.155625 (+ 82us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0923 22:12:28.155827 (+ 202us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 22:12:28.156350 (+ 523us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 22:12:28.157431 (+ 1081us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 22:12:28.157830 (+ 399us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 22:12:28.157898 (+ 68us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 22:12:28.157906 (+ 8us) server_negotiation.cc:1035] Waiting for connection context
0923 22:12:28.158062 (+ 156us) server_negotiation.cc:299] Negotiation successful
0923 22:12:28.158097 (+ 35us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":24}
I20260923 22:12:28.158178 3848 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 22:12:28.155191 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.135.195:35699 (local address 127.1.135.193:39973)
0923 22:12:28.155353 (+ 162us) negotiation.cc:107] Waiting for socket to connect
0923 22:12:28.155364 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0923 22:12:28.155422 (+ 58us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0923 22:12:28.155631 (+ 209us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0923 22:12:28.155632 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0923 22:12:28.155646 (+ 14us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0923 22:12:28.155718 (+ 72us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 22:12:28.155720 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 22:12:28.156451 (+ 731us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 22:12:28.156452 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 22:12:28.157310 (+ 858us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 22:12:28.157312 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 22:12:28.157914 (+ 602us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 22:12:28.157915 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 22:12:28.157989 (+ 74us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 22:12:28.157994 (+ 5us) client_negotiation.cc:769] Sending connection context
0923 22:12:28.158049 (+ 55us) client_negotiation.cc:240] Negotiation successful
0923 22:12:28.158085 (+ 36us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":147,"mutex_wait_us":59,"thread_start_us":75,"threads_started":1}
I20260923 22:12:28.158545 3847 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 22:12:28.154968 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.135.194:36299 (local address 127.1.135.193:49217)
0923 22:12:28.155126 (+ 158us) negotiation.cc:107] Waiting for socket to connect
0923 22:12:28.155138 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0923 22:12:28.155198 (+ 60us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0923 22:12:28.155417 (+ 219us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0923 22:12:28.155418 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0923 22:12:28.155421 (+ 3us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0923 22:12:28.155497 (+ 76us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 22:12:28.155499 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 22:12:28.156479 (+ 980us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 22:12:28.156480 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 22:12:28.157670 (+ 1190us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 22:12:28.157673 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 22:12:28.158324 (+ 651us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 22:12:28.158325 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 22:12:28.158390 (+ 65us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 22:12:28.158393 (+ 3us) client_negotiation.cc:769] Sending connection context
0923 22:12:28.158447 (+ 54us) client_negotiation.cc:240] Negotiation successful
0923 22:12:28.158479 (+ 32us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":139,"thread_start_us":87,"threads_started":1}
I20260923 22:12:28.158550 3795 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 22:12:28.155035 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.135.193:49217 (local address 127.1.135.194:36299)
0923 22:12:28.155082 (+ 47us) server_negotiation.cc:206] Beginning negotiation
0923 22:12:28.155084 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0923 22:12:28.155198 (+ 114us) server_negotiation.cc:407] Connection header received
0923 22:12:28.155256 (+ 58us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 22:12:28.155256 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 22:12:28.155267 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 22:12:28.155345 (+ 78us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0923 22:12:28.155576 (+ 231us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 22:12:28.156385 (+ 809us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 22:12:28.157759 (+ 1374us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 22:12:28.158254 (+ 495us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 22:12:28.158331 (+ 77us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 22:12:28.158341 (+ 10us) server_negotiation.cc:1035] Waiting for connection context
0923 22:12:28.158462 (+ 121us) server_negotiation.cc:299] Negotiation successful
0923 22:12:28.158494 (+ 32us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":22}
I20260923 22:12:28.158771 3730 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "3d2bade3ede8413fbc76a068a1844cc7" candidate_uuid: "04978c60cca64c6783ac03221353dbe6" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ece2a20b74874eb7acf32b82a1691f97" is_pre_election: true
I20260923 22:12:28.158918 3730 raft_consensus.cc:2454] T 3d2bade3ede8413fbc76a068a1844cc7 P ece2a20b74874eb7acf32b82a1691f97 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 04978c60cca64c6783ac03221353dbe6 in term 0.
I20260923 22:12:28.159056 3589 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "3d2bade3ede8413fbc76a068a1844cc7" candidate_uuid: "04978c60cca64c6783ac03221353dbe6" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d269cfa9b6874c6fb8b07b0da9f00ea5" is_pre_election: true
I20260923 22:12:28.159137 3589 raft_consensus.cc:2454] T 3d2bade3ede8413fbc76a068a1844cc7 P d269cfa9b6874c6fb8b07b0da9f00ea5 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 04978c60cca64c6783ac03221353dbe6 in term 0.
I20260923 22:12:28.159301 3381 leader_election.cc:302] T 3d2bade3ede8413fbc76a068a1844cc7 P 04978c60cca64c6783ac03221353dbe6 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 04978c60cca64c6783ac03221353dbe6, ece2a20b74874eb7acf32b82a1691f97; no voters:
I20260923 22:12:28.159456 3828 raft_consensus.cc:2781] T 3d2bade3ede8413fbc76a068a1844cc7 P 04978c60cca64c6783ac03221353dbe6 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260923 22:12:28.159495 3828 raft_consensus.cc:482] T 3d2bade3ede8413fbc76a068a1844cc7 P 04978c60cca64c6783ac03221353dbe6 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260923 22:12:28.159533 3828 raft_consensus.cc:3037] T 3d2bade3ede8413fbc76a068a1844cc7 P 04978c60cca64c6783ac03221353dbe6 [term 0 FOLLOWER]: Advancing to term 1
I20260923 22:12:28.160384 3828 raft_consensus.cc:504] T 3d2bade3ede8413fbc76a068a1844cc7 P 04978c60cca64c6783ac03221353dbe6 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "04978c60cca64c6783ac03221353dbe6" member_type: VOTER last_known_addr { host: "127.1.135.193" port: 42577 } } peers { permanent_uuid: "d269cfa9b6874c6fb8b07b0da9f00ea5" member_type: VOTER last_known_addr { host: "127.1.135.194" port: 36299 } } peers { permanent_uuid: "ece2a20b74874eb7acf32b82a1691f97" member_type: VOTER last_known_addr { host: "127.1.135.195" port: 35699 } }
I20260923 22:12:28.160506 3828 leader_election.cc:288] T 3d2bade3ede8413fbc76a068a1844cc7 P 04978c60cca64c6783ac03221353dbe6 [CANDIDATE]: Term 1 election: Requested vote from peers d269cfa9b6874c6fb8b07b0da9f00ea5 (127.1.135.194:36299), ece2a20b74874eb7acf32b82a1691f97 (127.1.135.195:35699)
I20260923 22:12:28.160674 3589 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "3d2bade3ede8413fbc76a068a1844cc7" candidate_uuid: "04978c60cca64c6783ac03221353dbe6" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d269cfa9b6874c6fb8b07b0da9f00ea5"
I20260923 22:12:28.160701 3730 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "3d2bade3ede8413fbc76a068a1844cc7" candidate_uuid: "04978c60cca64c6783ac03221353dbe6" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ece2a20b74874eb7acf32b82a1691f97"
I20260923 22:12:28.160771 3730 raft_consensus.cc:3037] T 3d2bade3ede8413fbc76a068a1844cc7 P ece2a20b74874eb7acf32b82a1691f97 [term 0 FOLLOWER]: Advancing to term 1
I20260923 22:12:28.160770 3589 raft_consensus.cc:3037] T 3d2bade3ede8413fbc76a068a1844cc7 P d269cfa9b6874c6fb8b07b0da9f00ea5 [term 0 FOLLOWER]: Advancing to term 1
I20260923 22:12:28.161572 3730 raft_consensus.cc:2454] T 3d2bade3ede8413fbc76a068a1844cc7 P ece2a20b74874eb7acf32b82a1691f97 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 04978c60cca64c6783ac03221353dbe6 in term 1.
I20260923 22:12:28.161702 3589 raft_consensus.cc:2454] T 3d2bade3ede8413fbc76a068a1844cc7 P d269cfa9b6874c6fb8b07b0da9f00ea5 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 04978c60cca64c6783ac03221353dbe6 in term 1.
I20260923 22:12:28.161763 3381 leader_election.cc:302] T 3d2bade3ede8413fbc76a068a1844cc7 P 04978c60cca64c6783ac03221353dbe6 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 04978c60cca64c6783ac03221353dbe6, ece2a20b74874eb7acf32b82a1691f97; no voters:
I20260923 22:12:28.164170 3828 raft_consensus.cc:2781] T 3d2bade3ede8413fbc76a068a1844cc7 P 04978c60cca64c6783ac03221353dbe6 [term 1 FOLLOWER]: Leader election won for term 1
I20260923 22:12:28.164259 3828 raft_consensus.cc:686] T 3d2bade3ede8413fbc76a068a1844cc7 P 04978c60cca64c6783ac03221353dbe6 [term 1 LEADER]: Becoming Leader. State: Replica: 04978c60cca64c6783ac03221353dbe6, State: Running, Role: LEADER
I20260923 22:12:28.164330 3828 consensus_queue.cc:237] T 3d2bade3ede8413fbc76a068a1844cc7 P 04978c60cca64c6783ac03221353dbe6 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "04978c60cca64c6783ac03221353dbe6" member_type: VOTER last_known_addr { host: "127.1.135.193" port: 42577 } } peers { permanent_uuid: "d269cfa9b6874c6fb8b07b0da9f00ea5" member_type: VOTER last_known_addr { host: "127.1.135.194" port: 36299 } } peers { permanent_uuid: "ece2a20b74874eb7acf32b82a1691f97" member_type: VOTER last_known_addr { host: "127.1.135.195" port: 35699 } }
I20260923 22:12:28.165002 3292 catalog_manager.cc:5495] T 3d2bade3ede8413fbc76a068a1844cc7 P 04978c60cca64c6783ac03221353dbe6 reported cstate change: term changed from 0 to 1, leader changed from <none> to 04978c60cca64c6783ac03221353dbe6 (127.1.135.193). New cstate: current_term: 1 leader_uuid: "04978c60cca64c6783ac03221353dbe6" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "04978c60cca64c6783ac03221353dbe6" member_type: VOTER last_known_addr { host: "127.1.135.193" port: 42577 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "d269cfa9b6874c6fb8b07b0da9f00ea5" member_type: VOTER last_known_addr { host: "127.1.135.194" port: 36299 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "ece2a20b74874eb7acf32b82a1691f97" member_type: VOTER last_known_addr { host: "127.1.135.195" port: 35699 } health_report { overall_health: UNKNOWN } } }
I20260923 22:12:28.178153 3792 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 22:12:28.175872 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37928 (local address 127.1.135.193:42577)
0923 22:12:28.175922 (+ 50us) server_negotiation.cc:206] Beginning negotiation
0923 22:12:28.175924 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0923 22:12:28.175932 (+ 8us) server_negotiation.cc:407] Connection header received
0923 22:12:28.175986 (+ 54us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 22:12:28.175987 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 22:12:28.176004 (+ 17us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 22:12:28.176082 (+ 78us) server_negotiation.cc:226] Negotiated authn=TOKEN
0923 22:12:28.176244 (+ 162us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 22:12:28.176668 (+ 424us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 22:12:28.177375 (+ 707us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 22:12:28.177577 (+ 202us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 22:12:28.177650 (+ 73us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 22:12:28.177795 (+ 145us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0923 22:12:28.177903 (+ 108us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0923 22:12:28.177980 (+ 77us) server_negotiation.cc:1035] Waiting for connection context
0923 22:12:28.178062 (+ 82us) server_negotiation.cc:299] Negotiation successful
0923 22:12:28.178086 (+ 24us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":21}
I20260923 22:12:28.183228 3821 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 22:12:28.180830 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.135.193:49337 (local address 127.1.135.195:35699)
0923 22:12:28.180866 (+ 36us) server_negotiation.cc:206] Beginning negotiation
0923 22:12:28.180868 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0923 22:12:28.180937 (+ 69us) server_negotiation.cc:407] Connection header received
0923 22:12:28.180992 (+ 55us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 22:12:28.180993 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 22:12:28.181002 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 22:12:28.181046 (+ 44us) server_negotiation.cc:226] Negotiated authn=TOKEN
0923 22:12:28.181150 (+ 104us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 22:12:28.181561 (+ 411us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 22:12:28.182585 (+ 1024us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 22:12:28.182773 (+ 188us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 22:12:28.182824 (+ 51us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 22:12:28.183028 (+ 204us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0923 22:12:28.183051 (+ 23us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0923 22:12:28.183089 (+ 38us) server_negotiation.cc:1035] Waiting for connection context
0923 22:12:28.183163 (+ 74us) server_negotiation.cc:299] Negotiation successful
0923 22:12:28.183180 (+ 17us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":17}
I20260923 22:12:28.183292 3852 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 22:12:28.180723 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.135.195:35699 (local address 127.1.135.193:49337)
0923 22:12:28.180871 (+ 148us) negotiation.cc:107] Waiting for socket to connect
0923 22:12:28.180880 (+ 9us) client_negotiation.cc:174] Beginning negotiation
0923 22:12:28.180938 (+ 58us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0923 22:12:28.181052 (+ 114us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0923 22:12:28.181052 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0923 22:12:28.181054 (+ 2us) client_negotiation.cc:189] Negotiated authn=TOKEN
0923 22:12:28.181092 (+ 38us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 22:12:28.181093 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 22:12:28.181648 (+ 555us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 22:12:28.181649 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 22:12:28.182498 (+ 849us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 22:12:28.182501 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 22:12:28.182873 (+ 372us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 22:12:28.182875 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 22:12:28.182930 (+ 55us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 22:12:28.182944 (+ 14us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0923 22:12:28.183109 (+ 165us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0923 22:12:28.183111 (+ 2us) client_negotiation.cc:769] Sending connection context
0923 22:12:28.183162 (+ 51us) client_negotiation.cc:240] Negotiation successful
0923 22:12:28.183199 (+ 37us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":117,"thread_start_us":69,"threads_started":1}
I20260923 22:12:28.185999 3589 raft_consensus.cc:1261] T 3d2bade3ede8413fbc76a068a1844cc7 P d269cfa9b6874c6fb8b07b0da9f00ea5 [term 1 FOLLOWER]: Refusing update from remote peer 04978c60cca64c6783ac03221353dbe6: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260923 22:12:28.186281 3730 raft_consensus.cc:1261] T 3d2bade3ede8413fbc76a068a1844cc7 P ece2a20b74874eb7acf32b82a1691f97 [term 1 FOLLOWER]: Refusing update from remote peer 04978c60cca64c6783ac03221353dbe6: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260923 22:12:28.186264 3828 consensus_queue.cc:1035] T 3d2bade3ede8413fbc76a068a1844cc7 P 04978c60cca64c6783ac03221353dbe6 [LEADER]: Connected to new peer: Peer: permanent_uuid: "d269cfa9b6874c6fb8b07b0da9f00ea5" member_type: VOTER last_known_addr { host: "127.1.135.194" port: 36299 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260923 22:12:28.186489 3828 consensus_queue.cc:1035] T 3d2bade3ede8413fbc76a068a1844cc7 P 04978c60cca64c6783ac03221353dbe6 [LEADER]: Connected to new peer: Peer: permanent_uuid: "ece2a20b74874eb7acf32b82a1691f97" member_type: VOTER last_known_addr { host: "127.1.135.195" port: 35699 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260923 22:12:28.187888 3823 mvcc.cc:205] Tried to move back new op lower bound from 7332665541368422400 to 7332665541281636352. Current Snapshot: MvccSnapshot[applied={T|T < 7332665541368422400}]
I20260923 22:12:28.194375 3792 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 22:12:28.191780 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.135.195:42577 (local address 127.1.135.193:42577)
0923 22:12:28.191813 (+ 33us) server_negotiation.cc:206] Beginning negotiation
0923 22:12:28.191815 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0923 22:12:28.191823 (+ 8us) server_negotiation.cc:407] Connection header received
0923 22:12:28.191879 (+ 56us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 22:12:28.191880 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 22:12:28.191888 (+ 8us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 22:12:28.191941 (+ 53us) server_negotiation.cc:226] Negotiated authn=TOKEN
0923 22:12:28.192049 (+ 108us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 22:12:28.192757 (+ 708us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 22:12:28.193637 (+ 880us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 22:12:28.193939 (+ 302us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 22:12:28.194008 (+ 69us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 22:12:28.194157 (+ 149us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0923 22:12:28.194187 (+ 30us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0923 22:12:28.194237 (+ 50us) server_negotiation.cc:1035] Waiting for connection context
0923 22:12:28.194297 (+ 60us) server_negotiation.cc:299] Negotiation successful
0923 22:12:28.194315 (+ 18us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":18}
I20260923 22:12:28.194391 3836 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 22:12:28.191718 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.135.193:42577 (local address 127.1.135.195:42577)
0923 22:12:28.191780 (+ 62us) negotiation.cc:107] Waiting for socket to connect
0923 22:12:28.191791 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0923 22:12:28.191826 (+ 35us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0923 22:12:28.191955 (+ 129us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0923 22:12:28.191956 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0923 22:12:28.191957 (+ 1us) client_negotiation.cc:189] Negotiated authn=TOKEN
0923 22:12:28.191993 (+ 36us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 22:12:28.191995 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 22:12:28.192846 (+ 851us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 22:12:28.192846 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 22:12:28.193512 (+ 666us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 22:12:28.193515 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 22:12:28.194007 (+ 492us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 22:12:28.194008 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 22:12:28.194072 (+ 64us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 22:12:28.194077 (+ 5us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0923 22:12:28.194248 (+ 171us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0923 22:12:28.194249 (+ 1us) client_negotiation.cc:769] Sending connection context
0923 22:12:28.194286 (+ 37us) client_negotiation.cc:240] Negotiation successful
0923 22:12:28.194327 (+ 41us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":43}
W20260923 22:12:28.194669 3823 tablet_replica.cc:1298] Aborted: operation has been aborted: cancelling pending write operations
I20260923 22:12:29.713821 3292 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:35744:
table { table_name: "test-table" } modify_external_catalogs: true
I20260923 22:12:29.713924 3292 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:35744:
table { table_name: "test-table" } modify_external_catalogs: true
I20260923 22:12:29.714681 3292 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 24a871a68671419f89d1a0d5a2bdcd3f: Sending DeleteTablet for 3 replicas of tablet 3d2bade3ede8413fbc76a068a1844cc7
I20260923 22:12:29.714916 3430 tablet_service.cc:1514] Processing DeleteTablet for tablet 3d2bade3ede8413fbc76a068a1844cc7 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-23 22:12:29 UTC) from {username='kudu', principal='kudu/127.1.135.254@KRBTEST.COM'} at 127.0.0.1:37922
I20260923 22:12:29.715047 3710 tablet_service.cc:1514] Processing DeleteTablet for tablet 3d2bade3ede8413fbc76a068a1844cc7 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-23 22:12:29 UTC) from {username='kudu', principal='kudu/127.1.135.254@KRBTEST.COM'} at 127.0.0.1:41660
I20260923 22:12:29.715109 3570 tablet_service.cc:1514] Processing DeleteTablet for tablet 3d2bade3ede8413fbc76a068a1844cc7 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-23 22:12:29 UTC) from {username='kudu', principal='kudu/127.1.135.254@KRBTEST.COM'} at 127.0.0.1:48168
I20260923 22:12:29.715126 3877 tablet_replica.cc:330] T 3d2bade3ede8413fbc76a068a1844cc7 P 04978c60cca64c6783ac03221353dbe6: stopping tablet replica
I20260923 22:12:29.715412 3877 raft_consensus.cc:2229] T 3d2bade3ede8413fbc76a068a1844cc7 P 04978c60cca64c6783ac03221353dbe6 [term 1 LEADER]: Raft consensus shutting down.
I20260923 22:12:29.715497 3879 tablet_replica.cc:330] T 3d2bade3ede8413fbc76a068a1844cc7 P d269cfa9b6874c6fb8b07b0da9f00ea5: stopping tablet replica
I20260923 22:12:29.715512 3877 raft_consensus.cc:2258] T 3d2bade3ede8413fbc76a068a1844cc7 P 04978c60cca64c6783ac03221353dbe6 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260923 22:12:29.715564 3879 raft_consensus.cc:2229] T 3d2bade3ede8413fbc76a068a1844cc7 P d269cfa9b6874c6fb8b07b0da9f00ea5 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260923 22:12:29.715605 3879 raft_consensus.cc:2258] T 3d2bade3ede8413fbc76a068a1844cc7 P d269cfa9b6874c6fb8b07b0da9f00ea5 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260923 22:12:29.715759 3877 ts_tablet_manager.cc:1905] T 3d2bade3ede8413fbc76a068a1844cc7 P 04978c60cca64c6783ac03221353dbe6: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260923 22:12:29.715855 3879 ts_tablet_manager.cc:1905] T 3d2bade3ede8413fbc76a068a1844cc7 P d269cfa9b6874c6fb8b07b0da9f00ea5: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260923 22:12:29.716167 3878 tablet_replica.cc:330] T 3d2bade3ede8413fbc76a068a1844cc7 P ece2a20b74874eb7acf32b82a1691f97: stopping tablet replica
I20260923 22:12:29.716274 3878 raft_consensus.cc:2229] T 3d2bade3ede8413fbc76a068a1844cc7 P ece2a20b74874eb7acf32b82a1691f97 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260923 22:12:29.716356 3878 raft_consensus.cc:2258] T 3d2bade3ede8413fbc76a068a1844cc7 P ece2a20b74874eb7acf32b82a1691f97 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260923 22:12:29.716634 3878 ts_tablet_manager.cc:1905] T 3d2bade3ede8413fbc76a068a1844cc7 P ece2a20b74874eb7acf32b82a1691f97: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260923 22:12:29.717590 3879 ts_tablet_manager.cc:1918] T 3d2bade3ede8413fbc76a068a1844cc7 P d269cfa9b6874c6fb8b07b0da9f00ea5: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.8
I20260923 22:12:29.717605 3877 ts_tablet_manager.cc:1918] T 3d2bade3ede8413fbc76a068a1844cc7 P 04978c60cca64c6783ac03221353dbe6: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.8
I20260923 22:12:29.717658 3877 log.cc:1196] T 3d2bade3ede8413fbc76a068a1844cc7 P 04978c60cca64c6783ac03221353dbe6: Deleting WAL directory at /tmp/dist-test-taskRiQe_a/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790201539513534-1567-0/minicluster-data/ts-0/wal/wals/3d2bade3ede8413fbc76a068a1844cc7
I20260923 22:12:29.717721 3879 log.cc:1196] T 3d2bade3ede8413fbc76a068a1844cc7 P d269cfa9b6874c6fb8b07b0da9f00ea5: Deleting WAL directory at /tmp/dist-test-taskRiQe_a/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790201539513534-1567-0/minicluster-data/ts-1/wal/wals/3d2bade3ede8413fbc76a068a1844cc7
I20260923 22:12:29.717921 3877 ts_tablet_manager.cc:1939] T 3d2bade3ede8413fbc76a068a1844cc7 P 04978c60cca64c6783ac03221353dbe6: Deleting consensus metadata
I20260923 22:12:29.717963 3879 ts_tablet_manager.cc:1939] T 3d2bade3ede8413fbc76a068a1844cc7 P d269cfa9b6874c6fb8b07b0da9f00ea5: Deleting consensus metadata
I20260923 22:12:29.718479 3878 ts_tablet_manager.cc:1918] T 3d2bade3ede8413fbc76a068a1844cc7 P ece2a20b74874eb7acf32b82a1691f97: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.8
I20260923 22:12:29.718544 3278 catalog_manager.cc:4841] TS 04978c60cca64c6783ac03221353dbe6 (127.1.135.193:42577): tablet 3d2bade3ede8413fbc76a068a1844cc7 (table test-table [id=f44f81eaa8d64f9a8def07c3a588a570]) successfully deleted
I20260923 22:12:29.718590 3878 log.cc:1196] T 3d2bade3ede8413fbc76a068a1844cc7 P ece2a20b74874eb7acf32b82a1691f97: Deleting WAL directory at /tmp/dist-test-taskRiQe_a/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790201539513534-1567-0/minicluster-data/ts-2/wal/wals/3d2bade3ede8413fbc76a068a1844cc7
I20260923 22:12:29.718626 3277 catalog_manager.cc:4841] TS d269cfa9b6874c6fb8b07b0da9f00ea5 (127.1.135.194:36299): tablet 3d2bade3ede8413fbc76a068a1844cc7 (table test-table [id=f44f81eaa8d64f9a8def07c3a588a570]) successfully deleted
I20260923 22:12:29.718802 3878 ts_tablet_manager.cc:1939] T 3d2bade3ede8413fbc76a068a1844cc7 P ece2a20b74874eb7acf32b82a1691f97: Deleting consensus metadata
I20260923 22:12:29.719098 3277 catalog_manager.cc:4841] TS ece2a20b74874eb7acf32b82a1691f97 (127.1.135.195:35699): tablet 3d2bade3ede8413fbc76a068a1844cc7 (table test-table [id=f44f81eaa8d64f9a8def07c3a588a570]) successfully deleted
Sep 23 22:12:29 dist-test-slave-s2fj krb5kdc[3248](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790201549, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-user@KRBTEST.COM:
Sep 23 22:12:29 dist-test-slave-s2fj krb5kdc[3248](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790201549, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for kudu/127.1.135.254@KRBTEST.COM
I20260923 22:12:29.740499 3887 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 22:12:29.732846 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35770 (local address 127.1.135.254:37377)
0923 22:12:29.732996 (+ 150us) server_negotiation.cc:206] Beginning negotiation
0923 22:12:29.732998 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0923 22:12:29.733011 (+ 13us) server_negotiation.cc:407] Connection header received
0923 22:12:29.733114 (+ 103us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 22:12:29.733115 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 22:12:29.733127 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 22:12:29.733207 (+ 80us) server_negotiation.cc:226] Negotiated authn=SASL
0923 22:12:29.733779 (+ 572us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 22:12:29.734293 (+ 514us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 22:12:29.735058 (+ 765us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 22:12:29.735271 (+ 213us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 22:12:29.735356 (+ 85us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 22:12:29.737652 (+ 2296us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0923 22:12:29.737653 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0923 22:12:29.737653 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0923 22:12:29.737661 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0923 22:12:29.739293 (+ 1632us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0923 22:12:29.739645 (+ 352us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0923 22:12:29.739647 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0923 22:12:29.739647 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0923 22:12:29.739708 (+ 61us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0923 22:12:29.739909 (+ 201us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0923 22:12:29.739910 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0923 22:12:29.739910 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0923 22:12:29.740026 (+ 116us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0923 22:12:29.740198 (+ 172us) server_negotiation.cc:1035] Waiting for connection context
0923 22:12:29.740306 (+ 108us) server_negotiation.cc:299] Negotiation successful
0923 22:12:29.740386 (+ 80us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":120,"thread_start_us":77,"threads_started":1}
I20260923 22:12:29.742008 3292 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:35770:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260923 22:12:29.742188 3292 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260923 22:12:29.743659 3710 tablet_service.cc:1467] Processing CreateTablet for tablet c117a3888ff548ba98e428042f09149b (DEFAULT_TABLE table=test-table [id=3299e8a1615c467faba379667a4520cb]), partition=RANGE (key) PARTITION UNBOUNDED
I20260923 22:12:29.743659 3570 tablet_service.cc:1467] Processing CreateTablet for tablet c117a3888ff548ba98e428042f09149b (DEFAULT_TABLE table=test-table [id=3299e8a1615c467faba379667a4520cb]), partition=RANGE (key) PARTITION UNBOUNDED
I20260923 22:12:29.743708 3430 tablet_service.cc:1467] Processing CreateTablet for tablet c117a3888ff548ba98e428042f09149b (DEFAULT_TABLE table=test-table [id=3299e8a1615c467faba379667a4520cb]), partition=RANGE (key) PARTITION UNBOUNDED
I20260923 22:12:29.743793 3710 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet c117a3888ff548ba98e428042f09149b. 1 dirs total, 0 dirs full, 0 dirs failed
I20260923 22:12:29.743795 3570 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet c117a3888ff548ba98e428042f09149b. 1 dirs total, 0 dirs full, 0 dirs failed
I20260923 22:12:29.743856 3430 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet c117a3888ff548ba98e428042f09149b. 1 dirs total, 0 dirs full, 0 dirs failed
I20260923 22:12:29.745188 3888 tablet_bootstrap.cc:492] T c117a3888ff548ba98e428042f09149b P d269cfa9b6874c6fb8b07b0da9f00ea5: Bootstrap starting.
I20260923 22:12:29.745307 3890 tablet_bootstrap.cc:492] T c117a3888ff548ba98e428042f09149b P 04978c60cca64c6783ac03221353dbe6: Bootstrap starting.
I20260923 22:12:29.745538 3888 tablet_bootstrap.cc:654] T c117a3888ff548ba98e428042f09149b P d269cfa9b6874c6fb8b07b0da9f00ea5: Neither blocks nor log segments found. Creating new log.
I20260923 22:12:29.745657 3890 tablet_bootstrap.cc:654] T c117a3888ff548ba98e428042f09149b P 04978c60cca64c6783ac03221353dbe6: Neither blocks nor log segments found. Creating new log.
I20260923 22:12:29.745849 3889 tablet_bootstrap.cc:492] T c117a3888ff548ba98e428042f09149b P ece2a20b74874eb7acf32b82a1691f97: Bootstrap starting.
I20260923 22:12:29.746230 3890 tablet_bootstrap.cc:492] T c117a3888ff548ba98e428042f09149b P 04978c60cca64c6783ac03221353dbe6: No bootstrap required, opened a new log
I20260923 22:12:29.746248 3889 tablet_bootstrap.cc:654] T c117a3888ff548ba98e428042f09149b P ece2a20b74874eb7acf32b82a1691f97: Neither blocks nor log segments found. Creating new log.
I20260923 22:12:29.746299 3890 ts_tablet_manager.cc:1397] T c117a3888ff548ba98e428042f09149b P 04978c60cca64c6783ac03221353dbe6: Time spent bootstrapping tablet: real 0.001s user 0.001s sys 0.000s
I20260923 22:12:29.746420 3890 raft_consensus.cc:348] T c117a3888ff548ba98e428042f09149b P 04978c60cca64c6783ac03221353dbe6 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "04978c60cca64c6783ac03221353dbe6" member_type: VOTER last_known_addr { host: "127.1.135.193" port: 42577 } } peers { permanent_uuid: "d269cfa9b6874c6fb8b07b0da9f00ea5" member_type: VOTER last_known_addr { host: "127.1.135.194" port: 36299 } } peers { permanent_uuid: "ece2a20b74874eb7acf32b82a1691f97" member_type: VOTER last_known_addr { host: "127.1.135.195" port: 35699 } }
I20260923 22:12:29.746495 3890 raft_consensus.cc:374] T c117a3888ff548ba98e428042f09149b P 04978c60cca64c6783ac03221353dbe6 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260923 22:12:29.746542 3890 raft_consensus.cc:729] T c117a3888ff548ba98e428042f09149b P 04978c60cca64c6783ac03221353dbe6 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 04978c60cca64c6783ac03221353dbe6, State: Initialized, Role: FOLLOWER
I20260923 22:12:29.746613 3890 consensus_queue.cc:260] T c117a3888ff548ba98e428042f09149b P 04978c60cca64c6783ac03221353dbe6 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "04978c60cca64c6783ac03221353dbe6" member_type: VOTER last_known_addr { host: "127.1.135.193" port: 42577 } } peers { permanent_uuid: "d269cfa9b6874c6fb8b07b0da9f00ea5" member_type: VOTER last_known_addr { host: "127.1.135.194" port: 36299 } } peers { permanent_uuid: "ece2a20b74874eb7acf32b82a1691f97" member_type: VOTER last_known_addr { host: "127.1.135.195" port: 35699 } }
I20260923 22:12:29.746770 3889 tablet_bootstrap.cc:492] T c117a3888ff548ba98e428042f09149b P ece2a20b74874eb7acf32b82a1691f97: No bootstrap required, opened a new log
I20260923 22:12:29.746826 3889 ts_tablet_manager.cc:1397] T c117a3888ff548ba98e428042f09149b P ece2a20b74874eb7acf32b82a1691f97: Time spent bootstrapping tablet: real 0.001s user 0.001s sys 0.000s
I20260923 22:12:29.746894 3890 ts_tablet_manager.cc:1428] T c117a3888ff548ba98e428042f09149b P 04978c60cca64c6783ac03221353dbe6: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20260923 22:12:29.746950 3889 raft_consensus.cc:348] T c117a3888ff548ba98e428042f09149b P ece2a20b74874eb7acf32b82a1691f97 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "04978c60cca64c6783ac03221353dbe6" member_type: VOTER last_known_addr { host: "127.1.135.193" port: 42577 } } peers { permanent_uuid: "d269cfa9b6874c6fb8b07b0da9f00ea5" member_type: VOTER last_known_addr { host: "127.1.135.194" port: 36299 } } peers { permanent_uuid: "ece2a20b74874eb7acf32b82a1691f97" member_type: VOTER last_known_addr { host: "127.1.135.195" port: 35699 } }
I20260923 22:12:29.747032 3889 raft_consensus.cc:374] T c117a3888ff548ba98e428042f09149b P ece2a20b74874eb7acf32b82a1691f97 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260923 22:12:29.747165 3888 tablet_bootstrap.cc:492] T c117a3888ff548ba98e428042f09149b P d269cfa9b6874c6fb8b07b0da9f00ea5: No bootstrap required, opened a new log
I20260923 22:12:29.747223 3888 ts_tablet_manager.cc:1397] T c117a3888ff548ba98e428042f09149b P d269cfa9b6874c6fb8b07b0da9f00ea5: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20260923 22:12:29.747380 3888 raft_consensus.cc:348] T c117a3888ff548ba98e428042f09149b P d269cfa9b6874c6fb8b07b0da9f00ea5 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "04978c60cca64c6783ac03221353dbe6" member_type: VOTER last_known_addr { host: "127.1.135.193" port: 42577 } } peers { permanent_uuid: "d269cfa9b6874c6fb8b07b0da9f00ea5" member_type: VOTER last_known_addr { host: "127.1.135.194" port: 36299 } } peers { permanent_uuid: "ece2a20b74874eb7acf32b82a1691f97" member_type: VOTER last_known_addr { host: "127.1.135.195" port: 35699 } }
I20260923 22:12:29.747474 3888 raft_consensus.cc:374] T c117a3888ff548ba98e428042f09149b P d269cfa9b6874c6fb8b07b0da9f00ea5 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260923 22:12:29.747500 3888 raft_consensus.cc:729] T c117a3888ff548ba98e428042f09149b P d269cfa9b6874c6fb8b07b0da9f00ea5 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d269cfa9b6874c6fb8b07b0da9f00ea5, State: Initialized, Role: FOLLOWER
I20260923 22:12:29.747556 3888 consensus_queue.cc:260] T c117a3888ff548ba98e428042f09149b P d269cfa9b6874c6fb8b07b0da9f00ea5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "04978c60cca64c6783ac03221353dbe6" member_type: VOTER last_known_addr { host: "127.1.135.193" port: 42577 } } peers { permanent_uuid: "d269cfa9b6874c6fb8b07b0da9f00ea5" member_type: VOTER last_known_addr { host: "127.1.135.194" port: 36299 } } peers { permanent_uuid: "ece2a20b74874eb7acf32b82a1691f97" member_type: VOTER last_known_addr { host: "127.1.135.195" port: 35699 } }
I20260923 22:12:29.747720 3888 ts_tablet_manager.cc:1428] T c117a3888ff548ba98e428042f09149b P d269cfa9b6874c6fb8b07b0da9f00ea5: Time spent starting tablet: real 0.000s user 0.001s sys 0.000s
I20260923 22:12:29.747803 3889 raft_consensus.cc:729] T c117a3888ff548ba98e428042f09149b P ece2a20b74874eb7acf32b82a1691f97 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: ece2a20b74874eb7acf32b82a1691f97, State: Initialized, Role: FOLLOWER
I20260923 22:12:29.747884 3889 consensus_queue.cc:260] T c117a3888ff548ba98e428042f09149b P ece2a20b74874eb7acf32b82a1691f97 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "04978c60cca64c6783ac03221353dbe6" member_type: VOTER last_known_addr { host: "127.1.135.193" port: 42577 } } peers { permanent_uuid: "d269cfa9b6874c6fb8b07b0da9f00ea5" member_type: VOTER last_known_addr { host: "127.1.135.194" port: 36299 } } peers { permanent_uuid: "ece2a20b74874eb7acf32b82a1691f97" member_type: VOTER last_known_addr { host: "127.1.135.195" port: 35699 } }
I20260923 22:12:29.748003 3889 ts_tablet_manager.cc:1428] T c117a3888ff548ba98e428042f09149b P ece2a20b74874eb7acf32b82a1691f97: Time spent starting tablet: real 0.001s user 0.000s sys 0.000s
I20260923 22:12:29.812995 3894 raft_consensus.cc:482] T c117a3888ff548ba98e428042f09149b P d269cfa9b6874c6fb8b07b0da9f00ea5 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260923 22:12:29.813089 3894 raft_consensus.cc:504] T c117a3888ff548ba98e428042f09149b P d269cfa9b6874c6fb8b07b0da9f00ea5 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "04978c60cca64c6783ac03221353dbe6" member_type: VOTER last_known_addr { host: "127.1.135.193" port: 42577 } } peers { permanent_uuid: "d269cfa9b6874c6fb8b07b0da9f00ea5" member_type: VOTER last_known_addr { host: "127.1.135.194" port: 36299 } } peers { permanent_uuid: "ece2a20b74874eb7acf32b82a1691f97" member_type: VOTER last_known_addr { host: "127.1.135.195" port: 35699 } }
I20260923 22:12:29.813467 3894 leader_election.cc:288] T c117a3888ff548ba98e428042f09149b P d269cfa9b6874c6fb8b07b0da9f00ea5 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 04978c60cca64c6783ac03221353dbe6 (127.1.135.193:42577), ece2a20b74874eb7acf32b82a1691f97 (127.1.135.195:35699)
I20260923 22:12:29.816753 3896 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 22:12:29.813633 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.135.195:35699 (local address 127.1.135.194:41251)
0923 22:12:29.813814 (+ 181us) negotiation.cc:107] Waiting for socket to connect
0923 22:12:29.813826 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0923 22:12:29.813900 (+ 74us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0923 22:12:29.814099 (+ 199us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0923 22:12:29.814100 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0923 22:12:29.814103 (+ 3us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0923 22:12:29.814185 (+ 82us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 22:12:29.814187 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 22:12:29.814964 (+ 777us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 22:12:29.814965 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 22:12:29.815798 (+ 833us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 22:12:29.815800 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 22:12:29.816437 (+ 637us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 22:12:29.816438 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 22:12:29.816517 (+ 79us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 22:12:29.816522 (+ 5us) client_negotiation.cc:769] Sending connection context
0923 22:12:29.816583 (+ 61us) client_negotiation.cc:240] Negotiation successful
0923 22:12:29.816628 (+ 45us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":163,"thread_start_us":128,"threads_started":1}
I20260923 22:12:29.816735 3898 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 22:12:29.813709 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.135.194:41251 (local address 127.1.135.195:35699)
0923 22:12:29.813832 (+ 123us) server_negotiation.cc:206] Beginning negotiation
0923 22:12:29.813834 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0923 22:12:29.813888 (+ 54us) server_negotiation.cc:407] Connection header received
0923 22:12:29.813994 (+ 106us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 22:12:29.813995 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 22:12:29.814008 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 22:12:29.814072 (+ 64us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0923 22:12:29.814253 (+ 181us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 22:12:29.814868 (+ 615us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 22:12:29.815883 (+ 1015us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 22:12:29.816369 (+ 486us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 22:12:29.816425 (+ 56us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 22:12:29.816432 (+ 7us) server_negotiation.cc:1035] Waiting for connection context
0923 22:12:29.816611 (+ 179us) server_negotiation.cc:299] Negotiation successful
0923 22:12:29.816648 (+ 37us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":96,"thread_start_us":72,"threads_started":1}
I20260923 22:12:29.816970 3897 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 22:12:29.813676 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.135.194:59133 (local address 127.1.135.193:42577)
0923 22:12:29.813817 (+ 141us) server_negotiation.cc:206] Beginning negotiation
0923 22:12:29.813820 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0923 22:12:29.813944 (+ 124us) server_negotiation.cc:407] Connection header received
0923 22:12:29.814016 (+ 72us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 22:12:29.814017 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 22:12:29.814026 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 22:12:29.814107 (+ 81us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0923 22:12:29.814586 (+ 479us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 22:12:29.815145 (+ 559us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 22:12:29.816104 (+ 959us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 22:12:29.816551 (+ 447us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 22:12:29.816612 (+ 61us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 22:12:29.816619 (+ 7us) server_negotiation.cc:1035] Waiting for connection context
0923 22:12:29.816787 (+ 168us) server_negotiation.cc:299] Negotiation successful
0923 22:12:29.816895 (+ 108us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":116,"thread_start_us":79,"threads_started":1}
I20260923 22:12:29.816973 3895 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 22:12:29.813633 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.135.193:42577 (local address 127.1.135.194:59133)
0923 22:12:29.813859 (+ 226us) negotiation.cc:107] Waiting for socket to connect
0923 22:12:29.813869 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0923 22:12:29.813933 (+ 64us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0923 22:12:29.814122 (+ 189us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0923 22:12:29.814123 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0923 22:12:29.814125 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0923 22:12:29.814499 (+ 374us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 22:12:29.814502 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 22:12:29.815223 (+ 721us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 22:12:29.815223 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 22:12:29.815993 (+ 770us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 22:12:29.815995 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 22:12:29.816648 (+ 653us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 22:12:29.816648 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 22:12:29.816707 (+ 59us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 22:12:29.816710 (+ 3us) client_negotiation.cc:769] Sending connection context
0923 22:12:29.816762 (+ 52us) client_negotiation.cc:240] Negotiation successful
0923 22:12:29.816798 (+ 36us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":207,"thread_start_us":120,"threads_started":1}
I20260923 22:12:29.817653 3450 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "c117a3888ff548ba98e428042f09149b" candidate_uuid: "d269cfa9b6874c6fb8b07b0da9f00ea5" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "04978c60cca64c6783ac03221353dbe6" is_pre_election: true
I20260923 22:12:29.817682 3730 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "c117a3888ff548ba98e428042f09149b" candidate_uuid: "d269cfa9b6874c6fb8b07b0da9f00ea5" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ece2a20b74874eb7acf32b82a1691f97" is_pre_election: true
I20260923 22:12:29.817755 3450 raft_consensus.cc:2454] T c117a3888ff548ba98e428042f09149b P 04978c60cca64c6783ac03221353dbe6 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d269cfa9b6874c6fb8b07b0da9f00ea5 in term 0.
I20260923 22:12:29.817760 3730 raft_consensus.cc:2454] T c117a3888ff548ba98e428042f09149b P ece2a20b74874eb7acf32b82a1691f97 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d269cfa9b6874c6fb8b07b0da9f00ea5 in term 0.
I20260923 22:12:29.817925 3522 leader_election.cc:302] T c117a3888ff548ba98e428042f09149b P d269cfa9b6874c6fb8b07b0da9f00ea5 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 04978c60cca64c6783ac03221353dbe6, d269cfa9b6874c6fb8b07b0da9f00ea5; no voters:
I20260923 22:12:29.818058 3894 raft_consensus.cc:2781] T c117a3888ff548ba98e428042f09149b P d269cfa9b6874c6fb8b07b0da9f00ea5 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260923 22:12:29.818101 3894 raft_consensus.cc:482] T c117a3888ff548ba98e428042f09149b P d269cfa9b6874c6fb8b07b0da9f00ea5 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260923 22:12:29.818116 3894 raft_consensus.cc:3037] T c117a3888ff548ba98e428042f09149b P d269cfa9b6874c6fb8b07b0da9f00ea5 [term 0 FOLLOWER]: Advancing to term 1
I20260923 22:12:29.818774 3894 raft_consensus.cc:504] T c117a3888ff548ba98e428042f09149b P d269cfa9b6874c6fb8b07b0da9f00ea5 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "04978c60cca64c6783ac03221353dbe6" member_type: VOTER last_known_addr { host: "127.1.135.193" port: 42577 } } peers { permanent_uuid: "d269cfa9b6874c6fb8b07b0da9f00ea5" member_type: VOTER last_known_addr { host: "127.1.135.194" port: 36299 } } peers { permanent_uuid: "ece2a20b74874eb7acf32b82a1691f97" member_type: VOTER last_known_addr { host: "127.1.135.195" port: 35699 } }
I20260923 22:12:29.818919 3894 leader_election.cc:288] T c117a3888ff548ba98e428042f09149b P d269cfa9b6874c6fb8b07b0da9f00ea5 [CANDIDATE]: Term 1 election: Requested vote from peers 04978c60cca64c6783ac03221353dbe6 (127.1.135.193:42577), ece2a20b74874eb7acf32b82a1691f97 (127.1.135.195:35699)
I20260923 22:12:29.819039 3450 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "c117a3888ff548ba98e428042f09149b" candidate_uuid: "d269cfa9b6874c6fb8b07b0da9f00ea5" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "04978c60cca64c6783ac03221353dbe6"
I20260923 22:12:29.819106 3450 raft_consensus.cc:3037] T c117a3888ff548ba98e428042f09149b P 04978c60cca64c6783ac03221353dbe6 [term 0 FOLLOWER]: Advancing to term 1
I20260923 22:12:29.819083 3730 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "c117a3888ff548ba98e428042f09149b" candidate_uuid: "d269cfa9b6874c6fb8b07b0da9f00ea5" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ece2a20b74874eb7acf32b82a1691f97"
I20260923 22:12:29.819185 3730 raft_consensus.cc:3037] T c117a3888ff548ba98e428042f09149b P ece2a20b74874eb7acf32b82a1691f97 [term 0 FOLLOWER]: Advancing to term 1
I20260923 22:12:29.819679 3450 raft_consensus.cc:2454] T c117a3888ff548ba98e428042f09149b P 04978c60cca64c6783ac03221353dbe6 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d269cfa9b6874c6fb8b07b0da9f00ea5 in term 1.
I20260923 22:12:29.819842 3522 leader_election.cc:302] T c117a3888ff548ba98e428042f09149b P d269cfa9b6874c6fb8b07b0da9f00ea5 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 04978c60cca64c6783ac03221353dbe6, d269cfa9b6874c6fb8b07b0da9f00ea5; no voters:
I20260923 22:12:29.819882 3730 raft_consensus.cc:2454] T c117a3888ff548ba98e428042f09149b P ece2a20b74874eb7acf32b82a1691f97 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d269cfa9b6874c6fb8b07b0da9f00ea5 in term 1.
I20260923 22:12:29.819936 3894 raft_consensus.cc:2781] T c117a3888ff548ba98e428042f09149b P d269cfa9b6874c6fb8b07b0da9f00ea5 [term 1 FOLLOWER]: Leader election won for term 1
I20260923 22:12:29.820119 3894 raft_consensus.cc:686] T c117a3888ff548ba98e428042f09149b P d269cfa9b6874c6fb8b07b0da9f00ea5 [term 1 LEADER]: Becoming Leader. State: Replica: d269cfa9b6874c6fb8b07b0da9f00ea5, State: Running, Role: LEADER
I20260923 22:12:29.820218 3894 consensus_queue.cc:237] T c117a3888ff548ba98e428042f09149b P d269cfa9b6874c6fb8b07b0da9f00ea5 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "04978c60cca64c6783ac03221353dbe6" member_type: VOTER last_known_addr { host: "127.1.135.193" port: 42577 } } peers { permanent_uuid: "d269cfa9b6874c6fb8b07b0da9f00ea5" member_type: VOTER last_known_addr { host: "127.1.135.194" port: 36299 } } peers { permanent_uuid: "ece2a20b74874eb7acf32b82a1691f97" member_type: VOTER last_known_addr { host: "127.1.135.195" port: 35699 } }
I20260923 22:12:29.820904 3293 catalog_manager.cc:5495] T c117a3888ff548ba98e428042f09149b P d269cfa9b6874c6fb8b07b0da9f00ea5 reported cstate change: term changed from 0 to 1, leader changed from <none> to d269cfa9b6874c6fb8b07b0da9f00ea5 (127.1.135.194). New cstate: current_term: 1 leader_uuid: "d269cfa9b6874c6fb8b07b0da9f00ea5" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "04978c60cca64c6783ac03221353dbe6" member_type: VOTER last_known_addr { host: "127.1.135.193" port: 42577 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "d269cfa9b6874c6fb8b07b0da9f00ea5" member_type: VOTER last_known_addr { host: "127.1.135.194" port: 36299 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "ece2a20b74874eb7acf32b82a1691f97" member_type: VOTER last_known_addr { host: "127.1.135.195" port: 35699 } health_report { overall_health: UNKNOWN } } }
I20260923 22:12:29.841603 3902 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 22:12:29.839164 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48184 (local address 127.1.135.194:36299)
0923 22:12:29.839300 (+ 136us) server_negotiation.cc:206] Beginning negotiation
0923 22:12:29.839302 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0923 22:12:29.839310 (+ 8us) server_negotiation.cc:407] Connection header received
0923 22:12:29.839351 (+ 41us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 22:12:29.839352 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 22:12:29.839374 (+ 22us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 22:12:29.839456 (+ 82us) server_negotiation.cc:226] Negotiated authn=TOKEN
0923 22:12:29.839578 (+ 122us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 22:12:29.840128 (+ 550us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 22:12:29.840850 (+ 722us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 22:12:29.841045 (+ 195us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 22:12:29.841102 (+ 57us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 22:12:29.841242 (+ 140us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0923 22:12:29.841355 (+ 113us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0923 22:12:29.841411 (+ 56us) server_negotiation.cc:1035] Waiting for connection context
0923 22:12:29.841448 (+ 37us) server_negotiation.cc:299] Negotiation successful
0923 22:12:29.841464 (+ 16us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":115,"thread_start_us":89,"threads_started":1}
I20260923 22:12:29.845952 3898 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 22:12:29.843940 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.135.194:34327 (local address 127.1.135.195:35699)
0923 22:12:29.843977 (+ 37us) server_negotiation.cc:206] Beginning negotiation
0923 22:12:29.843979 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0923 22:12:29.844019 (+ 40us) server_negotiation.cc:407] Connection header received
0923 22:12:29.844109 (+ 90us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 22:12:29.844109 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 22:12:29.844120 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 22:12:29.844168 (+ 48us) server_negotiation.cc:226] Negotiated authn=TOKEN
0923 22:12:29.844301 (+ 133us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 22:12:29.844752 (+ 451us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 22:12:29.845436 (+ 684us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 22:12:29.845607 (+ 171us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 22:12:29.845654 (+ 47us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 22:12:29.845782 (+ 128us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0923 22:12:29.845804 (+ 22us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0923 22:12:29.845839 (+ 35us) server_negotiation.cc:1035] Waiting for connection context
0923 22:12:29.845889 (+ 50us) server_negotiation.cc:299] Negotiation successful
0923 22:12:29.845904 (+ 15us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":19}
I20260923 22:12:29.845984 3905 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 22:12:29.843789 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.135.195:35699 (local address 127.1.135.194:34327)
0923 22:12:29.843961 (+ 172us) negotiation.cc:107] Waiting for socket to connect
0923 22:12:29.843972 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0923 22:12:29.844031 (+ 59us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0923 22:12:29.844183 (+ 152us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0923 22:12:29.844183 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0923 22:12:29.844185 (+ 2us) client_negotiation.cc:189] Negotiated authn=TOKEN
0923 22:12:29.844242 (+ 57us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 22:12:29.844243 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 22:12:29.844825 (+ 582us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 22:12:29.844826 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 22:12:29.845372 (+ 546us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 22:12:29.845373 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 22:12:29.845670 (+ 297us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 22:12:29.845671 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 22:12:29.845714 (+ 43us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 22:12:29.845724 (+ 10us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0923 22:12:29.845847 (+ 123us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0923 22:12:29.845848 (+ 1us) client_negotiation.cc:769] Sending connection context
0923 22:12:29.845885 (+ 37us) client_negotiation.cc:240] Negotiation successful
0923 22:12:29.845914 (+ 29us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":136,"thread_start_us":75,"threads_started":1}
I20260923 22:12:29.847958 3450 raft_consensus.cc:1261] T c117a3888ff548ba98e428042f09149b P 04978c60cca64c6783ac03221353dbe6 [term 1 FOLLOWER]: Refusing update from remote peer d269cfa9b6874c6fb8b07b0da9f00ea5: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260923 22:12:29.848007 3730 raft_consensus.cc:1261] T c117a3888ff548ba98e428042f09149b P ece2a20b74874eb7acf32b82a1691f97 [term 1 FOLLOWER]: Refusing update from remote peer d269cfa9b6874c6fb8b07b0da9f00ea5: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260923 22:12:29.848253 3894 consensus_queue.cc:1035] T c117a3888ff548ba98e428042f09149b P d269cfa9b6874c6fb8b07b0da9f00ea5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "04978c60cca64c6783ac03221353dbe6" member_type: VOTER last_known_addr { host: "127.1.135.193" port: 42577 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260923 22:12:29.848387 3899 consensus_queue.cc:1035] T c117a3888ff548ba98e428042f09149b P d269cfa9b6874c6fb8b07b0da9f00ea5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "ece2a20b74874eb7acf32b82a1691f97" member_type: VOTER last_known_addr { host: "127.1.135.195" port: 35699 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260923 22:12:29.855084 3902 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 22:12:29.852829 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.135.195:60133 (local address 127.1.135.194:36299)
0923 22:12:29.852899 (+ 70us) server_negotiation.cc:206] Beginning negotiation
0923 22:12:29.852901 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0923 22:12:29.853016 (+ 115us) server_negotiation.cc:407] Connection header received
0923 22:12:29.853095 (+ 79us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 22:12:29.853096 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 22:12:29.853106 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 22:12:29.853148 (+ 42us) server_negotiation.cc:226] Negotiated authn=TOKEN
0923 22:12:29.853244 (+ 96us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 22:12:29.853685 (+ 441us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 22:12:29.854406 (+ 721us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 22:12:29.854681 (+ 275us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 22:12:29.854743 (+ 62us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 22:12:29.854892 (+ 149us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0923 22:12:29.854923 (+ 31us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0923 22:12:29.854970 (+ 47us) server_negotiation.cc:1035] Waiting for connection context
0923 22:12:29.855026 (+ 56us) server_negotiation.cc:299] Negotiation successful
0923 22:12:29.855041 (+ 15us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":34}
I20260923 22:12:29.855118 3911 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 22:12:29.852800 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.135.194:36299 (local address 127.1.135.195:60133)
0923 22:12:29.852967 (+ 167us) negotiation.cc:107] Waiting for socket to connect
0923 22:12:29.852978 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0923 22:12:29.853029 (+ 51us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0923 22:12:29.853159 (+ 130us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0923 22:12:29.853159 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0923 22:12:29.853161 (+ 2us) client_negotiation.cc:189] Negotiated authn=TOKEN
0923 22:12:29.853199 (+ 38us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 22:12:29.853200 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 22:12:29.853770 (+ 570us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 22:12:29.853771 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 22:12:29.854307 (+ 536us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 22:12:29.854308 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 22:12:29.854741 (+ 433us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 22:12:29.854743 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 22:12:29.854806 (+ 63us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 22:12:29.854812 (+ 6us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0923 22:12:29.854972 (+ 160us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0923 22:12:29.854974 (+ 2us) client_negotiation.cc:769] Sending connection context
0923 22:12:29.855023 (+ 49us) client_negotiation.cc:240] Negotiation successful
0923 22:12:29.855067 (+ 44us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":143,"thread_start_us":95,"threads_started":1}
I20260923 22:12:29.861578 3292 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:35770:
table { table_name: "test-table" } modify_external_catalogs: true
I20260923 22:12:29.861685 3292 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:35770:
table { table_name: "test-table" } modify_external_catalogs: true
I20260923 22:12:29.862306 3292 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 24a871a68671419f89d1a0d5a2bdcd3f: Sending DeleteTablet for 3 replicas of tablet c117a3888ff548ba98e428042f09149b
I20260923 22:12:29.862517 3430 tablet_service.cc:1514] Processing DeleteTablet for tablet c117a3888ff548ba98e428042f09149b with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-23 22:12:29 UTC) from {username='kudu', principal='kudu/127.1.135.254@KRBTEST.COM'} at 127.0.0.1:37922
I20260923 22:12:29.862540 3570 tablet_service.cc:1514] Processing DeleteTablet for tablet c117a3888ff548ba98e428042f09149b with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-23 22:12:29 UTC) from {username='kudu', principal='kudu/127.1.135.254@KRBTEST.COM'} at 127.0.0.1:48168
I20260923 22:12:29.862586 3710 tablet_service.cc:1514] Processing DeleteTablet for tablet c117a3888ff548ba98e428042f09149b with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-23 22:12:29 UTC) from {username='kudu', principal='kudu/127.1.135.254@KRBTEST.COM'} at 127.0.0.1:41660
I20260923 22:12:29.862612 3877 tablet_replica.cc:330] T c117a3888ff548ba98e428042f09149b P 04978c60cca64c6783ac03221353dbe6: stopping tablet replica
I20260923 22:12:29.862628 3879 tablet_replica.cc:330] T c117a3888ff548ba98e428042f09149b P d269cfa9b6874c6fb8b07b0da9f00ea5: stopping tablet replica
I20260923 22:12:29.862664 3877 raft_consensus.cc:2229] T c117a3888ff548ba98e428042f09149b P 04978c60cca64c6783ac03221353dbe6 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260923 22:12:29.862663 3878 tablet_replica.cc:330] T c117a3888ff548ba98e428042f09149b P ece2a20b74874eb7acf32b82a1691f97: stopping tablet replica
I20260923 22:12:29.862677 3879 raft_consensus.cc:2229] T c117a3888ff548ba98e428042f09149b P d269cfa9b6874c6fb8b07b0da9f00ea5 [term 1 LEADER]: Raft consensus shutting down.
I20260923 22:12:29.862700 3877 raft_consensus.cc:2258] T c117a3888ff548ba98e428042f09149b P 04978c60cca64c6783ac03221353dbe6 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260923 22:12:29.862713 3878 raft_consensus.cc:2229] T c117a3888ff548ba98e428042f09149b P ece2a20b74874eb7acf32b82a1691f97 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260923 22:12:29.862749 3878 raft_consensus.cc:2258] T c117a3888ff548ba98e428042f09149b P ece2a20b74874eb7acf32b82a1691f97 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260923 22:12:29.862754 3879 raft_consensus.cc:2258] T c117a3888ff548ba98e428042f09149b P d269cfa9b6874c6fb8b07b0da9f00ea5 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260923 22:12:29.862949 3292 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:35770:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20260923 22:12:29.863067 3877 ts_tablet_manager.cc:1905] T c117a3888ff548ba98e428042f09149b P 04978c60cca64c6783ac03221353dbe6: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260923 22:12:29.863065 3879 ts_tablet_manager.cc:1905] T c117a3888ff548ba98e428042f09149b P d269cfa9b6874c6fb8b07b0da9f00ea5: Deleting tablet data with delete state TABLET_DATA_DELETED
W20260923 22:12:29.863113 3292 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260923 22:12:29.863687 3878 ts_tablet_manager.cc:1905] T c117a3888ff548ba98e428042f09149b P ece2a20b74874eb7acf32b82a1691f97: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260923 22:12:29.864557 3879 ts_tablet_manager.cc:1918] T c117a3888ff548ba98e428042f09149b P d269cfa9b6874c6fb8b07b0da9f00ea5: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260923 22:12:29.864581 3877 ts_tablet_manager.cc:1918] T c117a3888ff548ba98e428042f09149b P 04978c60cca64c6783ac03221353dbe6: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260923 22:12:29.864611 3879 log.cc:1196] T c117a3888ff548ba98e428042f09149b P d269cfa9b6874c6fb8b07b0da9f00ea5: Deleting WAL directory at /tmp/dist-test-taskRiQe_a/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790201539513534-1567-0/minicluster-data/ts-1/wal/wals/c117a3888ff548ba98e428042f09149b
I20260923 22:12:29.864646 3877 log.cc:1196] T c117a3888ff548ba98e428042f09149b P 04978c60cca64c6783ac03221353dbe6: Deleting WAL directory at /tmp/dist-test-taskRiQe_a/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790201539513534-1567-0/minicluster-data/ts-0/wal/wals/c117a3888ff548ba98e428042f09149b
I20260923 22:12:29.864886 3879 ts_tablet_manager.cc:1939] T c117a3888ff548ba98e428042f09149b P d269cfa9b6874c6fb8b07b0da9f00ea5: Deleting consensus metadata
I20260923 22:12:29.864910 3877 ts_tablet_manager.cc:1939] T c117a3888ff548ba98e428042f09149b P 04978c60cca64c6783ac03221353dbe6: Deleting consensus metadata
I20260923 22:12:29.865199 3878 ts_tablet_manager.cc:1918] T c117a3888ff548ba98e428042f09149b P ece2a20b74874eb7acf32b82a1691f97: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260923 22:12:29.865255 3878 log.cc:1196] T c117a3888ff548ba98e428042f09149b P ece2a20b74874eb7acf32b82a1691f97: Deleting WAL directory at /tmp/dist-test-taskRiQe_a/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790201539513534-1567-0/minicluster-data/ts-2/wal/wals/c117a3888ff548ba98e428042f09149b
I20260923 22:12:29.865372 3278 catalog_manager.cc:4841] TS 04978c60cca64c6783ac03221353dbe6 (127.1.135.193:42577): tablet c117a3888ff548ba98e428042f09149b (table test-table [id=3299e8a1615c467faba379667a4520cb]) successfully deleted
I20260923 22:12:29.865454 3878 ts_tablet_manager.cc:1939] T c117a3888ff548ba98e428042f09149b P ece2a20b74874eb7acf32b82a1691f97: Deleting consensus metadata
I20260923 22:12:29.865563 3277 catalog_manager.cc:4841] TS d269cfa9b6874c6fb8b07b0da9f00ea5 (127.1.135.194:36299): tablet c117a3888ff548ba98e428042f09149b (table test-table [id=3299e8a1615c467faba379667a4520cb]) successfully deleted
I20260923 22:12:29.866429 3430 tablet_service.cc:1467] Processing CreateTablet for tablet 2455c4cf08d14be8a152fa1bfaba9b1b (DEFAULT_TABLE table=test-table [id=8c89a0d6c8604b68ab0ebc68b1d9ade8]), partition=RANGE (key) PARTITION UNBOUNDED
I20260923 22:12:29.866540 3430 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 2455c4cf08d14be8a152fa1bfaba9b1b. 1 dirs total, 0 dirs full, 0 dirs failed
I20260923 22:12:29.866521 3710 tablet_service.cc:1467] Processing CreateTablet for tablet 2455c4cf08d14be8a152fa1bfaba9b1b (DEFAULT_TABLE table=test-table [id=8c89a0d6c8604b68ab0ebc68b1d9ade8]), partition=RANGE (key) PARTITION UNBOUNDED
I20260923 22:12:29.866540 3570 tablet_service.cc:1467] Processing CreateTablet for tablet 2455c4cf08d14be8a152fa1bfaba9b1b (DEFAULT_TABLE table=test-table [id=8c89a0d6c8604b68ab0ebc68b1d9ade8]), partition=RANGE (key) PARTITION UNBOUNDED
I20260923 22:12:29.866616 3570 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 2455c4cf08d14be8a152fa1bfaba9b1b. 1 dirs total, 0 dirs full, 0 dirs failed
I20260923 22:12:29.866703 3710 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 2455c4cf08d14be8a152fa1bfaba9b1b. 1 dirs total, 0 dirs full, 0 dirs failed
I20260923 22:12:29.867514 3889 tablet_bootstrap.cc:492] T 2455c4cf08d14be8a152fa1bfaba9b1b P ece2a20b74874eb7acf32b82a1691f97: Bootstrap starting.
I20260923 22:12:29.867689 3888 tablet_bootstrap.cc:492] T 2455c4cf08d14be8a152fa1bfaba9b1b P d269cfa9b6874c6fb8b07b0da9f00ea5: Bootstrap starting.
I20260923 22:12:29.867689 3890 tablet_bootstrap.cc:492] T 2455c4cf08d14be8a152fa1bfaba9b1b P 04978c60cca64c6783ac03221353dbe6: Bootstrap starting.
I20260923 22:12:29.867769 3277 catalog_manager.cc:4841] TS ece2a20b74874eb7acf32b82a1691f97 (127.1.135.195:35699): tablet c117a3888ff548ba98e428042f09149b (table test-table [id=3299e8a1615c467faba379667a4520cb]) successfully deleted
I20260923 22:12:29.867799 3889 tablet_bootstrap.cc:654] T 2455c4cf08d14be8a152fa1bfaba9b1b P ece2a20b74874eb7acf32b82a1691f97: Neither blocks nor log segments found. Creating new log.
I20260923 22:12:29.868040 3890 tablet_bootstrap.cc:654] T 2455c4cf08d14be8a152fa1bfaba9b1b P 04978c60cca64c6783ac03221353dbe6: Neither blocks nor log segments found. Creating new log.
I20260923 22:12:29.868098 3888 tablet_bootstrap.cc:654] T 2455c4cf08d14be8a152fa1bfaba9b1b P d269cfa9b6874c6fb8b07b0da9f00ea5: Neither blocks nor log segments found. Creating new log.
I20260923 22:12:29.868610 3890 tablet_bootstrap.cc:492] T 2455c4cf08d14be8a152fa1bfaba9b1b P 04978c60cca64c6783ac03221353dbe6: No bootstrap required, opened a new log
I20260923 22:12:29.868686 3890 ts_tablet_manager.cc:1397] T 2455c4cf08d14be8a152fa1bfaba9b1b P 04978c60cca64c6783ac03221353dbe6: Time spent bootstrapping tablet: real 0.001s user 0.001s sys 0.000s
I20260923 22:12:29.868824 3889 tablet_bootstrap.cc:492] T 2455c4cf08d14be8a152fa1bfaba9b1b P ece2a20b74874eb7acf32b82a1691f97: No bootstrap required, opened a new log
I20260923 22:12:29.868793 3890 raft_consensus.cc:348] T 2455c4cf08d14be8a152fa1bfaba9b1b P 04978c60cca64c6783ac03221353dbe6 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "04978c60cca64c6783ac03221353dbe6" member_type: VOTER last_known_addr { host: "127.1.135.193" port: 42577 } } peers { permanent_uuid: "d269cfa9b6874c6fb8b07b0da9f00ea5" member_type: VOTER last_known_addr { host: "127.1.135.194" port: 36299 } } peers { permanent_uuid: "ece2a20b74874eb7acf32b82a1691f97" member_type: VOTER last_known_addr { host: "127.1.135.195" port: 35699 } }
I20260923 22:12:29.868884 3890 raft_consensus.cc:374] T 2455c4cf08d14be8a152fa1bfaba9b1b P 04978c60cca64c6783ac03221353dbe6 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260923 22:12:29.868889 3889 ts_tablet_manager.cc:1397] T 2455c4cf08d14be8a152fa1bfaba9b1b P ece2a20b74874eb7acf32b82a1691f97: Time spent bootstrapping tablet: real 0.001s user 0.001s sys 0.000s
I20260923 22:12:29.868916 3890 raft_consensus.cc:729] T 2455c4cf08d14be8a152fa1bfaba9b1b P 04978c60cca64c6783ac03221353dbe6 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 04978c60cca64c6783ac03221353dbe6, State: Initialized, Role: FOLLOWER
I20260923 22:12:29.868973 3890 consensus_queue.cc:260] T 2455c4cf08d14be8a152fa1bfaba9b1b P 04978c60cca64c6783ac03221353dbe6 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "04978c60cca64c6783ac03221353dbe6" member_type: VOTER last_known_addr { host: "127.1.135.193" port: 42577 } } peers { permanent_uuid: "d269cfa9b6874c6fb8b07b0da9f00ea5" member_type: VOTER last_known_addr { host: "127.1.135.194" port: 36299 } } peers { permanent_uuid: "ece2a20b74874eb7acf32b82a1691f97" member_type: VOTER last_known_addr { host: "127.1.135.195" port: 35699 } }
I20260923 22:12:29.869002 3889 raft_consensus.cc:348] T 2455c4cf08d14be8a152fa1bfaba9b1b P ece2a20b74874eb7acf32b82a1691f97 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "04978c60cca64c6783ac03221353dbe6" member_type: VOTER last_known_addr { host: "127.1.135.193" port: 42577 } } peers { permanent_uuid: "d269cfa9b6874c6fb8b07b0da9f00ea5" member_type: VOTER last_known_addr { host: "127.1.135.194" port: 36299 } } peers { permanent_uuid: "ece2a20b74874eb7acf32b82a1691f97" member_type: VOTER last_known_addr { host: "127.1.135.195" port: 35699 } }
I20260923 22:12:29.869060 3889 raft_consensus.cc:374] T 2455c4cf08d14be8a152fa1bfaba9b1b P ece2a20b74874eb7acf32b82a1691f97 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260923 22:12:29.869061 3890 ts_tablet_manager.cc:1428] T 2455c4cf08d14be8a152fa1bfaba9b1b P 04978c60cca64c6783ac03221353dbe6: Time spent starting tablet: real 0.000s user 0.001s sys 0.000s
I20260923 22:12:29.869081 3889 raft_consensus.cc:729] T 2455c4cf08d14be8a152fa1bfaba9b1b P ece2a20b74874eb7acf32b82a1691f97 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: ece2a20b74874eb7acf32b82a1691f97, State: Initialized, Role: FOLLOWER
I20260923 22:12:29.869122 3889 consensus_queue.cc:260] T 2455c4cf08d14be8a152fa1bfaba9b1b P ece2a20b74874eb7acf32b82a1691f97 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "04978c60cca64c6783ac03221353dbe6" member_type: VOTER last_known_addr { host: "127.1.135.193" port: 42577 } } peers { permanent_uuid: "d269cfa9b6874c6fb8b07b0da9f00ea5" member_type: VOTER last_known_addr { host: "127.1.135.194" port: 36299 } } peers { permanent_uuid: "ece2a20b74874eb7acf32b82a1691f97" member_type: VOTER last_known_addr { host: "127.1.135.195" port: 35699 } }
I20260923 22:12:29.869205 3889 ts_tablet_manager.cc:1428] T 2455c4cf08d14be8a152fa1bfaba9b1b P ece2a20b74874eb7acf32b82a1691f97: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20260923 22:12:29.875334 3888 tablet_bootstrap.cc:492] T 2455c4cf08d14be8a152fa1bfaba9b1b P d269cfa9b6874c6fb8b07b0da9f00ea5: No bootstrap required, opened a new log
I20260923 22:12:29.875392 3888 ts_tablet_manager.cc:1397] T 2455c4cf08d14be8a152fa1bfaba9b1b P d269cfa9b6874c6fb8b07b0da9f00ea5: Time spent bootstrapping tablet: real 0.008s user 0.000s sys 0.001s
I20260923 22:12:29.875519 3888 raft_consensus.cc:348] T 2455c4cf08d14be8a152fa1bfaba9b1b P d269cfa9b6874c6fb8b07b0da9f00ea5 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "04978c60cca64c6783ac03221353dbe6" member_type: VOTER last_known_addr { host: "127.1.135.193" port: 42577 } } peers { permanent_uuid: "d269cfa9b6874c6fb8b07b0da9f00ea5" member_type: VOTER last_known_addr { host: "127.1.135.194" port: 36299 } } peers { permanent_uuid: "ece2a20b74874eb7acf32b82a1691f97" member_type: VOTER last_known_addr { host: "127.1.135.195" port: 35699 } }
I20260923 22:12:29.875591 3888 raft_consensus.cc:374] T 2455c4cf08d14be8a152fa1bfaba9b1b P d269cfa9b6874c6fb8b07b0da9f00ea5 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260923 22:12:29.875612 3888 raft_consensus.cc:729] T 2455c4cf08d14be8a152fa1bfaba9b1b P d269cfa9b6874c6fb8b07b0da9f00ea5 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d269cfa9b6874c6fb8b07b0da9f00ea5, State: Initialized, Role: FOLLOWER
I20260923 22:12:29.875650 3888 consensus_queue.cc:260] T 2455c4cf08d14be8a152fa1bfaba9b1b P d269cfa9b6874c6fb8b07b0da9f00ea5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "04978c60cca64c6783ac03221353dbe6" member_type: VOTER last_known_addr { host: "127.1.135.193" port: 42577 } } peers { permanent_uuid: "d269cfa9b6874c6fb8b07b0da9f00ea5" member_type: VOTER last_known_addr { host: "127.1.135.194" port: 36299 } } peers { permanent_uuid: "ece2a20b74874eb7acf32b82a1691f97" member_type: VOTER last_known_addr { host: "127.1.135.195" port: 35699 } }
I20260923 22:12:29.875763 3888 ts_tablet_manager.cc:1428] T 2455c4cf08d14be8a152fa1bfaba9b1b P d269cfa9b6874c6fb8b07b0da9f00ea5: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20260923 22:12:30.034732 3871 raft_consensus.cc:482] T 2455c4cf08d14be8a152fa1bfaba9b1b P 04978c60cca64c6783ac03221353dbe6 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260923 22:12:30.034852 3871 raft_consensus.cc:504] T 2455c4cf08d14be8a152fa1bfaba9b1b P 04978c60cca64c6783ac03221353dbe6 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "04978c60cca64c6783ac03221353dbe6" member_type: VOTER last_known_addr { host: "127.1.135.193" port: 42577 } } peers { permanent_uuid: "d269cfa9b6874c6fb8b07b0da9f00ea5" member_type: VOTER last_known_addr { host: "127.1.135.194" port: 36299 } } peers { permanent_uuid: "ece2a20b74874eb7acf32b82a1691f97" member_type: VOTER last_known_addr { host: "127.1.135.195" port: 35699 } }
I20260923 22:12:30.035058 3871 leader_election.cc:288] T 2455c4cf08d14be8a152fa1bfaba9b1b P 04978c60cca64c6783ac03221353dbe6 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers d269cfa9b6874c6fb8b07b0da9f00ea5 (127.1.135.194:36299), ece2a20b74874eb7acf32b82a1691f97 (127.1.135.195:35699)
I20260923 22:12:30.035300 3730 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "2455c4cf08d14be8a152fa1bfaba9b1b" candidate_uuid: "04978c60cca64c6783ac03221353dbe6" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ece2a20b74874eb7acf32b82a1691f97" is_pre_election: true
I20260923 22:12:30.035300 3590 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "2455c4cf08d14be8a152fa1bfaba9b1b" candidate_uuid: "04978c60cca64c6783ac03221353dbe6" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d269cfa9b6874c6fb8b07b0da9f00ea5" is_pre_election: true
I20260923 22:12:30.035410 3590 raft_consensus.cc:2454] T 2455c4cf08d14be8a152fa1bfaba9b1b P d269cfa9b6874c6fb8b07b0da9f00ea5 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 04978c60cca64c6783ac03221353dbe6 in term 0.
I20260923 22:12:30.035410 3730 raft_consensus.cc:2454] T 2455c4cf08d14be8a152fa1bfaba9b1b P ece2a20b74874eb7acf32b82a1691f97 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 04978c60cca64c6783ac03221353dbe6 in term 0.
I20260923 22:12:30.035594 3381 leader_election.cc:302] T 2455c4cf08d14be8a152fa1bfaba9b1b P 04978c60cca64c6783ac03221353dbe6 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 04978c60cca64c6783ac03221353dbe6, ece2a20b74874eb7acf32b82a1691f97; no voters:
I20260923 22:12:30.035710 3871 raft_consensus.cc:2781] T 2455c4cf08d14be8a152fa1bfaba9b1b P 04978c60cca64c6783ac03221353dbe6 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260923 22:12:30.035787 3871 raft_consensus.cc:482] T 2455c4cf08d14be8a152fa1bfaba9b1b P 04978c60cca64c6783ac03221353dbe6 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260923 22:12:30.035821 3871 raft_consensus.cc:3037] T 2455c4cf08d14be8a152fa1bfaba9b1b P 04978c60cca64c6783ac03221353dbe6 [term 0 FOLLOWER]: Advancing to term 1
I20260923 22:12:30.036558 3871 raft_consensus.cc:504] T 2455c4cf08d14be8a152fa1bfaba9b1b P 04978c60cca64c6783ac03221353dbe6 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "04978c60cca64c6783ac03221353dbe6" member_type: VOTER last_known_addr { host: "127.1.135.193" port: 42577 } } peers { permanent_uuid: "d269cfa9b6874c6fb8b07b0da9f00ea5" member_type: VOTER last_known_addr { host: "127.1.135.194" port: 36299 } } peers { permanent_uuid: "ece2a20b74874eb7acf32b82a1691f97" member_type: VOTER last_known_addr { host: "127.1.135.195" port: 35699 } }
I20260923 22:12:30.036698 3871 leader_election.cc:288] T 2455c4cf08d14be8a152fa1bfaba9b1b P 04978c60cca64c6783ac03221353dbe6 [CANDIDATE]: Term 1 election: Requested vote from peers d269cfa9b6874c6fb8b07b0da9f00ea5 (127.1.135.194:36299), ece2a20b74874eb7acf32b82a1691f97 (127.1.135.195:35699)
I20260923 22:12:30.036916 3590 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "2455c4cf08d14be8a152fa1bfaba9b1b" candidate_uuid: "04978c60cca64c6783ac03221353dbe6" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d269cfa9b6874c6fb8b07b0da9f00ea5"
I20260923 22:12:30.036911 3730 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "2455c4cf08d14be8a152fa1bfaba9b1b" candidate_uuid: "04978c60cca64c6783ac03221353dbe6" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ece2a20b74874eb7acf32b82a1691f97"
I20260923 22:12:30.036991 3730 raft_consensus.cc:3037] T 2455c4cf08d14be8a152fa1bfaba9b1b P ece2a20b74874eb7acf32b82a1691f97 [term 0 FOLLOWER]: Advancing to term 1
I20260923 22:12:30.037009 3590 raft_consensus.cc:3037] T 2455c4cf08d14be8a152fa1bfaba9b1b P d269cfa9b6874c6fb8b07b0da9f00ea5 [term 0 FOLLOWER]: Advancing to term 1
I20260923 22:12:30.037604 3730 raft_consensus.cc:2454] T 2455c4cf08d14be8a152fa1bfaba9b1b P ece2a20b74874eb7acf32b82a1691f97 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 04978c60cca64c6783ac03221353dbe6 in term 1.
I20260923 22:12:30.037658 3590 raft_consensus.cc:2454] T 2455c4cf08d14be8a152fa1bfaba9b1b P d269cfa9b6874c6fb8b07b0da9f00ea5 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 04978c60cca64c6783ac03221353dbe6 in term 1.
I20260923 22:12:30.037812 3381 leader_election.cc:302] T 2455c4cf08d14be8a152fa1bfaba9b1b P 04978c60cca64c6783ac03221353dbe6 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 04978c60cca64c6783ac03221353dbe6, d269cfa9b6874c6fb8b07b0da9f00ea5; no voters:
I20260923 22:12:30.037917 3871 raft_consensus.cc:2781] T 2455c4cf08d14be8a152fa1bfaba9b1b P 04978c60cca64c6783ac03221353dbe6 [term 1 FOLLOWER]: Leader election won for term 1
I20260923 22:12:30.037977 3871 raft_consensus.cc:686] T 2455c4cf08d14be8a152fa1bfaba9b1b P 04978c60cca64c6783ac03221353dbe6 [term 1 LEADER]: Becoming Leader. State: Replica: 04978c60cca64c6783ac03221353dbe6, State: Running, Role: LEADER
I20260923 22:12:30.038033 3871 consensus_queue.cc:237] T 2455c4cf08d14be8a152fa1bfaba9b1b P 04978c60cca64c6783ac03221353dbe6 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "04978c60cca64c6783ac03221353dbe6" member_type: VOTER last_known_addr { host: "127.1.135.193" port: 42577 } } peers { permanent_uuid: "d269cfa9b6874c6fb8b07b0da9f00ea5" member_type: VOTER last_known_addr { host: "127.1.135.194" port: 36299 } } peers { permanent_uuid: "ece2a20b74874eb7acf32b82a1691f97" member_type: VOTER last_known_addr { host: "127.1.135.195" port: 35699 } }
I20260923 22:12:30.038625 3292 catalog_manager.cc:5495] T 2455c4cf08d14be8a152fa1bfaba9b1b P 04978c60cca64c6783ac03221353dbe6 reported cstate change: term changed from 0 to 1, leader changed from <none> to 04978c60cca64c6783ac03221353dbe6 (127.1.135.193). New cstate: current_term: 1 leader_uuid: "04978c60cca64c6783ac03221353dbe6" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "04978c60cca64c6783ac03221353dbe6" member_type: VOTER last_known_addr { host: "127.1.135.193" port: 42577 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "d269cfa9b6874c6fb8b07b0da9f00ea5" member_type: VOTER last_known_addr { host: "127.1.135.194" port: 36299 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "ece2a20b74874eb7acf32b82a1691f97" member_type: VOTER last_known_addr { host: "127.1.135.195" port: 35699 } health_report { overall_health: UNKNOWN } } }
I20260923 22:12:30.052985 3897 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 22:12:30.050711 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37944 (local address 127.1.135.193:42577)
0923 22:12:30.050753 (+ 42us) server_negotiation.cc:206] Beginning negotiation
0923 22:12:30.050755 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0923 22:12:30.050763 (+ 8us) server_negotiation.cc:407] Connection header received
0923 22:12:30.050831 (+ 68us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 22:12:30.050832 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 22:12:30.050841 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 22:12:30.050894 (+ 53us) server_negotiation.cc:226] Negotiated authn=TOKEN
0923 22:12:30.051006 (+ 112us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 22:12:30.051477 (+ 471us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 22:12:30.052281 (+ 804us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 22:12:30.052489 (+ 208us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 22:12:30.052574 (+ 85us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 22:12:30.052715 (+ 141us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0923 22:12:30.052755 (+ 40us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0923 22:12:30.052815 (+ 60us) server_negotiation.cc:1035] Waiting for connection context
0923 22:12:30.052880 (+ 65us) server_negotiation.cc:299] Negotiation successful
0923 22:12:30.052903 (+ 23us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":20}
I20260923 22:12:30.055058 3590 raft_consensus.cc:1261] T 2455c4cf08d14be8a152fa1bfaba9b1b P d269cfa9b6874c6fb8b07b0da9f00ea5 [term 1 FOLLOWER]: Refusing update from remote peer 04978c60cca64c6783ac03221353dbe6: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260923 22:12:30.055104 3730 raft_consensus.cc:1261] T 2455c4cf08d14be8a152fa1bfaba9b1b P ece2a20b74874eb7acf32b82a1691f97 [term 1 FOLLOWER]: Refusing update from remote peer 04978c60cca64c6783ac03221353dbe6: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260923 22:12:30.055294 3871 consensus_queue.cc:1035] T 2455c4cf08d14be8a152fa1bfaba9b1b P 04978c60cca64c6783ac03221353dbe6 [LEADER]: Connected to new peer: Peer: permanent_uuid: "d269cfa9b6874c6fb8b07b0da9f00ea5" member_type: VOTER last_known_addr { host: "127.1.135.194" port: 36299 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260923 22:12:30.055388 3868 consensus_queue.cc:1035] T 2455c4cf08d14be8a152fa1bfaba9b1b P 04978c60cca64c6783ac03221353dbe6 [LEADER]: Connected to new peer: Peer: permanent_uuid: "ece2a20b74874eb7acf32b82a1691f97" member_type: VOTER last_known_addr { host: "127.1.135.195" port: 35699 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
W20260923 22:12:30.059645 3859 tablet_replica.cc:1298] Aborted: operation has been aborted: cancelling pending write operations
I20260923 22:12:31.576753 3291 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:35770:
table { table_name: "test-table" } modify_external_catalogs: true
I20260923 22:12:31.576875 3291 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:35770:
table { table_name: "test-table" } modify_external_catalogs: true
I20260923 22:12:31.577796 3291 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 24a871a68671419f89d1a0d5a2bdcd3f: Sending DeleteTablet for 3 replicas of tablet 2455c4cf08d14be8a152fa1bfaba9b1b
I20260923 22:12:31.578096 3430 tablet_service.cc:1514] Processing DeleteTablet for tablet 2455c4cf08d14be8a152fa1bfaba9b1b with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-23 22:12:31 UTC) from {username='kudu', principal='kudu/127.1.135.254@KRBTEST.COM'} at 127.0.0.1:37922
I20260923 22:12:31.578119 3570 tablet_service.cc:1514] Processing DeleteTablet for tablet 2455c4cf08d14be8a152fa1bfaba9b1b with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-23 22:12:31 UTC) from {username='kudu', principal='kudu/127.1.135.254@KRBTEST.COM'} at 127.0.0.1:48168
I20260923 22:12:31.578169 3710 tablet_service.cc:1514] Processing DeleteTablet for tablet 2455c4cf08d14be8a152fa1bfaba9b1b with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-23 22:12:31 UTC) from {username='kudu', principal='kudu/127.1.135.254@KRBTEST.COM'} at 127.0.0.1:41660
I20260923 22:12:31.578305 3941 tablet_replica.cc:330] T 2455c4cf08d14be8a152fa1bfaba9b1b P 04978c60cca64c6783ac03221353dbe6: stopping tablet replica
I20260923 22:12:31.578357 3943 tablet_replica.cc:330] T 2455c4cf08d14be8a152fa1bfaba9b1b P ece2a20b74874eb7acf32b82a1691f97: stopping tablet replica
I20260923 22:12:31.578436 1567 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskRiQe_a/build/release/bin/kudu with pid 3365
I20260923 22:12:31.579772 3943 raft_consensus.cc:2229] T 2455c4cf08d14be8a152fa1bfaba9b1b P ece2a20b74874eb7acf32b82a1691f97 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260923 22:12:31.579842 3943 raft_consensus.cc:2258] T 2455c4cf08d14be8a152fa1bfaba9b1b P ece2a20b74874eb7acf32b82a1691f97 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260923 22:12:31.580144 3943 ts_tablet_manager.cc:1905] T 2455c4cf08d14be8a152fa1bfaba9b1b P ece2a20b74874eb7acf32b82a1691f97: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260923 22:12:31.581612 3943 ts_tablet_manager.cc:1918] T 2455c4cf08d14be8a152fa1bfaba9b1b P ece2a20b74874eb7acf32b82a1691f97: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.8
I20260923 22:12:31.581661 3943 log.cc:1196] T 2455c4cf08d14be8a152fa1bfaba9b1b P ece2a20b74874eb7acf32b82a1691f97: Deleting WAL directory at /tmp/dist-test-taskRiQe_a/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790201539513534-1567-0/minicluster-data/ts-2/wal/wals/2455c4cf08d14be8a152fa1bfaba9b1b
I20260923 22:12:31.581889 3943 ts_tablet_manager.cc:1939] T 2455c4cf08d14be8a152fa1bfaba9b1b P ece2a20b74874eb7acf32b82a1691f97: Deleting consensus metadata
I20260923 22:12:31.582214 3277 catalog_manager.cc:4841] TS ece2a20b74874eb7acf32b82a1691f97 (127.1.135.195:35699): tablet 2455c4cf08d14be8a152fa1bfaba9b1b (table test-table [id=8c89a0d6c8604b68ab0ebc68b1d9ade8]) successfully deleted
I20260923 22:12:31.584488 3942 tablet_replica.cc:330] T 2455c4cf08d14be8a152fa1bfaba9b1b P d269cfa9b6874c6fb8b07b0da9f00ea5: stopping tablet replica
I20260923 22:12:31.584571 3942 raft_consensus.cc:2229] T 2455c4cf08d14be8a152fa1bfaba9b1b P d269cfa9b6874c6fb8b07b0da9f00ea5 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260923 22:12:31.584618 3942 raft_consensus.cc:2258] T 2455c4cf08d14be8a152fa1bfaba9b1b P d269cfa9b6874c6fb8b07b0da9f00ea5 [term 1 FOLLOWER]: Raft consensus is shut down!
W20260923 22:12:31.584849 3278 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.1.135.193:42577 (error 108)
I20260923 22:12:31.584934 3942 ts_tablet_manager.cc:1905] T 2455c4cf08d14be8a152fa1bfaba9b1b P d269cfa9b6874c6fb8b07b0da9f00ea5: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260923 22:12:31.585175 1567 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskRiQe_a/build/release/bin/kudu with pid 3505
I20260923 22:12:31.585994 3945 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 22:12:31.585289 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.135.193:42577 (local address 127.0.0.1:37946)
0923 22:12:31.585886 (+ 597us) negotiation.cc:107] Waiting for socket to connect
0923 22:12:31.585942 (+ 56us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.135.193:42577: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":570,"thread_start_us":78,"threads_started":1}
W20260923 22:12:31.586149 3278 catalog_manager.cc:4584] TS 04978c60cca64c6783ac03221353dbe6 (127.1.135.193:42577): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet 2455c4cf08d14be8a152fa1bfaba9b1b: Network error: Client connection negotiation failed: client connection to 127.1.135.193:42577: connect: Connection refused (error 111)
I20260923 22:12:31.591974 3945 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 22:12:31.591836 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.135.194:36299 (local address 127.0.0.1:48192)
0923 22:12:31.591879 (+ 43us) negotiation.cc:107] Waiting for socket to connect
0923 22:12:31.591939 (+ 60us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.135.194:36299: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":28}
I20260923 22:12:31.591969 1567 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskRiQe_a/build/release/bin/kudu with pid 3645
I20260923 22:12:31.597429 1567 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskRiQe_a/build/release/bin/kudu with pid 3261
2026-09-23T22:12:31Z chronyd exiting
[ OK ] SecurityITest.TxnSmokeWithDifferentUserTypes (5891 ms)
[ RUN ] SecurityITest.TestNoKerberosCredentials
Loading random data
Initializing database '/tmp/dist-test-taskRiQe_a/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790201539513534-1567-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Sep 23 22:12:31 dist-test-slave-s2fj krb5kdc[3948](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Sep 23 22:12:31 dist-test-slave-s2fj krb5kdc[3948](info): set up 2 sockets
Sep 23 22:12:31 dist-test-slave-s2fj krb5kdc[3948](info): commencing operation
krb5kdc: starting...
W20260923 22:12:32.636125 1567 mini_kdc.cc:121] Time spent starting KDC: real 1.017s user 0.000s sys 0.005s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskRiQe_a/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790201539513534-1567-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskRiQe_a/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790201539513534-1567-0/krb5kdc/test-user.keytab.
Sep 23 22:12:32 dist-test-slave-s2fj krb5kdc[3948](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790201552, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-09-23T22:12:32Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-09-23T22:12:32Z Disabled control of system clock
WARNING: no policy specified for kudu/127.1.135.254@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.1.135.254@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.1.135.254 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskRiQe_a/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790201539513534-1567-0/krb5kdc/kudu_127.1.135.254.keytab.
Entry for principal kudu/127.1.135.254 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskRiQe_a/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790201539513534-1567-0/krb5kdc/kudu_127.1.135.254.keytab.
I20260923 22:12:32.746793 1567 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskRiQe_a/build/release/bin/kudu
/tmp/dist-test-taskRiQe_a/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskRiQe_a/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790201539513534-1567-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskRiQe_a/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790201539513534-1567-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskRiQe_a/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790201539513534-1567-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskRiQe_a/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790201539513534-1567-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.1.135.254:36643
--webserver_interface=127.1.135.254
--webserver_port=0
--keytab_file=/tmp/dist-test-taskRiQe_a/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790201539513534-1567-0/krb5kdc/kudu_127.1.135.254.keytab
--principal=kudu/127.1.135.254
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.1.135.212:37125
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.1.135.254:36643
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskRiQe_a/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790201539513534-1567-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskRiQe_a/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790201539513534-1567-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskRiQe_a/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790201539513534-1567-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260923 22:12:32.825232 3961 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260923 22:12:32.825426 3961 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260923 22:12:32.825448 3961 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260923 22:12:32.826504 3961 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260923 22:12:32.826537 3961 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260923 22:12:32.826552 3961 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260923 22:12:32.826565 3961 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260923 22:12:32.826577 3961 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260923 22:12:32.827929 3961 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.1.135.212:37125
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskRiQe_a/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790201539513534-1567-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskRiQe_a/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790201539513534-1567-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.1.135.254:36643
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskRiQe_a/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790201539513534-1567-0/krb5kdc/kudu_127.1.135.254.keytab
--principal=kudu/127.1.135.254
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.1.135.254:36643
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskRiQe_a/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790201539513534-1567-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.1.135.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.3961
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskRiQe_a/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790201539513534-1567-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 23 Sep 2026 20:53:37 UTC on ccycloud.quasar-rkviyx.root.comops.site
build id 4668083
I20260923 22:12:32.828176 3961 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260923 22:12:32.828356 3961 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260923 22:12:32.833242 3967 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260923 22:12:32.833225 3966 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260923 22:12:32.833350 3969 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260923 22:12:32.833452 3961 server_base.cc:1034] running on GCE node
I20260923 22:12:32.833765 3961 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260923 22:12:32.834048 3961 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260923 22:12:32.835218 3961 hybrid_clock.cc:648] HybridClock initialized: now 1790201552835194 us; error 46 us; skew 500 ppm
Sep 23 22:12:32 dist-test-slave-s2fj krb5kdc[3948](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790201552, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.135.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260923 22:12:32.837654 3961 init.cc:377] Logged in from keytab as kudu/127.1.135.254@KRBTEST.COM (short username kudu)
I20260923 22:12:32.838073 3961 webserver.cc:460] Webserver started at http://127.1.135.254:41681/ using document root <none> and password file <none>
I20260923 22:12:32.838289 3961 fs_manager.cc:359] Metadata directory not provided
I20260923 22:12:32.838353 3961 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260923 22:12:32.838507 3961 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260923 22:12:32.839533 3961 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskRiQe_a/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790201539513534-1567-0/minicluster-data/master-0/data/instance:
uuid: "63caabc44a404d1f8ee83517775b8d74"
format_stamp: "Formatted at 2026-09-23 22:12:32 on dist-test-slave-s2fj"
server_key: "bda6316ca098923c2b5155ee4e9911af"
server_key_iv: "c22428436c80b2ae2b5b6bcc745bc9e1"
server_key_version: "encryptionkey@0"
I20260923 22:12:32.839900 3961 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskRiQe_a/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790201539513534-1567-0/minicluster-data/master-0/wal/instance:
uuid: "63caabc44a404d1f8ee83517775b8d74"
format_stamp: "Formatted at 2026-09-23 22:12:32 on dist-test-slave-s2fj"
server_key: "bda6316ca098923c2b5155ee4e9911af"
server_key_iv: "c22428436c80b2ae2b5b6bcc745bc9e1"
server_key_version: "encryptionkey@0"
I20260923 22:12:32.841259 3961 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.001s
I20260923 22:12:32.841962 3976 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260923 22:12:32.842134 3961 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.001s
I20260923 22:12:32.842231 3961 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskRiQe_a/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790201539513534-1567-0/minicluster-data/master-0/data,/tmp/dist-test-taskRiQe_a/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790201539513534-1567-0/minicluster-data/master-0/wal
uuid: "63caabc44a404d1f8ee83517775b8d74"
format_stamp: "Formatted at 2026-09-23 22:12:32 on dist-test-slave-s2fj"
server_key: "bda6316ca098923c2b5155ee4e9911af"
server_key_iv: "c22428436c80b2ae2b5b6bcc745bc9e1"
server_key_version: "encryptionkey@0"
I20260923 22:12:32.842314 3961 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskRiQe_a/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790201539513534-1567-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskRiQe_a/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790201539513534-1567-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskRiQe_a/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790201539513534-1567-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260923 22:12:32.856169 3961 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260923 22:12:32.857991 3961 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260923 22:12:32.858146 3961 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260923 22:12:32.862171 3961 rpc_server.cc:307] RPC server started. Bound to: 127.1.135.254:36643
I20260923 22:12:32.862202 4028 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.1.135.254:36643 every 8 connection(s)
I20260923 22:12:32.862529 3961 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskRiQe_a/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790201539513534-1567-0/minicluster-data/master-0/data/info.pb
I20260923 22:12:32.863380 4029 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260923 22:12:32.865527 4029 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 63caabc44a404d1f8ee83517775b8d74: Bootstrap starting.
I20260923 22:12:32.866015 4029 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 63caabc44a404d1f8ee83517775b8d74: Neither blocks nor log segments found. Creating new log.
I20260923 22:12:32.866246 4029 log.cc:824] T 00000000000000000000000000000000 P 63caabc44a404d1f8ee83517775b8d74: Log is configured to *not* fsync() on all Append() calls
I20260923 22:12:32.866882 4029 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 63caabc44a404d1f8ee83517775b8d74: No bootstrap required, opened a new log
I20260923 22:12:32.868381 4029 raft_consensus.cc:348] T 00000000000000000000000000000000 P 63caabc44a404d1f8ee83517775b8d74 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "63caabc44a404d1f8ee83517775b8d74" member_type: VOTER last_known_addr { host: "127.1.135.254" port: 36643 } }
I20260923 22:12:32.868510 4029 raft_consensus.cc:374] T 00000000000000000000000000000000 P 63caabc44a404d1f8ee83517775b8d74 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260923 22:12:32.868531 4029 raft_consensus.cc:729] T 00000000000000000000000000000000 P 63caabc44a404d1f8ee83517775b8d74 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 63caabc44a404d1f8ee83517775b8d74, State: Initialized, Role: FOLLOWER
I20260923 22:12:32.868661 4029 consensus_queue.cc:260] T 00000000000000000000000000000000 P 63caabc44a404d1f8ee83517775b8d74 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "63caabc44a404d1f8ee83517775b8d74" member_type: VOTER last_known_addr { host: "127.1.135.254" port: 36643 } }
I20260923 22:12:32.868742 4029 raft_consensus.cc:388] T 00000000000000000000000000000000 P 63caabc44a404d1f8ee83517775b8d74 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260923 22:12:32.868764 4029 raft_consensus.cc:482] T 00000000000000000000000000000000 P 63caabc44a404d1f8ee83517775b8d74 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260923 22:12:32.868796 4029 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 63caabc44a404d1f8ee83517775b8d74 [term 0 FOLLOWER]: Advancing to term 1
I20260923 22:12:32.869246 4029 raft_consensus.cc:504] T 00000000000000000000000000000000 P 63caabc44a404d1f8ee83517775b8d74 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "63caabc44a404d1f8ee83517775b8d74" member_type: VOTER last_known_addr { host: "127.1.135.254" port: 36643 } }
I20260923 22:12:32.869334 4029 leader_election.cc:302] T 00000000000000000000000000000000 P 63caabc44a404d1f8ee83517775b8d74 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 63caabc44a404d1f8ee83517775b8d74; no voters:
I20260923 22:12:32.869457 4029 leader_election.cc:288] T 00000000000000000000000000000000 P 63caabc44a404d1f8ee83517775b8d74 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260923 22:12:32.869505 4032 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 63caabc44a404d1f8ee83517775b8d74 [term 1 FOLLOWER]: Leader election won for term 1
I20260923 22:12:32.869707 4029 sys_catalog.cc:564] T 00000000000000000000000000000000 P 63caabc44a404d1f8ee83517775b8d74 [sys.catalog]: configured and running, proceeding with master startup.
I20260923 22:12:32.869813 4032 raft_consensus.cc:686] T 00000000000000000000000000000000 P 63caabc44a404d1f8ee83517775b8d74 [term 1 LEADER]: Becoming Leader. State: Replica: 63caabc44a404d1f8ee83517775b8d74, State: Running, Role: LEADER
I20260923 22:12:32.869969 4032 consensus_queue.cc:237] T 00000000000000000000000000000000 P 63caabc44a404d1f8ee83517775b8d74 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "63caabc44a404d1f8ee83517775b8d74" member_type: VOTER last_known_addr { host: "127.1.135.254" port: 36643 } }
I20260923 22:12:32.870410 4033 sys_catalog.cc:455] T 00000000000000000000000000000000 P 63caabc44a404d1f8ee83517775b8d74 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "63caabc44a404d1f8ee83517775b8d74" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "63caabc44a404d1f8ee83517775b8d74" member_type: VOTER last_known_addr { host: "127.1.135.254" port: 36643 } } }
I20260923 22:12:32.870419 4034 sys_catalog.cc:455] T 00000000000000000000000000000000 P 63caabc44a404d1f8ee83517775b8d74 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 63caabc44a404d1f8ee83517775b8d74. Latest consensus state: current_term: 1 leader_uuid: "63caabc44a404d1f8ee83517775b8d74" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "63caabc44a404d1f8ee83517775b8d74" member_type: VOTER last_known_addr { host: "127.1.135.254" port: 36643 } } }
I20260923 22:12:32.870549 4033 sys_catalog.cc:458] T 00000000000000000000000000000000 P 63caabc44a404d1f8ee83517775b8d74 [sys.catalog]: This master's current role is: LEADER
I20260923 22:12:32.870556 4034 sys_catalog.cc:458] T 00000000000000000000000000000000 P 63caabc44a404d1f8ee83517775b8d74 [sys.catalog]: This master's current role is: LEADER
I20260923 22:12:32.870738 4041 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260923 22:12:32.871179 1567 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskRiQe_a/build/release/bin/kudu as pid 3961
I20260923 22:12:32.871259 4041 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260923 22:12:32.871299 1567 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskRiQe_a/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790201539513534-1567-0/minicluster-data/master-0/wal/instance
I20260923 22:12:32.871483 1567 external_mini_cluster.cc:1428] Setting key 978c1b468ab2b816017b7fc464b33b85
I20260923 22:12:32.873800 4041 catalog_manager.cc:1347] Generated new cluster ID: 3487ff63af3844348393984d4df4810a
I20260923 22:12:32.873855 4041 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
Sep 23 22:12:32 dist-test-slave-s2fj krb5kdc[3948](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790201552, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.1.135.254@KRBTEST.COM
I20260923 22:12:32.881194 4051 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 22:12:32.871899 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33672 (local address 127.1.135.254:36643)
0923 22:12:32.872086 (+ 187us) server_negotiation.cc:206] Beginning negotiation
0923 22:12:32.872096 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0923 22:12:32.872288 (+ 192us) server_negotiation.cc:407] Connection header received
0923 22:12:32.872431 (+ 143us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 22:12:32.872438 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 22:12:32.872507 (+ 69us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 22:12:32.872651 (+ 144us) server_negotiation.cc:226] Negotiated authn=SASL
0923 22:12:32.873406 (+ 755us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 22:12:32.874439 (+ 1033us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 22:12:32.875205 (+ 766us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 22:12:32.875635 (+ 430us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 22:12:32.875751 (+ 116us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 22:12:32.878036 (+ 2285us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0923 22:12:32.878040 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0923 22:12:32.878040 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0923 22:12:32.878051 (+ 11us) server_negotiation.cc:922] calling sasl_server_start()
0923 22:12:32.879657 (+ 1606us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0923 22:12:32.880022 (+ 365us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0923 22:12:32.880025 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0923 22:12:32.880026 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0923 22:12:32.880171 (+ 145us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0923 22:12:32.880385 (+ 214us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0923 22:12:32.880386 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0923 22:12:32.880386 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0923 22:12:32.880615 (+ 229us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0923 22:12:32.880780 (+ 165us) server_negotiation.cc:1035] Waiting for connection context
0923 22:12:32.880862 (+ 82us) server_negotiation.cc:299] Negotiation successful
0923 22:12:32.880997 (+ 135us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":126,"thread_start_us":74,"threads_started":1}
I20260923 22:12:32.886039 4041 catalog_manager.cc:1370] Generated new certificate authority record
I20260923 22:12:32.886582 4041 catalog_manager.cc:1504] Loading token signing keys...
I20260923 22:12:32.892865 4041 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 63caabc44a404d1f8ee83517775b8d74: Generated new TSK 0
I20260923 22:12:32.893049 4041 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.1.135.193@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.1.135.193@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.1.135.193 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskRiQe_a/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790201539513534-1567-0/krb5kdc/kudu_127.1.135.193.keytab.
Entry for principal kudu/127.1.135.193 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskRiQe_a/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790201539513534-1567-0/krb5kdc/kudu_127.1.135.193.keytab.
I20260923 22:12:32.924675 1567 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskRiQe_a/build/release/bin/kudu
/tmp/dist-test-taskRiQe_a/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskRiQe_a/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790201539513534-1567-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskRiQe_a/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790201539513534-1567-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskRiQe_a/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790201539513534-1567-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskRiQe_a/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790201539513534-1567-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.1.135.193:0
--local_ip_for_outbound_sockets=127.1.135.193
--webserver_interface=127.1.135.193
--webserver_port=0
--tserver_master_addrs=127.1.135.254:36643
--keytab_file=/tmp/dist-test-taskRiQe_a/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790201539513534-1567-0/krb5kdc/kudu_127.1.135.193.keytab
--principal=kudu/127.1.135.193
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.1.135.212:37125
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskRiQe_a/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790201539513534-1567-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskRiQe_a/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790201539513534-1567-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskRiQe_a/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790201539513534-1567-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260923 22:12:33.024008 4055 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260923 22:12:33.024271 4055 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260923 22:12:33.024295 4055 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260923 22:12:33.025471 4055 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260923 22:12:33.025512 4055 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260923 22:12:33.025538 4055 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.1.135.193
I20260923 22:12:33.026949 4055 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.1.135.212:37125
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskRiQe_a/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790201539513534-1567-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskRiQe_a/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790201539513534-1567-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskRiQe_a/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790201539513534-1567-0/krb5kdc/kudu_127.1.135.193.keytab
--principal=kudu/127.1.135.193
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.1.135.193:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskRiQe_a/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790201539513534-1567-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.1.135.193
--webserver_port=0
--tserver_master_addrs=127.1.135.254:36643
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.4055
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.1.135.193
--log_dir=/tmp/dist-test-taskRiQe_a/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790201539513534-1567-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 23 Sep 2026 20:53:37 UTC on ccycloud.quasar-rkviyx.root.comops.site
build id 4668083
I20260923 22:12:33.027187 4055 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260923 22:12:33.027419 4055 file_cache.cc:493] Constructed file cache file cache with capacity 419430
I20260923 22:12:33.032341 4055 server_base.cc:1034] running on GCE node
W20260923 22:12:33.032269 4061 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260923 22:12:33.032291 4060 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260923 22:12:33.032255 4063 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260923 22:12:33.032684 4055 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260923 22:12:33.032955 4055 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260923 22:12:33.034142 4055 hybrid_clock.cc:648] HybridClock initialized: now 1790201553034119 us; error 34 us; skew 500 ppm
Sep 23 22:12:33 dist-test-slave-s2fj krb5kdc[3948](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790201553, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.135.193@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260923 22:12:33.036659 4055 init.cc:377] Logged in from keytab as kudu/127.1.135.193@KRBTEST.COM (short username kudu)
I20260923 22:12:33.037086 4055 webserver.cc:460] Webserver started at http://127.1.135.193:37763/ using document root <none> and password file <none>
I20260923 22:12:33.037308 4055 fs_manager.cc:359] Metadata directory not provided
I20260923 22:12:33.037375 4055 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260923 22:12:33.037518 4055 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260923 22:12:33.038583 4055 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskRiQe_a/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790201539513534-1567-0/minicluster-data/ts-0/data/instance:
uuid: "f40542cdb9f24f58a243ea4564e32e20"
format_stamp: "Formatted at 2026-09-23 22:12:33 on dist-test-slave-s2fj"
server_key: "cb9b029083650fae867d0bc7fcdd2846"
server_key_iv: "8cd6a959534fea7b01216599b657d963"
server_key_version: "encryptionkey@0"
I20260923 22:12:33.038947 4055 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskRiQe_a/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790201539513534-1567-0/minicluster-data/ts-0/wal/instance:
uuid: "f40542cdb9f24f58a243ea4564e32e20"
format_stamp: "Formatted at 2026-09-23 22:12:33 on dist-test-slave-s2fj"
server_key: "cb9b029083650fae867d0bc7fcdd2846"
server_key_iv: "8cd6a959534fea7b01216599b657d963"
server_key_version: "encryptionkey@0"
I20260923 22:12:33.040309 4055 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.001s
I20260923 22:12:33.041062 4070 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260923 22:12:33.041249 4055 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260923 22:12:33.041356 4055 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskRiQe_a/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790201539513534-1567-0/minicluster-data/ts-0/data,/tmp/dist-test-taskRiQe_a/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790201539513534-1567-0/minicluster-data/ts-0/wal
uuid: "f40542cdb9f24f58a243ea4564e32e20"
format_stamp: "Formatted at 2026-09-23 22:12:33 on dist-test-slave-s2fj"
server_key: "cb9b029083650fae867d0bc7fcdd2846"
server_key_iv: "8cd6a959534fea7b01216599b657d963"
server_key_version: "encryptionkey@0"
I20260923 22:12:33.041429 4055 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskRiQe_a/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790201539513534-1567-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskRiQe_a/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790201539513534-1567-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskRiQe_a/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790201539513534-1567-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260923 22:12:33.051846 4055 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260923 22:12:33.053629 4055 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260923 22:12:33.053797 4055 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260923 22:12:33.054013 4055 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260923 22:12:33.054319 4055 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260923 22:12:33.054374 4055 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260923 22:12:33.054409 4055 ts_tablet_manager.cc:610] Registered 0 tablets
I20260923 22:12:33.054437 4055 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260923 22:12:33.060052 4055 rpc_server.cc:307] RPC server started. Bound to: 127.1.135.193:36561
I20260923 22:12:33.060149 4183 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.1.135.193:36561 every 8 connection(s)
I20260923 22:12:33.060485 4055 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskRiQe_a/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790201539513534-1567-0/minicluster-data/ts-0/data/info.pb
Sep 23 22:12:33 dist-test-slave-s2fj krb5kdc[3948](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790201553, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.135.193@KRBTEST.COM for kudu/127.1.135.254@KRBTEST.COM
I20260923 22:12:33.069070 4051 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 22:12:33.061191 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.135.193:44355 (local address 127.1.135.254:36643)
0923 22:12:33.061260 (+ 69us) server_negotiation.cc:206] Beginning negotiation
0923 22:12:33.061263 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0923 22:12:33.061498 (+ 235us) server_negotiation.cc:407] Connection header received
0923 22:12:33.061974 (+ 476us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 22:12:33.061975 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 22:12:33.061987 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 22:12:33.062040 (+ 53us) server_negotiation.cc:226] Negotiated authn=SASL
0923 22:12:33.062907 (+ 867us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 22:12:33.063397 (+ 490us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 22:12:33.064329 (+ 932us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 22:12:33.064527 (+ 198us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 22:12:33.064581 (+ 54us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 22:12:33.066638 (+ 2057us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0923 22:12:33.066639 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0923 22:12:33.066640 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0923 22:12:33.066649 (+ 9us) server_negotiation.cc:922] calling sasl_server_start()
0923 22:12:33.067909 (+ 1260us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0923 22:12:33.068313 (+ 404us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0923 22:12:33.068314 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0923 22:12:33.068314 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0923 22:12:33.068364 (+ 50us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0923 22:12:33.068568 (+ 204us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0923 22:12:33.068569 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0923 22:12:33.068569 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0923 22:12:33.068658 (+ 89us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0923 22:12:33.068703 (+ 45us) server_negotiation.cc:1035] Waiting for connection context
0923 22:12:33.068923 (+ 220us) server_negotiation.cc:299] Negotiation successful
0923 22:12:33.068993 (+ 70us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":38}
I20260923 22:12:33.069190 4186 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 22:12:33.061172 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.135.254:36643 (local address 127.1.135.193:44355)
0923 22:12:33.061364 (+ 192us) negotiation.cc:107] Waiting for socket to connect
0923 22:12:33.061398 (+ 34us) client_negotiation.cc:174] Beginning negotiation
0923 22:12:33.061890 (+ 492us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0923 22:12:33.062098 (+ 208us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0923 22:12:33.062102 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0923 22:12:33.062644 (+ 542us) client_negotiation.cc:189] Negotiated authn=SASL
0923 22:12:33.062829 (+ 185us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 22:12:33.062834 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 22:12:33.063469 (+ 635us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 22:12:33.063470 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 22:12:33.064254 (+ 784us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 22:12:33.064256 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 22:12:33.064596 (+ 340us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 22:12:33.064597 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 22:12:33.064710 (+ 113us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 22:12:33.065216 (+ 506us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0923 22:12:33.065224 (+ 8us) client_negotiation.cc:656] calling sasl_client_start()
0923 22:12:33.066552 (+ 1328us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0923 22:12:33.067978 (+ 1426us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0923 22:12:33.067983 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0923 22:12:33.067985 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0923 22:12:33.068252 (+ 267us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0923 22:12:33.068422 (+ 170us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0923 22:12:33.068423 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0923 22:12:33.068423 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0923 22:12:33.068520 (+ 97us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0923 22:12:33.068706 (+ 186us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0923 22:12:33.068709 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0923 22:12:33.068813 (+ 104us) client_negotiation.cc:769] Sending connection context
0923 22:12:33.068896 (+ 83us) client_negotiation.cc:240] Negotiation successful
0923 22:12:33.069006 (+ 110us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":158,"thread_start_us":111,"threads_started":1}
I20260923 22:12:33.069713 4184 heartbeater.cc:347] Connected to a master server at 127.1.135.254:36643
I20260923 22:12:33.069768 1567 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskRiQe_a/build/release/bin/kudu as pid 4055
I20260923 22:12:33.069837 4184 heartbeater.cc:464] Registering TS with master...
I20260923 22:12:33.069854 1567 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskRiQe_a/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790201539513534-1567-0/minicluster-data/ts-0/wal/instance
I20260923 22:12:33.070039 1567 external_mini_cluster.cc:1428] Setting key e1b128baa94f2584ac5721edd6f7026c
I20260923 22:12:33.070066 4184 heartbeater.cc:511] Master 127.1.135.254:36643 requested a full tablet report, sending...
I20260923 22:12:33.070601 3993 ts_manager.cc:194] Registered new tserver with Master: f40542cdb9f24f58a243ea4564e32e20 (127.1.135.193:36561)
I20260923 22:12:33.071482 3993 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.1.135.193@KRBTEST.COM'} at 127.1.135.193:44355
WARNING: no policy specified for kudu/127.1.135.194@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.1.135.194@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.1.135.194 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskRiQe_a/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790201539513534-1567-0/krb5kdc/kudu_127.1.135.194.keytab.
Entry for principal kudu/127.1.135.194 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskRiQe_a/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790201539513534-1567-0/krb5kdc/kudu_127.1.135.194.keytab.
I20260923 22:12:33.094409 1567 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskRiQe_a/build/release/bin/kudu
/tmp/dist-test-taskRiQe_a/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskRiQe_a/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790201539513534-1567-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskRiQe_a/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790201539513534-1567-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskRiQe_a/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790201539513534-1567-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskRiQe_a/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790201539513534-1567-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.1.135.194:0
--local_ip_for_outbound_sockets=127.1.135.194
--webserver_interface=127.1.135.194
--webserver_port=0
--tserver_master_addrs=127.1.135.254:36643
--keytab_file=/tmp/dist-test-taskRiQe_a/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790201539513534-1567-0/krb5kdc/kudu_127.1.135.194.keytab
--principal=kudu/127.1.135.194
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.1.135.212:37125
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskRiQe_a/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790201539513534-1567-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskRiQe_a/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790201539513534-1567-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskRiQe_a/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790201539513534-1567-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260923 22:12:33.174445 4189 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260923 22:12:33.174631 4189 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260923 22:12:33.174654 4189 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260923 22:12:33.175733 4189 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260923 22:12:33.175769 4189 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260923 22:12:33.175796 4189 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.1.135.194
I20260923 22:12:33.178192 4189 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.1.135.212:37125
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskRiQe_a/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790201539513534-1567-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskRiQe_a/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790201539513534-1567-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskRiQe_a/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790201539513534-1567-0/krb5kdc/kudu_127.1.135.194.keytab
--principal=kudu/127.1.135.194
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.1.135.194:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskRiQe_a/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790201539513534-1567-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.1.135.194
--webserver_port=0
--tserver_master_addrs=127.1.135.254:36643
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.4189
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.1.135.194
--log_dir=/tmp/dist-test-taskRiQe_a/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790201539513534-1567-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 23 Sep 2026 20:53:37 UTC on ccycloud.quasar-rkviyx.root.comops.site
build id 4668083
I20260923 22:12:33.178601 4189 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260923 22:12:33.178865 4189 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260923 22:12:33.185357 4195 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260923 22:12:33.185374 4194 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260923 22:12:33.185521 4197 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260923 22:12:33.186903 4189 server_base.cc:1034] running on GCE node
I20260923 22:12:33.187098 4189 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260923 22:12:33.187325 4189 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260923 22:12:33.188476 4189 hybrid_clock.cc:648] HybridClock initialized: now 1790201553188464 us; error 27 us; skew 500 ppm
Sep 23 22:12:33 dist-test-slave-s2fj krb5kdc[3948](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790201553, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.135.194@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260923 22:12:33.191038 4189 init.cc:377] Logged in from keytab as kudu/127.1.135.194@KRBTEST.COM (short username kudu)
I20260923 22:12:33.191509 4189 webserver.cc:460] Webserver started at http://127.1.135.194:36445/ using document root <none> and password file <none>
I20260923 22:12:33.191737 4189 fs_manager.cc:359] Metadata directory not provided
I20260923 22:12:33.191803 4189 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260923 22:12:33.191967 4189 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260923 22:12:33.193059 4189 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskRiQe_a/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790201539513534-1567-0/minicluster-data/ts-1/data/instance:
uuid: "74eaae0e603243578fc68d9a2d081376"
format_stamp: "Formatted at 2026-09-23 22:12:33 on dist-test-slave-s2fj"
server_key: "f29f03fffc4d87921e4a5e286f83cfe3"
server_key_iv: "15804355beb5cd0c3bb7bd92e2ad1e0a"
server_key_version: "encryptionkey@0"
I20260923 22:12:33.193439 4189 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskRiQe_a/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790201539513534-1567-0/minicluster-data/ts-1/wal/instance:
uuid: "74eaae0e603243578fc68d9a2d081376"
format_stamp: "Formatted at 2026-09-23 22:12:33 on dist-test-slave-s2fj"
server_key: "f29f03fffc4d87921e4a5e286f83cfe3"
server_key_iv: "15804355beb5cd0c3bb7bd92e2ad1e0a"
server_key_version: "encryptionkey@0"
I20260923 22:12:33.194703 4189 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.002s
I20260923 22:12:33.195384 4204 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260923 22:12:33.195550 4189 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.001s
I20260923 22:12:33.195647 4189 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskRiQe_a/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790201539513534-1567-0/minicluster-data/ts-1/data,/tmp/dist-test-taskRiQe_a/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790201539513534-1567-0/minicluster-data/ts-1/wal
uuid: "74eaae0e603243578fc68d9a2d081376"
format_stamp: "Formatted at 2026-09-23 22:12:33 on dist-test-slave-s2fj"
server_key: "f29f03fffc4d87921e4a5e286f83cfe3"
server_key_iv: "15804355beb5cd0c3bb7bd92e2ad1e0a"
server_key_version: "encryptionkey@0"
I20260923 22:12:33.195726 4189 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskRiQe_a/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790201539513534-1567-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskRiQe_a/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790201539513534-1567-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskRiQe_a/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790201539513534-1567-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260923 22:12:33.214143 4189 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260923 22:12:33.216413 4189 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260923 22:12:33.216575 4189 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260923 22:12:33.216813 4189 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260923 22:12:33.217132 4189 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260923 22:12:33.217185 4189 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260923 22:12:33.217226 4189 ts_tablet_manager.cc:610] Registered 0 tablets
I20260923 22:12:33.217254 4189 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260923 22:12:33.222944 4189 rpc_server.cc:307] RPC server started. Bound to: 127.1.135.194:40943
I20260923 22:12:33.222994 4317 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.1.135.194:40943 every 8 connection(s)
I20260923 22:12:33.223328 4189 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskRiQe_a/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790201539513534-1567-0/minicluster-data/ts-1/data/info.pb
I20260923 22:12:33.229008 1567 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskRiQe_a/build/release/bin/kudu as pid 4189
I20260923 22:12:33.229118 1567 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskRiQe_a/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790201539513534-1567-0/minicluster-data/ts-1/wal/instance
Sep 23 22:12:33 dist-test-slave-s2fj krb5kdc[3948](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790201553, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.135.194@KRBTEST.COM for kudu/127.1.135.254@KRBTEST.COM
I20260923 22:12:33.229310 1567 external_mini_cluster.cc:1428] Setting key d8b529d5d667adb83460740245a9e5c9
I20260923 22:12:33.232555 4051 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 22:12:33.224107 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.135.194:56167 (local address 127.1.135.254:36643)
0923 22:12:33.224185 (+ 78us) server_negotiation.cc:206] Beginning negotiation
0923 22:12:33.224188 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0923 22:12:33.224374 (+ 186us) server_negotiation.cc:407] Connection header received
0923 22:12:33.224835 (+ 461us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 22:12:33.224836 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 22:12:33.224846 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 22:12:33.224896 (+ 50us) server_negotiation.cc:226] Negotiated authn=SASL
0923 22:12:33.225740 (+ 844us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 22:12:33.226242 (+ 502us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 22:12:33.227168 (+ 926us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 22:12:33.227369 (+ 201us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 22:12:33.227428 (+ 59us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 22:12:33.229640 (+ 2212us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0923 22:12:33.229641 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0923 22:12:33.229642 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0923 22:12:33.229649 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0923 22:12:33.231185 (+ 1536us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0923 22:12:33.231603 (+ 418us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0923 22:12:33.231604 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0923 22:12:33.231605 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0923 22:12:33.231662 (+ 57us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0923 22:12:33.231901 (+ 239us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0923 22:12:33.231902 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0923 22:12:33.231902 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0923 22:12:33.232024 (+ 122us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0923 22:12:33.232110 (+ 86us) server_negotiation.cc:1035] Waiting for connection context
0923 22:12:33.232384 (+ 274us) server_negotiation.cc:299] Negotiation successful
0923 22:12:33.232471 (+ 87us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":43}
I20260923 22:12:33.232755 4320 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 22:12:33.224054 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.135.254:36643 (local address 127.1.135.194:56167)
0923 22:12:33.224255 (+ 201us) negotiation.cc:107] Waiting for socket to connect
0923 22:12:33.224288 (+ 33us) client_negotiation.cc:174] Beginning negotiation
0923 22:12:33.224748 (+ 460us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0923 22:12:33.224956 (+ 208us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0923 22:12:33.224959 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0923 22:12:33.225487 (+ 528us) client_negotiation.cc:189] Negotiated authn=SASL
0923 22:12:33.225664 (+ 177us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 22:12:33.225668 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 22:12:33.226317 (+ 649us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 22:12:33.226318 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 22:12:33.227099 (+ 781us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 22:12:33.227101 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 22:12:33.227439 (+ 338us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 22:12:33.227440 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 22:12:33.227531 (+ 91us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 22:12:33.228033 (+ 502us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0923 22:12:33.228047 (+ 14us) client_negotiation.cc:656] calling sasl_client_start()
0923 22:12:33.229550 (+ 1503us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0923 22:12:33.231268 (+ 1718us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0923 22:12:33.231273 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0923 22:12:33.231275 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0923 22:12:33.231512 (+ 237us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0923 22:12:33.231731 (+ 219us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0923 22:12:33.231732 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0923 22:12:33.231733 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0923 22:12:33.231834 (+ 101us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0923 22:12:33.232100 (+ 266us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0923 22:12:33.232105 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0923 22:12:33.232250 (+ 145us) client_negotiation.cc:769] Sending connection context
0923 22:12:33.232351 (+ 101us) client_negotiation.cc:240] Negotiation successful
0923 22:12:33.232494 (+ 143us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":163,"thread_start_us":120,"threads_started":1}
I20260923 22:12:33.233167 4318 heartbeater.cc:347] Connected to a master server at 127.1.135.254:36643
I20260923 22:12:33.233269 4318 heartbeater.cc:464] Registering TS with master...
I20260923 22:12:33.233440 4318 heartbeater.cc:511] Master 127.1.135.254:36643 requested a full tablet report, sending...
I20260923 22:12:33.233821 3993 ts_manager.cc:194] Registered new tserver with Master: 74eaae0e603243578fc68d9a2d081376 (127.1.135.194:40943)
I20260923 22:12:33.234365 3993 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.1.135.194@KRBTEST.COM'} at 127.1.135.194:56167
WARNING: no policy specified for kudu/127.1.135.195@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.1.135.195@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.1.135.195 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskRiQe_a/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790201539513534-1567-0/krb5kdc/kudu_127.1.135.195.keytab.
Entry for principal kudu/127.1.135.195 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskRiQe_a/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790201539513534-1567-0/krb5kdc/kudu_127.1.135.195.keytab.
I20260923 22:12:33.255581 1567 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskRiQe_a/build/release/bin/kudu
/tmp/dist-test-taskRiQe_a/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskRiQe_a/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790201539513534-1567-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskRiQe_a/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790201539513534-1567-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskRiQe_a/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790201539513534-1567-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskRiQe_a/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790201539513534-1567-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.1.135.195:0
--local_ip_for_outbound_sockets=127.1.135.195
--webserver_interface=127.1.135.195
--webserver_port=0
--tserver_master_addrs=127.1.135.254:36643
--keytab_file=/tmp/dist-test-taskRiQe_a/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790201539513534-1567-0/krb5kdc/kudu_127.1.135.195.keytab
--principal=kudu/127.1.135.195
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.1.135.212:37125
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskRiQe_a/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790201539513534-1567-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskRiQe_a/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790201539513534-1567-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskRiQe_a/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790201539513534-1567-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260923 22:12:33.335368 4323 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260923 22:12:33.335561 4323 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260923 22:12:33.335583 4323 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260923 22:12:33.336766 4323 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260923 22:12:33.336807 4323 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260923 22:12:33.336833 4323 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.1.135.195
I20260923 22:12:33.338169 4323 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.1.135.212:37125
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskRiQe_a/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790201539513534-1567-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskRiQe_a/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790201539513534-1567-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskRiQe_a/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790201539513534-1567-0/krb5kdc/kudu_127.1.135.195.keytab
--principal=kudu/127.1.135.195
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.1.135.195:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskRiQe_a/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790201539513534-1567-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.1.135.195
--webserver_port=0
--tserver_master_addrs=127.1.135.254:36643
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.4323
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.1.135.195
--log_dir=/tmp/dist-test-taskRiQe_a/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790201539513534-1567-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 23 Sep 2026 20:53:37 UTC on ccycloud.quasar-rkviyx.root.comops.site
build id 4668083
I20260923 22:12:33.338335 4323 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260923 22:12:33.338495 4323 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260923 22:12:33.343446 4329 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260923 22:12:33.343422 4331 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260923 22:12:33.343587 4323 server_base.cc:1034] running on GCE node
W20260923 22:12:33.343430 4328 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260923 22:12:33.343945 4323 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260923 22:12:33.344211 4323 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260923 22:12:33.345399 4323 hybrid_clock.cc:648] HybridClock initialized: now 1790201553345386 us; error 35 us; skew 500 ppm
Sep 23 22:12:33 dist-test-slave-s2fj krb5kdc[3948](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790201553, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.135.195@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260923 22:12:33.347697 4323 init.cc:377] Logged in from keytab as kudu/127.1.135.195@KRBTEST.COM (short username kudu)
I20260923 22:12:33.348171 4323 webserver.cc:460] Webserver started at http://127.1.135.195:40299/ using document root <none> and password file <none>
I20260923 22:12:33.348387 4323 fs_manager.cc:359] Metadata directory not provided
I20260923 22:12:33.348453 4323 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260923 22:12:33.348608 4323 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260923 22:12:33.349608 4323 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskRiQe_a/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790201539513534-1567-0/minicluster-data/ts-2/data/instance:
uuid: "1d6b2a4610a34da385052445fac74550"
format_stamp: "Formatted at 2026-09-23 22:12:33 on dist-test-slave-s2fj"
server_key: "b1ba1d858c6e4009a713a00734c70b3c"
server_key_iv: "797fb590e68e9c9b062e823ef69872ae"
server_key_version: "encryptionkey@0"
I20260923 22:12:33.349973 4323 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskRiQe_a/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790201539513534-1567-0/minicluster-data/ts-2/wal/instance:
uuid: "1d6b2a4610a34da385052445fac74550"
format_stamp: "Formatted at 2026-09-23 22:12:33 on dist-test-slave-s2fj"
server_key: "b1ba1d858c6e4009a713a00734c70b3c"
server_key_iv: "797fb590e68e9c9b062e823ef69872ae"
server_key_version: "encryptionkey@0"
I20260923 22:12:33.351289 4323 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.001s
I20260923 22:12:33.352137 4338 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260923 22:12:33.352319 4323 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260923 22:12:33.352414 4323 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskRiQe_a/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790201539513534-1567-0/minicluster-data/ts-2/data,/tmp/dist-test-taskRiQe_a/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790201539513534-1567-0/minicluster-data/ts-2/wal
uuid: "1d6b2a4610a34da385052445fac74550"
format_stamp: "Formatted at 2026-09-23 22:12:33 on dist-test-slave-s2fj"
server_key: "b1ba1d858c6e4009a713a00734c70b3c"
server_key_iv: "797fb590e68e9c9b062e823ef69872ae"
server_key_version: "encryptionkey@0"
I20260923 22:12:33.352499 4323 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskRiQe_a/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790201539513534-1567-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskRiQe_a/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790201539513534-1567-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskRiQe_a/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790201539513534-1567-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260923 22:12:33.363483 4323 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260923 22:12:33.365410 4323 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260923 22:12:33.365564 4323 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260923 22:12:33.365782 4323 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260923 22:12:33.366081 4323 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260923 22:12:33.366135 4323 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260923 22:12:33.366176 4323 ts_tablet_manager.cc:610] Registered 0 tablets
I20260923 22:12:33.366205 4323 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260923 22:12:33.372448 4323 rpc_server.cc:307] RPC server started. Bound to: 127.1.135.195:43417
I20260923 22:12:33.372489 4451 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.1.135.195:43417 every 8 connection(s)
I20260923 22:12:33.372788 4323 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskRiQe_a/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790201539513534-1567-0/minicluster-data/ts-2/data/info.pb
Sep 23 22:12:33 dist-test-slave-s2fj krb5kdc[3948](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790201553, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.135.195@KRBTEST.COM for kudu/127.1.135.254@KRBTEST.COM
I20260923 22:12:33.380283 1567 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskRiQe_a/build/release/bin/kudu as pid 4323
I20260923 22:12:33.380383 1567 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskRiQe_a/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790201539513534-1567-0/minicluster-data/ts-2/wal/instance
I20260923 22:12:33.380652 1567 external_mini_cluster.cc:1428] Setting key 9b9037afa6446a238d398a2d1eed2116
I20260923 22:12:33.381455 4051 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 22:12:33.373432 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.135.195:46131 (local address 127.1.135.254:36643)
0923 22:12:33.373498 (+ 66us) server_negotiation.cc:206] Beginning negotiation
0923 22:12:33.373501 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0923 22:12:33.373725 (+ 224us) server_negotiation.cc:407] Connection header received
0923 22:12:33.374167 (+ 442us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 22:12:33.374168 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 22:12:33.374179 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 22:12:33.374237 (+ 58us) server_negotiation.cc:226] Negotiated authn=SASL
0923 22:12:33.375067 (+ 830us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 22:12:33.375561 (+ 494us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 22:12:33.376574 (+ 1013us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 22:12:33.376762 (+ 188us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 22:12:33.376818 (+ 56us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 22:12:33.378754 (+ 1936us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0923 22:12:33.378756 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0923 22:12:33.378756 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0923 22:12:33.378762 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0923 22:12:33.379884 (+ 1122us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0923 22:12:33.380370 (+ 486us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0923 22:12:33.380371 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0923 22:12:33.380371 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0923 22:12:33.380452 (+ 81us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0923 22:12:33.380716 (+ 264us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0923 22:12:33.380717 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0923 22:12:33.380717 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0923 22:12:33.380880 (+ 163us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0923 22:12:33.380942 (+ 62us) server_negotiation.cc:1035] Waiting for connection context
0923 22:12:33.381282 (+ 340us) server_negotiation.cc:299] Negotiation successful
0923 22:12:33.381363 (+ 81us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":36}
I20260923 22:12:33.381651 4454 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 22:12:33.373430 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.135.254:36643 (local address 127.1.135.195:46131)
0923 22:12:33.373604 (+ 174us) negotiation.cc:107] Waiting for socket to connect
0923 22:12:33.373637 (+ 33us) client_negotiation.cc:174] Beginning negotiation
0923 22:12:33.374091 (+ 454us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0923 22:12:33.374289 (+ 198us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0923 22:12:33.374292 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0923 22:12:33.374820 (+ 528us) client_negotiation.cc:189] Negotiated authn=SASL
0923 22:12:33.374997 (+ 177us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 22:12:33.375002 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 22:12:33.375636 (+ 634us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 22:12:33.375636 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 22:12:33.376501 (+ 865us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 22:12:33.376503 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 22:12:33.376835 (+ 332us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 22:12:33.376836 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 22:12:33.376928 (+ 92us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 22:12:33.377337 (+ 409us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0923 22:12:33.377345 (+ 8us) client_negotiation.cc:656] calling sasl_client_start()
0923 22:12:33.378659 (+ 1314us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0923 22:12:33.379953 (+ 1294us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0923 22:12:33.379959 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0923 22:12:33.379960 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0923 22:12:33.380296 (+ 336us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0923 22:12:33.380540 (+ 244us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0923 22:12:33.380541 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0923 22:12:33.380541 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0923 22:12:33.380648 (+ 107us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0923 22:12:33.380932 (+ 284us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0923 22:12:33.380939 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0923 22:12:33.381098 (+ 159us) client_negotiation.cc:769] Sending connection context
0923 22:12:33.381244 (+ 146us) client_negotiation.cc:240] Negotiation successful
0923 22:12:33.381426 (+ 182us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":139,"thread_start_us":105,"threads_started":1}
I20260923 22:12:33.382087 4452 heartbeater.cc:347] Connected to a master server at 127.1.135.254:36643
I20260923 22:12:33.382189 4452 heartbeater.cc:464] Registering TS with master...
I20260923 22:12:33.382365 4452 heartbeater.cc:511] Master 127.1.135.254:36643 requested a full tablet report, sending...
I20260923 22:12:33.382678 3993 ts_manager.cc:194] Registered new tserver with Master: 1d6b2a4610a34da385052445fac74550 (127.1.135.195:43417)
I20260923 22:12:33.383028 3993 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.1.135.195@KRBTEST.COM'} at 127.1.135.195:46131
I20260923 22:12:33.392549 1567 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
W20260923 22:12:33.403131 4051 negotiation.cc:337] Failed RPC negotiation. Trace:
0923 22:12:33.402180 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33686 (local address 127.1.135.254:36643)
0923 22:12:33.402270 (+ 90us) server_negotiation.cc:206] Beginning negotiation
0923 22:12:33.402272 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0923 22:12:33.402365 (+ 93us) server_negotiation.cc:407] Connection header received
0923 22:12:33.402445 (+ 80us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 22:12:33.402446 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 22:12:33.402455 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 22:12:33.402527 (+ 72us) server_negotiation.cc:226] Negotiated authn=SASL
0923 22:12:33.403051 (+ 524us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:33686: BlockingRecv error: recv got EOF from 127.0.0.1:33686 (error 108)
Metrics: {"server-negotiator.queue_time_us":64}
I20260923 22:12:33.403971 1567 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskRiQe_a/build/release/bin/kudu with pid 4055
I20260923 22:12:33.408445 1567 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskRiQe_a/build/release/bin/kudu with pid 4189
I20260923 22:12:33.412616 1567 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskRiQe_a/build/release/bin/kudu with pid 4323
I20260923 22:12:33.416450 1567 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskRiQe_a/build/release/bin/kudu with pid 3961
2026-09-23T22:12:33Z chronyd exiting
[ OK ] SecurityITest.TestNoKerberosCredentials (1816 ms)
[ RUN ] SecurityITest.SaslPlainFallback
Loading random data
Initializing database '/tmp/dist-test-taskRiQe_a/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1790201539513534-1567-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Sep 23 22:12:33 dist-test-slave-s2fj krb5kdc[4464](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Sep 23 22:12:33 dist-test-slave-s2fj krb5kdc[4464](info): set up 2 sockets
Sep 23 22:12:33 dist-test-slave-s2fj krb5kdc[4464](info): commencing operation
krb5kdc: starting...
W20260923 22:12:34.427948 1567 mini_kdc.cc:121] Time spent starting KDC: real 0.992s user 0.000s sys 0.005s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskRiQe_a/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1790201539513534-1567-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskRiQe_a/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1790201539513534-1567-0/krb5kdc/test-user.keytab.
Sep 23 22:12:34 dist-test-slave-s2fj krb5kdc[4464](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790201554, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-09-23T22:12:34Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-09-23T22:12:34Z Disabled control of system clock
WARNING: no policy specified for kudu/127.1.135.254@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.1.135.254@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.1.135.254 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskRiQe_a/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1790201539513534-1567-0/krb5kdc/kudu_127.1.135.254.keytab.
Entry for principal kudu/127.1.135.254 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskRiQe_a/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1790201539513534-1567-0/krb5kdc/kudu_127.1.135.254.keytab.
I20260923 22:12:34.542822 1567 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskRiQe_a/build/release/bin/kudu
/tmp/dist-test-taskRiQe_a/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskRiQe_a/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1790201539513534-1567-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskRiQe_a/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1790201539513534-1567-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskRiQe_a/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1790201539513534-1567-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskRiQe_a/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1790201539513534-1567-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.1.135.254:43999
--webserver_interface=127.1.135.254
--webserver_port=0
--keytab_file=/tmp/dist-test-taskRiQe_a/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1790201539513534-1567-0/krb5kdc/kudu_127.1.135.254.keytab
--principal=kudu/127.1.135.254
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.1.135.212:35569
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.1.135.254:43999
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--rpc-authentication=optional
--user-acl=* with env {KRB5CCNAME=/tmp/dist-test-taskRiQe_a/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1790201539513534-1567-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskRiQe_a/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1790201539513534-1567-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskRiQe_a/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1790201539513534-1567-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260923 22:12:34.620558 4477 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260923 22:12:34.620775 4477 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260923 22:12:34.620810 4477 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260923 22:12:34.621958 4477 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260923 22:12:34.622016 4477 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260923 22:12:34.622040 4477 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260923 22:12:34.622068 4477 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260923 22:12:34.622090 4477 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260923 22:12:34.623466 4477 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.1.135.212:35569
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskRiQe_a/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1790201539513534-1567-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskRiQe_a/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1790201539513534-1567-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.1.135.254:43999
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskRiQe_a/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1790201539513534-1567-0/krb5kdc/kudu_127.1.135.254.keytab
--principal=kudu/127.1.135.254
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.1.135.254:43999
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskRiQe_a/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1790201539513534-1567-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.1.135.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.4477
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskRiQe_a/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1790201539513534-1567-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 23 Sep 2026 20:53:37 UTC on ccycloud.quasar-rkviyx.root.comops.site
build id 4668083
I20260923 22:12:34.623720 4477 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260923 22:12:34.623948 4477 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260923 22:12:34.628595 4482 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260923 22:12:34.628619 4483 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260923 22:12:34.628654 4485 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260923 22:12:34.628805 4477 server_base.cc:1034] running on GCE node
I20260923 22:12:34.629086 4477 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260923 22:12:34.629354 4477 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260923 22:12:34.630502 4477 hybrid_clock.cc:648] HybridClock initialized: now 1790201554630488 us; error 28 us; skew 500 ppm
Sep 23 22:12:34 dist-test-slave-s2fj krb5kdc[4464](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790201554, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.135.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260923 22:12:34.633008 4477 init.cc:377] Logged in from keytab as kudu/127.1.135.254@KRBTEST.COM (short username kudu)
I20260923 22:12:34.633435 4477 webserver.cc:460] Webserver started at http://127.1.135.254:41997/ using document root <none> and password file <none>
I20260923 22:12:34.633657 4477 fs_manager.cc:359] Metadata directory not provided
I20260923 22:12:34.633701 4477 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260923 22:12:34.633826 4477 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260923 22:12:34.634791 4477 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskRiQe_a/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1790201539513534-1567-0/minicluster-data/master-0/data/instance:
uuid: "5c1842d1ac404e56a766c726be6e43dc"
format_stamp: "Formatted at 2026-09-23 22:12:34 on dist-test-slave-s2fj"
server_key: "6fe2e834adafea30f04278f1d3dd33d4"
server_key_iv: "2e1e984380649efd0f9805dcb264bca4"
server_key_version: "encryptionkey@0"
I20260923 22:12:34.635118 4477 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskRiQe_a/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1790201539513534-1567-0/minicluster-data/master-0/wal/instance:
uuid: "5c1842d1ac404e56a766c726be6e43dc"
format_stamp: "Formatted at 2026-09-23 22:12:34 on dist-test-slave-s2fj"
server_key: "6fe2e834adafea30f04278f1d3dd33d4"
server_key_iv: "2e1e984380649efd0f9805dcb264bca4"
server_key_version: "encryptionkey@0"
I20260923 22:12:34.636432 4477 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.003s
I20260923 22:12:34.637100 4492 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260923 22:12:34.637282 4477 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.000s
I20260923 22:12:34.637367 4477 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskRiQe_a/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1790201539513534-1567-0/minicluster-data/master-0/data,/tmp/dist-test-taskRiQe_a/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1790201539513534-1567-0/minicluster-data/master-0/wal
uuid: "5c1842d1ac404e56a766c726be6e43dc"
format_stamp: "Formatted at 2026-09-23 22:12:34 on dist-test-slave-s2fj"
server_key: "6fe2e834adafea30f04278f1d3dd33d4"
server_key_iv: "2e1e984380649efd0f9805dcb264bca4"
server_key_version: "encryptionkey@0"
I20260923 22:12:34.637437 4477 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskRiQe_a/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1790201539513534-1567-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskRiQe_a/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1790201539513534-1567-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskRiQe_a/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1790201539513534-1567-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260923 22:12:34.643775 4477 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260923 22:12:34.645896 4477 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260923 22:12:34.646049 4477 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260923 22:12:34.649840 4477 rpc_server.cc:307] RPC server started. Bound to: 127.1.135.254:43999
I20260923 22:12:34.649868 4544 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.1.135.254:43999 every 8 connection(s)
I20260923 22:12:34.650170 4477 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskRiQe_a/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1790201539513534-1567-0/minicluster-data/master-0/data/info.pb
I20260923 22:12:34.650883 4545 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260923 22:12:34.653035 4545 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 5c1842d1ac404e56a766c726be6e43dc: Bootstrap starting.
I20260923 22:12:34.653570 4545 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 5c1842d1ac404e56a766c726be6e43dc: Neither blocks nor log segments found. Creating new log.
I20260923 22:12:34.653870 4545 log.cc:824] T 00000000000000000000000000000000 P 5c1842d1ac404e56a766c726be6e43dc: Log is configured to *not* fsync() on all Append() calls
I20260923 22:12:34.654484 4545 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 5c1842d1ac404e56a766c726be6e43dc: No bootstrap required, opened a new log
I20260923 22:12:34.655956 4545 raft_consensus.cc:348] T 00000000000000000000000000000000 P 5c1842d1ac404e56a766c726be6e43dc [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5c1842d1ac404e56a766c726be6e43dc" member_type: VOTER last_known_addr { host: "127.1.135.254" port: 43999 } }
I20260923 22:12:34.656141 4545 raft_consensus.cc:374] T 00000000000000000000000000000000 P 5c1842d1ac404e56a766c726be6e43dc [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260923 22:12:34.656193 4545 raft_consensus.cc:729] T 00000000000000000000000000000000 P 5c1842d1ac404e56a766c726be6e43dc [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5c1842d1ac404e56a766c726be6e43dc, State: Initialized, Role: FOLLOWER
I20260923 22:12:34.656342 4545 consensus_queue.cc:260] T 00000000000000000000000000000000 P 5c1842d1ac404e56a766c726be6e43dc [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5c1842d1ac404e56a766c726be6e43dc" member_type: VOTER last_known_addr { host: "127.1.135.254" port: 43999 } }
I20260923 22:12:34.656402 4545 raft_consensus.cc:388] T 00000000000000000000000000000000 P 5c1842d1ac404e56a766c726be6e43dc [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260923 22:12:34.656423 4545 raft_consensus.cc:482] T 00000000000000000000000000000000 P 5c1842d1ac404e56a766c726be6e43dc [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260923 22:12:34.656481 4545 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 5c1842d1ac404e56a766c726be6e43dc [term 0 FOLLOWER]: Advancing to term 1
I20260923 22:12:34.656788 1567 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskRiQe_a/build/release/bin/kudu as pid 4477
I20260923 22:12:34.656859 1567 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskRiQe_a/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1790201539513534-1567-0/minicluster-data/master-0/wal/instance
I20260923 22:12:34.657028 1567 external_mini_cluster.cc:1428] Setting key 45c8c21e8785c01ada6852dbf9f719fe
I20260923 22:12:34.657358 4545 raft_consensus.cc:504] T 00000000000000000000000000000000 P 5c1842d1ac404e56a766c726be6e43dc [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5c1842d1ac404e56a766c726be6e43dc" member_type: VOTER last_known_addr { host: "127.1.135.254" port: 43999 } }
I20260923 22:12:34.657451 4545 leader_election.cc:302] T 00000000000000000000000000000000 P 5c1842d1ac404e56a766c726be6e43dc [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 5c1842d1ac404e56a766c726be6e43dc; no voters:
I20260923 22:12:34.657603 4545 leader_election.cc:288] T 00000000000000000000000000000000 P 5c1842d1ac404e56a766c726be6e43dc [CANDIDATE]: Term 1 election: Requested vote from peers
I20260923 22:12:34.657639 4550 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 5c1842d1ac404e56a766c726be6e43dc [term 1 FOLLOWER]: Leader election won for term 1
I20260923 22:12:34.657835 4550 raft_consensus.cc:686] T 00000000000000000000000000000000 P 5c1842d1ac404e56a766c726be6e43dc [term 1 LEADER]: Becoming Leader. State: Replica: 5c1842d1ac404e56a766c726be6e43dc, State: Running, Role: LEADER
I20260923 22:12:34.657851 4545 sys_catalog.cc:564] T 00000000000000000000000000000000 P 5c1842d1ac404e56a766c726be6e43dc [sys.catalog]: configured and running, proceeding with master startup.
I20260923 22:12:34.659060 4550 consensus_queue.cc:237] T 00000000000000000000000000000000 P 5c1842d1ac404e56a766c726be6e43dc [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5c1842d1ac404e56a766c726be6e43dc" member_type: VOTER last_known_addr { host: "127.1.135.254" port: 43999 } }
I20260923 22:12:34.659583 4551 sys_catalog.cc:455] T 00000000000000000000000000000000 P 5c1842d1ac404e56a766c726be6e43dc [sys.catalog]: SysCatalogTable state changed. Reason: New leader 5c1842d1ac404e56a766c726be6e43dc. Latest consensus state: current_term: 1 leader_uuid: "5c1842d1ac404e56a766c726be6e43dc" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5c1842d1ac404e56a766c726be6e43dc" member_type: VOTER last_known_addr { host: "127.1.135.254" port: 43999 } } }
I20260923 22:12:34.659725 4552 sys_catalog.cc:455] T 00000000000000000000000000000000 P 5c1842d1ac404e56a766c726be6e43dc [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "5c1842d1ac404e56a766c726be6e43dc" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5c1842d1ac404e56a766c726be6e43dc" member_type: VOTER last_known_addr { host: "127.1.135.254" port: 43999 } } }
I20260923 22:12:34.659868 4551 sys_catalog.cc:458] T 00000000000000000000000000000000 P 5c1842d1ac404e56a766c726be6e43dc [sys.catalog]: This master's current role is: LEADER
I20260923 22:12:34.659881 4552 sys_catalog.cc:458] T 00000000000000000000000000000000 P 5c1842d1ac404e56a766c726be6e43dc [sys.catalog]: This master's current role is: LEADER
W20260923 22:12:34.660415 4564 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 5c1842d1ac404e56a766c726be6e43dc: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260923 22:12:34.660472 4564 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260923 22:12:34.660524 4566 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260923 22:12:34.660672 4566 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260923 22:12:34.662127 4566 catalog_manager.cc:1347] Generated new cluster ID: d2c6b5f69b9543d4854c9b725ec3725a
I20260923 22:12:34.662175 4566 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
Sep 23 22:12:34 dist-test-slave-s2fj krb5kdc[4464](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790201554, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.1.135.254@KRBTEST.COM
I20260923 22:12:34.668223 4549 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 22:12:34.657451 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45888 (local address 127.1.135.254:43999)
0923 22:12:34.657611 (+ 160us) server_negotiation.cc:206] Beginning negotiation
0923 22:12:34.657619 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0923 22:12:34.657749 (+ 130us) server_negotiation.cc:407] Connection header received
0923 22:12:34.657869 (+ 120us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 22:12:34.657877 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 22:12:34.657932 (+ 55us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 22:12:34.658812 (+ 880us) server_negotiation.cc:226] Negotiated authn=SASL
0923 22:12:34.658903 (+ 91us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 22:12:34.660320 (+ 1417us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 22:12:34.661357 (+ 1037us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 22:12:34.661918 (+ 561us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 22:12:34.662026 (+ 108us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 22:12:34.664598 (+ 2572us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0923 22:12:34.664602 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0923 22:12:34.664602 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0923 22:12:34.664613 (+ 11us) server_negotiation.cc:922] calling sasl_server_start()
0923 22:12:34.666636 (+ 2023us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0923 22:12:34.667019 (+ 383us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0923 22:12:34.667022 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0923 22:12:34.667023 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0923 22:12:34.667140 (+ 117us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0923 22:12:34.667337 (+ 197us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0923 22:12:34.667338 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0923 22:12:34.667338 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0923 22:12:34.667624 (+ 286us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0923 22:12:34.667786 (+ 162us) server_negotiation.cc:1035] Waiting for connection context
0923 22:12:34.667864 (+ 78us) server_negotiation.cc:299] Negotiation successful
0923 22:12:34.667993 (+ 129us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":112,"thread_start_us":82,"threads_started":1}
I20260923 22:12:34.669168 1567 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260923 22:12:34.675315 4566 catalog_manager.cc:1370] Generated new certificate authority record
I20260923 22:12:34.675801 4566 catalog_manager.cc:1504] Loading token signing keys...
I20260923 22:12:34.684717 4549 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 22:12:34.680718 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45896 (local address 127.1.135.254:43999)
0923 22:12:34.680786 (+ 68us) server_negotiation.cc:206] Beginning negotiation
0923 22:12:34.680787 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0923 22:12:34.680930 (+ 143us) server_negotiation.cc:407] Connection header received
0923 22:12:34.680979 (+ 49us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 22:12:34.680979 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 22:12:34.680990 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 22:12:34.681041 (+ 51us) server_negotiation.cc:226] Negotiated authn=SASL
0923 22:12:34.681403 (+ 362us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 22:12:34.682152 (+ 749us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 22:12:34.683163 (+ 1011us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 22:12:34.683498 (+ 335us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 22:12:34.683566 (+ 68us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 22:12:34.684261 (+ 695us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0923 22:12:34.684263 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0923 22:12:34.684263 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0923 22:12:34.684457 (+ 194us) server_negotiation.cc:922] calling sasl_server_start()
0923 22:12:34.684472 (+ 15us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0923 22:12:34.684479 (+ 7us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0923 22:12:34.684539 (+ 60us) server_negotiation.cc:1035] Waiting for connection context
0923 22:12:34.684599 (+ 60us) server_negotiation.cc:299] Negotiation successful
0923 22:12:34.684619 (+ 20us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":27}
I20260923 22:12:34.687722 4566 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 5c1842d1ac404e56a766c726be6e43dc: Generated new TSK 0
I20260923 22:12:34.687892 4566 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260923 22:12:34.690449 1567 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskRiQe_a/build/release/bin/kudu with pid 4477
2026-09-23T22:12:34Z chronyd exiting
[ OK ] SecurityITest.SaslPlainFallback (1272 ms)
[ RUN ] SecurityITest.TestUnauthorizedClientKerberosCredentials
Loading random data
Initializing database '/tmp/dist-test-taskRiQe_a/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790201539513534-1567-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Sep 23 22:12:34 dist-test-slave-s2fj krb5kdc[4578](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Sep 23 22:12:34 dist-test-slave-s2fj krb5kdc[4578](info): set up 2 sockets
Sep 23 22:12:34 dist-test-slave-s2fj krb5kdc[4578](info): commencing operation
krb5kdc: starting...
W20260923 22:12:35.719739 1567 mini_kdc.cc:121] Time spent starting KDC: real 1.011s user 0.002s sys 0.003s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskRiQe_a/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790201539513534-1567-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskRiQe_a/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790201539513534-1567-0/krb5kdc/test-user.keytab.
Sep 23 22:12:35 dist-test-slave-s2fj krb5kdc[4578](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790201555, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-09-23T22:12:35Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-09-23T22:12:35Z Disabled control of system clock
WARNING: no policy specified for kudu/127.1.135.254@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.1.135.254@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.1.135.254 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskRiQe_a/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790201539513534-1567-0/krb5kdc/kudu_127.1.135.254.keytab.
Entry for principal kudu/127.1.135.254 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskRiQe_a/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790201539513534-1567-0/krb5kdc/kudu_127.1.135.254.keytab.
I20260923 22:12:35.827099 1567 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskRiQe_a/build/release/bin/kudu
/tmp/dist-test-taskRiQe_a/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskRiQe_a/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790201539513534-1567-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskRiQe_a/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790201539513534-1567-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskRiQe_a/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790201539513534-1567-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskRiQe_a/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790201539513534-1567-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.1.135.254:38865
--webserver_interface=127.1.135.254
--webserver_port=0
--keytab_file=/tmp/dist-test-taskRiQe_a/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790201539513534-1567-0/krb5kdc/kudu_127.1.135.254.keytab
--principal=kudu/127.1.135.254
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.1.135.212:36457
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.1.135.254:38865
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskRiQe_a/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790201539513534-1567-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskRiQe_a/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790201539513534-1567-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskRiQe_a/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790201539513534-1567-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260923 22:12:35.903115 4591 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260923 22:12:35.903343 4591 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260923 22:12:35.903383 4591 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260923 22:12:35.904553 4591 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260923 22:12:35.904610 4591 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260923 22:12:35.904654 4591 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260923 22:12:35.904703 4591 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260923 22:12:35.904727 4591 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260923 22:12:35.906137 4591 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.1.135.212:36457
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskRiQe_a/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790201539513534-1567-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskRiQe_a/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790201539513534-1567-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.1.135.254:38865
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskRiQe_a/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790201539513534-1567-0/krb5kdc/kudu_127.1.135.254.keytab
--principal=kudu/127.1.135.254
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.1.135.254:38865
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskRiQe_a/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790201539513534-1567-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.1.135.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.4591
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskRiQe_a/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790201539513534-1567-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 23 Sep 2026 20:53:37 UTC on ccycloud.quasar-rkviyx.root.comops.site
build id 4668083
I20260923 22:12:35.906404 4591 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260923 22:12:35.906620 4591 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260923 22:12:35.911123 4599 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260923 22:12:35.911170 4596 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260923 22:12:35.911135 4597 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260923 22:12:35.911299 4591 server_base.cc:1034] running on GCE node
I20260923 22:12:35.911610 4591 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260923 22:12:35.911896 4591 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260923 22:12:35.913066 4591 hybrid_clock.cc:648] HybridClock initialized: now 1790201555913036 us; error 46 us; skew 500 ppm
Sep 23 22:12:35 dist-test-slave-s2fj krb5kdc[4578](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790201555, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.135.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260923 22:12:35.915448 4591 init.cc:377] Logged in from keytab as kudu/127.1.135.254@KRBTEST.COM (short username kudu)
I20260923 22:12:35.915946 4591 webserver.cc:460] Webserver started at http://127.1.135.254:43711/ using document root <none> and password file <none>
I20260923 22:12:35.916208 4591 fs_manager.cc:359] Metadata directory not provided
I20260923 22:12:35.916280 4591 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260923 22:12:35.916414 4591 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260923 22:12:35.917455 4591 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskRiQe_a/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790201539513534-1567-0/minicluster-data/master-0/data/instance:
uuid: "9f76cdfc09604a399fecb97401673b76"
format_stamp: "Formatted at 2026-09-23 22:12:35 on dist-test-slave-s2fj"
server_key: "f68a8435bf25c6563c0c0e541f650462"
server_key_iv: "c804bdd3dfc9e9f4b7db0472d27f1a01"
server_key_version: "encryptionkey@0"
I20260923 22:12:35.917829 4591 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskRiQe_a/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790201539513534-1567-0/minicluster-data/master-0/wal/instance:
uuid: "9f76cdfc09604a399fecb97401673b76"
format_stamp: "Formatted at 2026-09-23 22:12:35 on dist-test-slave-s2fj"
server_key: "f68a8435bf25c6563c0c0e541f650462"
server_key_iv: "c804bdd3dfc9e9f4b7db0472d27f1a01"
server_key_version: "encryptionkey@0"
I20260923 22:12:35.919092 4591 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260923 22:12:35.919808 4606 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260923 22:12:35.920007 4591 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260923 22:12:35.920121 4591 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskRiQe_a/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790201539513534-1567-0/minicluster-data/master-0/data,/tmp/dist-test-taskRiQe_a/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790201539513534-1567-0/minicluster-data/master-0/wal
uuid: "9f76cdfc09604a399fecb97401673b76"
format_stamp: "Formatted at 2026-09-23 22:12:35 on dist-test-slave-s2fj"
server_key: "f68a8435bf25c6563c0c0e541f650462"
server_key_iv: "c804bdd3dfc9e9f4b7db0472d27f1a01"
server_key_version: "encryptionkey@0"
I20260923 22:12:35.920228 4591 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskRiQe_a/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790201539513534-1567-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskRiQe_a/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790201539513534-1567-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskRiQe_a/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790201539513534-1567-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260923 22:12:35.936059 4591 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260923 22:12:35.937726 4591 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260923 22:12:35.937873 4591 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260923 22:12:35.941707 4591 rpc_server.cc:307] RPC server started. Bound to: 127.1.135.254:38865
I20260923 22:12:35.941738 4658 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.1.135.254:38865 every 8 connection(s)
I20260923 22:12:35.942068 4591 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskRiQe_a/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790201539513534-1567-0/minicluster-data/master-0/data/info.pb
I20260923 22:12:35.942898 4659 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260923 22:12:35.945024 4659 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 9f76cdfc09604a399fecb97401673b76: Bootstrap starting.
I20260923 22:12:35.945565 4659 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 9f76cdfc09604a399fecb97401673b76: Neither blocks nor log segments found. Creating new log.
I20260923 22:12:35.945858 4659 log.cc:824] T 00000000000000000000000000000000 P 9f76cdfc09604a399fecb97401673b76: Log is configured to *not* fsync() on all Append() calls
I20260923 22:12:35.946480 4659 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 9f76cdfc09604a399fecb97401673b76: No bootstrap required, opened a new log
I20260923 22:12:35.947983 4659 raft_consensus.cc:348] T 00000000000000000000000000000000 P 9f76cdfc09604a399fecb97401673b76 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9f76cdfc09604a399fecb97401673b76" member_type: VOTER last_known_addr { host: "127.1.135.254" port: 38865 } }
I20260923 22:12:35.948165 4659 raft_consensus.cc:374] T 00000000000000000000000000000000 P 9f76cdfc09604a399fecb97401673b76 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260923 22:12:35.948210 4659 raft_consensus.cc:729] T 00000000000000000000000000000000 P 9f76cdfc09604a399fecb97401673b76 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 9f76cdfc09604a399fecb97401673b76, State: Initialized, Role: FOLLOWER
I20260923 22:12:35.948372 4659 consensus_queue.cc:260] T 00000000000000000000000000000000 P 9f76cdfc09604a399fecb97401673b76 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9f76cdfc09604a399fecb97401673b76" member_type: VOTER last_known_addr { host: "127.1.135.254" port: 38865 } }
I20260923 22:12:35.948432 4659 raft_consensus.cc:388] T 00000000000000000000000000000000 P 9f76cdfc09604a399fecb97401673b76 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260923 22:12:35.948484 4659 raft_consensus.cc:482] T 00000000000000000000000000000000 P 9f76cdfc09604a399fecb97401673b76 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260923 22:12:35.948532 4659 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 9f76cdfc09604a399fecb97401673b76 [term 0 FOLLOWER]: Advancing to term 1
I20260923 22:12:35.949270 4659 raft_consensus.cc:504] T 00000000000000000000000000000000 P 9f76cdfc09604a399fecb97401673b76 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9f76cdfc09604a399fecb97401673b76" member_type: VOTER last_known_addr { host: "127.1.135.254" port: 38865 } }
I20260923 22:12:35.949352 4659 leader_election.cc:302] T 00000000000000000000000000000000 P 9f76cdfc09604a399fecb97401673b76 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 9f76cdfc09604a399fecb97401673b76; no voters:
I20260923 22:12:35.949553 4659 leader_election.cc:288] T 00000000000000000000000000000000 P 9f76cdfc09604a399fecb97401673b76 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260923 22:12:35.949604 4662 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 9f76cdfc09604a399fecb97401673b76 [term 1 FOLLOWER]: Leader election won for term 1
I20260923 22:12:35.949781 4662 raft_consensus.cc:686] T 00000000000000000000000000000000 P 9f76cdfc09604a399fecb97401673b76 [term 1 LEADER]: Becoming Leader. State: Replica: 9f76cdfc09604a399fecb97401673b76, State: Running, Role: LEADER
I20260923 22:12:35.949791 4659 sys_catalog.cc:564] T 00000000000000000000000000000000 P 9f76cdfc09604a399fecb97401673b76 [sys.catalog]: configured and running, proceeding with master startup.
I20260923 22:12:35.949895 4662 consensus_queue.cc:237] T 00000000000000000000000000000000 P 9f76cdfc09604a399fecb97401673b76 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9f76cdfc09604a399fecb97401673b76" member_type: VOTER last_known_addr { host: "127.1.135.254" port: 38865 } }
I20260923 22:12:35.950253 4664 sys_catalog.cc:455] T 00000000000000000000000000000000 P 9f76cdfc09604a399fecb97401673b76 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 9f76cdfc09604a399fecb97401673b76. Latest consensus state: current_term: 1 leader_uuid: "9f76cdfc09604a399fecb97401673b76" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9f76cdfc09604a399fecb97401673b76" member_type: VOTER last_known_addr { host: "127.1.135.254" port: 38865 } } }
I20260923 22:12:35.950341 4664 sys_catalog.cc:458] T 00000000000000000000000000000000 P 9f76cdfc09604a399fecb97401673b76 [sys.catalog]: This master's current role is: LEADER
I20260923 22:12:35.950523 4663 sys_catalog.cc:455] T 00000000000000000000000000000000 P 9f76cdfc09604a399fecb97401673b76 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "9f76cdfc09604a399fecb97401673b76" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9f76cdfc09604a399fecb97401673b76" member_type: VOTER last_known_addr { host: "127.1.135.254" port: 38865 } } }
I20260923 22:12:35.950608 4663 sys_catalog.cc:458] T 00000000000000000000000000000000 P 9f76cdfc09604a399fecb97401673b76 [sys.catalog]: This master's current role is: LEADER
I20260923 22:12:35.950793 4671 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260923 22:12:35.951217 4671 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260923 22:12:35.951583 1567 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskRiQe_a/build/release/bin/kudu as pid 4591
I20260923 22:12:35.951685 1567 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskRiQe_a/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790201539513534-1567-0/minicluster-data/master-0/wal/instance
I20260923 22:12:35.951866 1567 external_mini_cluster.cc:1428] Setting key dca0ae1f950fec7c1626247e354f2e48
I20260923 22:12:35.952767 4671 catalog_manager.cc:1347] Generated new cluster ID: a92547065d63490bba8a4813091d3e0d
I20260923 22:12:35.952823 4671 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
Sep 23 22:12:35 dist-test-slave-s2fj krb5kdc[4578](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790201555, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.1.135.254@KRBTEST.COM
I20260923 22:12:35.966818 4680 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 22:12:35.952273 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46756 (local address 127.1.135.254:38865)
0923 22:12:35.956179 (+ 3906us) server_negotiation.cc:206] Beginning negotiation
0923 22:12:35.956188 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0923 22:12:35.956270 (+ 82us) server_negotiation.cc:407] Connection header received
0923 22:12:35.956423 (+ 153us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 22:12:35.956429 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 22:12:35.956489 (+ 60us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 22:12:35.956616 (+ 127us) server_negotiation.cc:226] Negotiated authn=SASL
0923 22:12:35.957400 (+ 784us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 22:12:35.958745 (+ 1345us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 22:12:35.959837 (+ 1092us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 22:12:35.960455 (+ 618us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 22:12:35.960549 (+ 94us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 22:12:35.963228 (+ 2679us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0923 22:12:35.963232 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0923 22:12:35.963232 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0923 22:12:35.963244 (+ 12us) server_negotiation.cc:922] calling sasl_server_start()
0923 22:12:35.965223 (+ 1979us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0923 22:12:35.965604 (+ 381us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0923 22:12:35.965607 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0923 22:12:35.965608 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0923 22:12:35.965712 (+ 104us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0923 22:12:35.965908 (+ 196us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0923 22:12:35.965909 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0923 22:12:35.965909 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0923 22:12:35.966200 (+ 291us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0923 22:12:35.966330 (+ 130us) server_negotiation.cc:1035] Waiting for connection context
0923 22:12:35.966437 (+ 107us) server_negotiation.cc:299] Negotiation successful
0923 22:12:35.966572 (+ 135us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":3865,"thread_start_us":72,"threads_started":1}
I20260923 22:12:35.967798 4671 catalog_manager.cc:1370] Generated new certificate authority record
I20260923 22:12:35.968324 4671 catalog_manager.cc:1504] Loading token signing keys...
I20260923 22:12:35.970839 4671 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 9f76cdfc09604a399fecb97401673b76: Generated new TSK 0
I20260923 22:12:35.970983 4671 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.1.135.193@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.1.135.193@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.1.135.193 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskRiQe_a/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790201539513534-1567-0/krb5kdc/kudu_127.1.135.193.keytab.
Entry for principal kudu/127.1.135.193 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskRiQe_a/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790201539513534-1567-0/krb5kdc/kudu_127.1.135.193.keytab.
I20260923 22:12:35.998821 1567 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskRiQe_a/build/release/bin/kudu
/tmp/dist-test-taskRiQe_a/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskRiQe_a/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790201539513534-1567-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskRiQe_a/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790201539513534-1567-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskRiQe_a/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790201539513534-1567-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskRiQe_a/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790201539513534-1567-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.1.135.193:0
--local_ip_for_outbound_sockets=127.1.135.193
--webserver_interface=127.1.135.193
--webserver_port=0
--tserver_master_addrs=127.1.135.254:38865
--keytab_file=/tmp/dist-test-taskRiQe_a/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790201539513534-1567-0/krb5kdc/kudu_127.1.135.193.keytab
--principal=kudu/127.1.135.193
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.1.135.212:36457
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskRiQe_a/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790201539513534-1567-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskRiQe_a/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790201539513534-1567-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskRiQe_a/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790201539513534-1567-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260923 22:12:36.077080 4685 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260923 22:12:36.077309 4685 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260923 22:12:36.077345 4685 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260923 22:12:36.078496 4685 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260923 22:12:36.078560 4685 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260923 22:12:36.078603 4685 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.1.135.193
I20260923 22:12:36.079998 4685 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.1.135.212:36457
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskRiQe_a/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790201539513534-1567-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskRiQe_a/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790201539513534-1567-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskRiQe_a/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790201539513534-1567-0/krb5kdc/kudu_127.1.135.193.keytab
--principal=kudu/127.1.135.193
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.1.135.193:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskRiQe_a/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790201539513534-1567-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.1.135.193
--webserver_port=0
--tserver_master_addrs=127.1.135.254:38865
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.4685
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.1.135.193
--log_dir=/tmp/dist-test-taskRiQe_a/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790201539513534-1567-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 23 Sep 2026 20:53:37 UTC on ccycloud.quasar-rkviyx.root.comops.site
build id 4668083
I20260923 22:12:36.080276 4685 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260923 22:12:36.080499 4685 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260923 22:12:36.085268 4690 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260923 22:12:36.085250 4693 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260923 22:12:36.085242 4691 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260923 22:12:36.085500 4685 server_base.cc:1034] running on GCE node
I20260923 22:12:36.085716 4685 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260923 22:12:36.085951 4685 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260923 22:12:36.087111 4685 hybrid_clock.cc:648] HybridClock initialized: now 1790201556087091 us; error 33 us; skew 500 ppm
Sep 23 22:12:36 dist-test-slave-s2fj krb5kdc[4578](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790201556, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.135.193@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260923 22:12:36.089494 4685 init.cc:377] Logged in from keytab as kudu/127.1.135.193@KRBTEST.COM (short username kudu)
I20260923 22:12:36.089946 4685 webserver.cc:460] Webserver started at http://127.1.135.193:39107/ using document root <none> and password file <none>
I20260923 22:12:36.090163 4685 fs_manager.cc:359] Metadata directory not provided
I20260923 22:12:36.090225 4685 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260923 22:12:36.090373 4685 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260923 22:12:36.091310 4685 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskRiQe_a/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790201539513534-1567-0/minicluster-data/ts-0/data/instance:
uuid: "0828fdf37222426492250640a12f3927"
format_stamp: "Formatted at 2026-09-23 22:12:36 on dist-test-slave-s2fj"
server_key: "9ca9aa6689b55bec07c1aeffdde08bc1"
server_key_iv: "c8a3ce920397c429bfb4fdbf8faa6d50"
server_key_version: "encryptionkey@0"
I20260923 22:12:36.091645 4685 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskRiQe_a/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790201539513534-1567-0/minicluster-data/ts-0/wal/instance:
uuid: "0828fdf37222426492250640a12f3927"
format_stamp: "Formatted at 2026-09-23 22:12:36 on dist-test-slave-s2fj"
server_key: "9ca9aa6689b55bec07c1aeffdde08bc1"
server_key_iv: "c8a3ce920397c429bfb4fdbf8faa6d50"
server_key_version: "encryptionkey@0"
I20260923 22:12:36.092991 4685 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.003s
I20260923 22:12:36.093636 4700 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260923 22:12:36.093798 4685 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.001s
I20260923 22:12:36.093889 4685 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskRiQe_a/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790201539513534-1567-0/minicluster-data/ts-0/data,/tmp/dist-test-taskRiQe_a/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790201539513534-1567-0/minicluster-data/ts-0/wal
uuid: "0828fdf37222426492250640a12f3927"
format_stamp: "Formatted at 2026-09-23 22:12:36 on dist-test-slave-s2fj"
server_key: "9ca9aa6689b55bec07c1aeffdde08bc1"
server_key_iv: "c8a3ce920397c429bfb4fdbf8faa6d50"
server_key_version: "encryptionkey@0"
I20260923 22:12:36.093972 4685 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskRiQe_a/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790201539513534-1567-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskRiQe_a/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790201539513534-1567-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskRiQe_a/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790201539513534-1567-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260923 22:12:36.106210 4685 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260923 22:12:36.107934 4685 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260923 22:12:36.108106 4685 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260923 22:12:36.108322 4685 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260923 22:12:36.108616 4685 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260923 22:12:36.108669 4685 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260923 22:12:36.108700 4685 ts_tablet_manager.cc:610] Registered 0 tablets
I20260923 22:12:36.108726 4685 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260923 22:12:36.114256 4685 rpc_server.cc:307] RPC server started. Bound to: 127.1.135.193:33301
I20260923 22:12:36.114296 4813 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.1.135.193:33301 every 8 connection(s)
I20260923 22:12:36.114603 4685 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskRiQe_a/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790201539513534-1567-0/minicluster-data/ts-0/data/info.pb
Sep 23 22:12:36 dist-test-slave-s2fj krb5kdc[4578](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790201556, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.135.193@KRBTEST.COM for kudu/127.1.135.254@KRBTEST.COM
I20260923 22:12:36.122844 4680 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 22:12:36.115297 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.135.193:50907 (local address 127.1.135.254:38865)
0923 22:12:36.115336 (+ 39us) server_negotiation.cc:206] Beginning negotiation
0923 22:12:36.115338 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0923 22:12:36.115522 (+ 184us) server_negotiation.cc:407] Connection header received
0923 22:12:36.116187 (+ 665us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 22:12:36.116188 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 22:12:36.116200 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 22:12:36.116242 (+ 42us) server_negotiation.cc:226] Negotiated authn=SASL
0923 22:12:36.117067 (+ 825us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 22:12:36.117531 (+ 464us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 22:12:36.118417 (+ 886us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 22:12:36.118599 (+ 182us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 22:12:36.118646 (+ 47us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 22:12:36.120461 (+ 1815us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0923 22:12:36.120462 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0923 22:12:36.120462 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0923 22:12:36.120469 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0923 22:12:36.121590 (+ 1121us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0923 22:12:36.121919 (+ 329us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0923 22:12:36.121920 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0923 22:12:36.121920 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0923 22:12:36.121965 (+ 45us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0923 22:12:36.122213 (+ 248us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0923 22:12:36.122214 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0923 22:12:36.122214 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0923 22:12:36.122302 (+ 88us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0923 22:12:36.122378 (+ 76us) server_negotiation.cc:1035] Waiting for connection context
0923 22:12:36.122655 (+ 277us) server_negotiation.cc:299] Negotiation successful
0923 22:12:36.122746 (+ 91us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":18}
I20260923 22:12:36.122946 4816 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 22:12:36.115262 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.135.254:38865 (local address 127.1.135.193:50907)
0923 22:12:36.115400 (+ 138us) negotiation.cc:107] Waiting for socket to connect
0923 22:12:36.115437 (+ 37us) client_negotiation.cc:174] Beginning negotiation
0923 22:12:36.116027 (+ 590us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0923 22:12:36.116294 (+ 267us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0923 22:12:36.116297 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0923 22:12:36.116829 (+ 532us) client_negotiation.cc:189] Negotiated authn=SASL
0923 22:12:36.116994 (+ 165us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 22:12:36.116998 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 22:12:36.117605 (+ 607us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 22:12:36.117606 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 22:12:36.118347 (+ 741us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 22:12:36.118349 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 22:12:36.118667 (+ 318us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 22:12:36.118668 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 22:12:36.118760 (+ 92us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 22:12:36.119133 (+ 373us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0923 22:12:36.119140 (+ 7us) client_negotiation.cc:656] calling sasl_client_start()
0923 22:12:36.120381 (+ 1241us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0923 22:12:36.121654 (+ 1273us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0923 22:12:36.121660 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0923 22:12:36.121661 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0923 22:12:36.121856 (+ 195us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0923 22:12:36.122020 (+ 164us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0923 22:12:36.122021 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0923 22:12:36.122021 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0923 22:12:36.122136 (+ 115us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0923 22:12:36.122371 (+ 235us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0923 22:12:36.122377 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0923 22:12:36.122504 (+ 127us) client_negotiation.cc:769] Sending connection context
0923 22:12:36.122608 (+ 104us) client_negotiation.cc:240] Negotiation successful
0923 22:12:36.122747 (+ 139us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":101,"thread_start_us":77,"threads_started":1}
I20260923 22:12:36.123272 1567 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskRiQe_a/build/release/bin/kudu as pid 4685
I20260923 22:12:36.123378 1567 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskRiQe_a/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790201539513534-1567-0/minicluster-data/ts-0/wal/instance
I20260923 22:12:36.123373 4814 heartbeater.cc:347] Connected to a master server at 127.1.135.254:38865
I20260923 22:12:36.123447 4814 heartbeater.cc:464] Registering TS with master...
I20260923 22:12:36.123569 1567 external_mini_cluster.cc:1428] Setting key b683804ca39f71c62deb84d5f7caa1eb
I20260923 22:12:36.123620 4814 heartbeater.cc:511] Master 127.1.135.254:38865 requested a full tablet report, sending...
I20260923 22:12:36.124193 4622 ts_manager.cc:194] Registered new tserver with Master: 0828fdf37222426492250640a12f3927 (127.1.135.193:33301)
I20260923 22:12:36.124814 4622 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.1.135.193@KRBTEST.COM'} at 127.1.135.193:50907
WARNING: no policy specified for kudu/127.1.135.194@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.1.135.194@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.1.135.194 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskRiQe_a/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790201539513534-1567-0/krb5kdc/kudu_127.1.135.194.keytab.
Entry for principal kudu/127.1.135.194 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskRiQe_a/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790201539513534-1567-0/krb5kdc/kudu_127.1.135.194.keytab.
I20260923 22:12:36.147089 1567 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskRiQe_a/build/release/bin/kudu
/tmp/dist-test-taskRiQe_a/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskRiQe_a/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790201539513534-1567-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskRiQe_a/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790201539513534-1567-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskRiQe_a/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790201539513534-1567-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskRiQe_a/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790201539513534-1567-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.1.135.194:0
--local_ip_for_outbound_sockets=127.1.135.194
--webserver_interface=127.1.135.194
--webserver_port=0
--tserver_master_addrs=127.1.135.254:38865
--keytab_file=/tmp/dist-test-taskRiQe_a/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790201539513534-1567-0/krb5kdc/kudu_127.1.135.194.keytab
--principal=kudu/127.1.135.194
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.1.135.212:36457
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskRiQe_a/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790201539513534-1567-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskRiQe_a/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790201539513534-1567-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskRiQe_a/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790201539513534-1567-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260923 22:12:36.227171 4819 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260923 22:12:36.227380 4819 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260923 22:12:36.227402 4819 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260923 22:12:36.228557 4819 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260923 22:12:36.228600 4819 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260923 22:12:36.228626 4819 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.1.135.194
I20260923 22:12:36.230052 4819 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.1.135.212:36457
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskRiQe_a/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790201539513534-1567-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskRiQe_a/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790201539513534-1567-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskRiQe_a/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790201539513534-1567-0/krb5kdc/kudu_127.1.135.194.keytab
--principal=kudu/127.1.135.194
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.1.135.194:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskRiQe_a/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790201539513534-1567-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.1.135.194
--webserver_port=0
--tserver_master_addrs=127.1.135.254:38865
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.4819
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.1.135.194
--log_dir=/tmp/dist-test-taskRiQe_a/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790201539513534-1567-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 23 Sep 2026 20:53:37 UTC on ccycloud.quasar-rkviyx.root.comops.site
build id 4668083
I20260923 22:12:36.230221 4819 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260923 22:12:36.230474 4819 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260923 22:12:36.235306 4825 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260923 22:12:36.235430 4819 server_base.cc:1034] running on GCE node
W20260923 22:12:36.235306 4824 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260923 22:12:36.235364 4827 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260923 22:12:36.235821 4819 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260923 22:12:36.236040 4819 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260923 22:12:36.237226 4819 hybrid_clock.cc:648] HybridClock initialized: now 1790201556237216 us; error 28 us; skew 500 ppm
Sep 23 22:12:36 dist-test-slave-s2fj krb5kdc[4578](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790201556, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.135.194@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260923 22:12:36.239566 4819 init.cc:377] Logged in from keytab as kudu/127.1.135.194@KRBTEST.COM (short username kudu)
I20260923 22:12:36.239964 4819 webserver.cc:460] Webserver started at http://127.1.135.194:33049/ using document root <none> and password file <none>
I20260923 22:12:36.240240 4819 fs_manager.cc:359] Metadata directory not provided
I20260923 22:12:36.240314 4819 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260923 22:12:36.240461 4819 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260923 22:12:36.241461 4819 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskRiQe_a/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790201539513534-1567-0/minicluster-data/ts-1/data/instance:
uuid: "04080d588767444ebc7acf91ff757e90"
format_stamp: "Formatted at 2026-09-23 22:12:36 on dist-test-slave-s2fj"
server_key: "e57225c0318b8de12af3256bce8c09f8"
server_key_iv: "8b27809a9fbbc7ec85bb777b6e52ded7"
server_key_version: "encryptionkey@0"
I20260923 22:12:36.241814 4819 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskRiQe_a/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790201539513534-1567-0/minicluster-data/ts-1/wal/instance:
uuid: "04080d588767444ebc7acf91ff757e90"
format_stamp: "Formatted at 2026-09-23 22:12:36 on dist-test-slave-s2fj"
server_key: "e57225c0318b8de12af3256bce8c09f8"
server_key_iv: "8b27809a9fbbc7ec85bb777b6e52ded7"
server_key_version: "encryptionkey@0"
I20260923 22:12:36.243021 4819 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260923 22:12:36.243661 4834 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260923 22:12:36.243831 4819 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260923 22:12:36.243924 4819 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskRiQe_a/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790201539513534-1567-0/minicluster-data/ts-1/data,/tmp/dist-test-taskRiQe_a/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790201539513534-1567-0/minicluster-data/ts-1/wal
uuid: "04080d588767444ebc7acf91ff757e90"
format_stamp: "Formatted at 2026-09-23 22:12:36 on dist-test-slave-s2fj"
server_key: "e57225c0318b8de12af3256bce8c09f8"
server_key_iv: "8b27809a9fbbc7ec85bb777b6e52ded7"
server_key_version: "encryptionkey@0"
I20260923 22:12:36.244005 4819 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskRiQe_a/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790201539513534-1567-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskRiQe_a/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790201539513534-1567-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskRiQe_a/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790201539513534-1567-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260923 22:12:36.278663 4819 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260923 22:12:36.281592 4819 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260923 22:12:36.281754 4819 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260923 22:12:36.282027 4819 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260923 22:12:36.282420 4819 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260923 22:12:36.282474 4819 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260923 22:12:36.282519 4819 ts_tablet_manager.cc:610] Registered 0 tablets
I20260923 22:12:36.282550 4819 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260923 22:12:36.290508 4819 rpc_server.cc:307] RPC server started. Bound to: 127.1.135.194:41973
I20260923 22:12:36.290591 4947 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.1.135.194:41973 every 8 connection(s)
I20260923 22:12:36.290954 4819 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskRiQe_a/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790201539513534-1567-0/minicluster-data/ts-1/data/info.pb
I20260923 22:12:36.291827 1567 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskRiQe_a/build/release/bin/kudu as pid 4819
I20260923 22:12:36.291920 1567 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskRiQe_a/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790201539513534-1567-0/minicluster-data/ts-1/wal/instance
I20260923 22:12:36.292200 1567 external_mini_cluster.cc:1428] Setting key cf580fea1ba1a7cb00d90f41e4a623d2
Sep 23 22:12:36 dist-test-slave-s2fj krb5kdc[4578](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790201556, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.135.194@KRBTEST.COM for kudu/127.1.135.254@KRBTEST.COM
WARNING: no policy specified for kudu/127.1.135.195@KRBTEST.COM; defaulting to no policy
I20260923 22:12:36.302776 4680 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 22:12:36.291662 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.135.194:47157 (local address 127.1.135.254:38865)
0923 22:12:36.291730 (+ 68us) server_negotiation.cc:206] Beginning negotiation
0923 22:12:36.291732 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0923 22:12:36.291995 (+ 263us) server_negotiation.cc:407] Connection header received
0923 22:12:36.292785 (+ 790us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 22:12:36.292786 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 22:12:36.292799 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 22:12:36.292833 (+ 34us) server_negotiation.cc:226] Negotiated authn=SASL
0923 22:12:36.294336 (+ 1503us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 22:12:36.295149 (+ 813us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 22:12:36.296857 (+ 1708us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 22:12:36.297145 (+ 288us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 22:12:36.297206 (+ 61us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 22:12:36.299575 (+ 2369us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0923 22:12:36.299576 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0923 22:12:36.299577 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0923 22:12:36.299583 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0923 22:12:36.301242 (+ 1659us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0923 22:12:36.301733 (+ 491us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0923 22:12:36.301735 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0923 22:12:36.301735 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0923 22:12:36.301791 (+ 56us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0923 22:12:36.302079 (+ 288us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0923 22:12:36.302079 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0923 22:12:36.302080 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0923 22:12:36.302194 (+ 114us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0923 22:12:36.302277 (+ 83us) server_negotiation.cc:1035] Waiting for connection context
0923 22:12:36.302537 (+ 260us) server_negotiation.cc:299] Negotiation successful
0923 22:12:36.302653 (+ 116us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":40}
I20260923 22:12:36.302899 4950 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 22:12:36.291653 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.135.254:38865 (local address 127.1.135.194:47157)
0923 22:12:36.291835 (+ 182us) negotiation.cc:107] Waiting for socket to connect
0923 22:12:36.291885 (+ 50us) client_negotiation.cc:174] Beginning negotiation
0923 22:12:36.292659 (+ 774us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0923 22:12:36.293034 (+ 375us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0923 22:12:36.293039 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0923 22:12:36.293923 (+ 884us) client_negotiation.cc:189] Negotiated authn=SASL
0923 22:12:36.294210 (+ 287us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 22:12:36.294215 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 22:12:36.295279 (+ 1064us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 22:12:36.295280 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 22:12:36.296769 (+ 1489us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 22:12:36.296772 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 22:12:36.297242 (+ 470us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 22:12:36.297243 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 22:12:36.297434 (+ 191us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 22:12:36.297925 (+ 491us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0923 22:12:36.297934 (+ 9us) client_negotiation.cc:656] calling sasl_client_start()
0923 22:12:36.299471 (+ 1537us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0923 22:12:36.301317 (+ 1846us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0923 22:12:36.301323 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0923 22:12:36.301325 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0923 22:12:36.301600 (+ 275us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0923 22:12:36.301878 (+ 278us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0923 22:12:36.301878 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0923 22:12:36.301879 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0923 22:12:36.302003 (+ 124us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0923 22:12:36.302267 (+ 264us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0923 22:12:36.302273 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0923 22:12:36.302409 (+ 136us) client_negotiation.cc:769] Sending connection context
0923 22:12:36.302517 (+ 108us) client_negotiation.cc:240] Negotiation successful
0923 22:12:36.302665 (+ 148us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":136,"thread_start_us":95,"threads_started":1}
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.1.135.195@KRBTEST.COM" created.
I20260923 22:12:36.303359 4948 heartbeater.cc:347] Connected to a master server at 127.1.135.254:38865
I20260923 22:12:36.303444 4948 heartbeater.cc:464] Registering TS with master...
I20260923 22:12:36.303654 4948 heartbeater.cc:511] Master 127.1.135.254:38865 requested a full tablet report, sending...
I20260923 22:12:36.304039 4622 ts_manager.cc:194] Registered new tserver with Master: 04080d588767444ebc7acf91ff757e90 (127.1.135.194:41973)
I20260923 22:12:36.304663 4622 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.1.135.194@KRBTEST.COM'} at 127.1.135.194:47157
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.1.135.195 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskRiQe_a/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790201539513534-1567-0/krb5kdc/kudu_127.1.135.195.keytab.
Entry for principal kudu/127.1.135.195 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskRiQe_a/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790201539513534-1567-0/krb5kdc/kudu_127.1.135.195.keytab.
I20260923 22:12:36.317122 1567 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskRiQe_a/build/release/bin/kudu
/tmp/dist-test-taskRiQe_a/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskRiQe_a/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790201539513534-1567-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskRiQe_a/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790201539513534-1567-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskRiQe_a/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790201539513534-1567-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskRiQe_a/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790201539513534-1567-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.1.135.195:0
--local_ip_for_outbound_sockets=127.1.135.195
--webserver_interface=127.1.135.195
--webserver_port=0
--tserver_master_addrs=127.1.135.254:38865
--keytab_file=/tmp/dist-test-taskRiQe_a/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790201539513534-1567-0/krb5kdc/kudu_127.1.135.195.keytab
--principal=kudu/127.1.135.195
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.1.135.212:36457
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskRiQe_a/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790201539513534-1567-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskRiQe_a/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790201539513534-1567-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskRiQe_a/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790201539513534-1567-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260923 22:12:36.395093 4953 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260923 22:12:36.395316 4953 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260923 22:12:36.395352 4953 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260923 22:12:36.396466 4953 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260923 22:12:36.396533 4953 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260923 22:12:36.396577 4953 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.1.135.195
I20260923 22:12:36.398072 4953 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.1.135.212:36457
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskRiQe_a/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790201539513534-1567-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskRiQe_a/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790201539513534-1567-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskRiQe_a/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790201539513534-1567-0/krb5kdc/kudu_127.1.135.195.keytab
--principal=kudu/127.1.135.195
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.1.135.195:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskRiQe_a/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790201539513534-1567-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.1.135.195
--webserver_port=0
--tserver_master_addrs=127.1.135.254:38865
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.4953
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.1.135.195
--log_dir=/tmp/dist-test-taskRiQe_a/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790201539513534-1567-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 23 Sep 2026 20:53:37 UTC on ccycloud.quasar-rkviyx.root.comops.site
build id 4668083
I20260923 22:12:36.398319 4953 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260923 22:12:36.398541 4953 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260923 22:12:36.403244 4958 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260923 22:12:36.403266 4959 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260923 22:12:36.403268 4961 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260923 22:12:36.403478 4953 server_base.cc:1034] running on GCE node
I20260923 22:12:36.403666 4953 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260923 22:12:36.403909 4953 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260923 22:12:36.405068 4953 hybrid_clock.cc:648] HybridClock initialized: now 1790201556405057 us; error 25 us; skew 500 ppm
Sep 23 22:12:36 dist-test-slave-s2fj krb5kdc[4578](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790201556, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.135.195@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260923 22:12:36.407562 4953 init.cc:377] Logged in from keytab as kudu/127.1.135.195@KRBTEST.COM (short username kudu)
I20260923 22:12:36.407917 4953 webserver.cc:460] Webserver started at http://127.1.135.195:43027/ using document root <none> and password file <none>
I20260923 22:12:36.408164 4953 fs_manager.cc:359] Metadata directory not provided
I20260923 22:12:36.408222 4953 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260923 22:12:36.408325 4953 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260923 22:12:36.409304 4953 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskRiQe_a/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790201539513534-1567-0/minicluster-data/ts-2/data/instance:
uuid: "6a09006c96974b0fa54c18208fc6ac2b"
format_stamp: "Formatted at 2026-09-23 22:12:36 on dist-test-slave-s2fj"
server_key: "6a02d7e8ec9ad60026a6e6fbfc1b27fd"
server_key_iv: "59cb82e6817a47d1b3af9660fbb4b3c1"
server_key_version: "encryptionkey@0"
I20260923 22:12:36.409588 4953 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskRiQe_a/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790201539513534-1567-0/minicluster-data/ts-2/wal/instance:
uuid: "6a09006c96974b0fa54c18208fc6ac2b"
format_stamp: "Formatted at 2026-09-23 22:12:36 on dist-test-slave-s2fj"
server_key: "6a02d7e8ec9ad60026a6e6fbfc1b27fd"
server_key_iv: "59cb82e6817a47d1b3af9660fbb4b3c1"
server_key_version: "encryptionkey@0"
I20260923 22:12:36.410763 4953 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.002s
I20260923 22:12:36.411458 4968 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260923 22:12:36.411603 4953 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.001s
I20260923 22:12:36.411680 4953 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskRiQe_a/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790201539513534-1567-0/minicluster-data/ts-2/data,/tmp/dist-test-taskRiQe_a/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790201539513534-1567-0/minicluster-data/ts-2/wal
uuid: "6a09006c96974b0fa54c18208fc6ac2b"
format_stamp: "Formatted at 2026-09-23 22:12:36 on dist-test-slave-s2fj"
server_key: "6a02d7e8ec9ad60026a6e6fbfc1b27fd"
server_key_iv: "59cb82e6817a47d1b3af9660fbb4b3c1"
server_key_version: "encryptionkey@0"
I20260923 22:12:36.411731 4953 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskRiQe_a/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790201539513534-1567-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskRiQe_a/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790201539513534-1567-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskRiQe_a/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790201539513534-1567-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260923 22:12:36.435856 4953 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260923 22:12:36.437716 4953 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260923 22:12:36.437824 4953 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260923 22:12:36.437999 4953 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260923 22:12:36.438238 4953 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260923 22:12:36.438277 4953 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260923 22:12:36.438298 4953 ts_tablet_manager.cc:610] Registered 0 tablets
I20260923 22:12:36.438310 4953 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260923 22:12:36.443871 4953 rpc_server.cc:307] RPC server started. Bound to: 127.1.135.195:36841
I20260923 22:12:36.443917 5081 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.1.135.195:36841 every 8 connection(s)
I20260923 22:12:36.444252 4953 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskRiQe_a/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790201539513534-1567-0/minicluster-data/ts-2/data/info.pb
Sep 23 22:12:36 dist-test-slave-s2fj krb5kdc[4578](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790201556, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.135.195@KRBTEST.COM for kudu/127.1.135.254@KRBTEST.COM
I20260923 22:12:36.451606 1567 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskRiQe_a/build/release/bin/kudu as pid 4953
I20260923 22:12:36.451721 1567 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskRiQe_a/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790201539513534-1567-0/minicluster-data/ts-2/wal/instance
I20260923 22:12:36.451979 1567 external_mini_cluster.cc:1428] Setting key 4028fdc2c6b0fc2a0c8cccd1d6310dd7
I20260923 22:12:36.453862 4680 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 22:12:36.444976 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.135.195:46993 (local address 127.1.135.254:38865)
0923 22:12:36.445039 (+ 63us) server_negotiation.cc:206] Beginning negotiation
0923 22:12:36.445042 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0923 22:12:36.445255 (+ 213us) server_negotiation.cc:407] Connection header received
0923 22:12:36.445706 (+ 451us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 22:12:36.445707 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 22:12:36.445718 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 22:12:36.445766 (+ 48us) server_negotiation.cc:226] Negotiated authn=SASL
0923 22:12:36.446736 (+ 970us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 22:12:36.447216 (+ 480us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 22:12:36.448298 (+ 1082us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 22:12:36.448477 (+ 179us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 22:12:36.448526 (+ 49us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 22:12:36.450669 (+ 2143us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0923 22:12:36.450671 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0923 22:12:36.450671 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0923 22:12:36.450678 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0923 22:12:36.452528 (+ 1850us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0923 22:12:36.453015 (+ 487us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0923 22:12:36.453016 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0923 22:12:36.453017 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0923 22:12:36.453082 (+ 65us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0923 22:12:36.453329 (+ 247us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0923 22:12:36.453330 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0923 22:12:36.453330 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0923 22:12:36.453439 (+ 109us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0923 22:12:36.453495 (+ 56us) server_negotiation.cc:1035] Waiting for connection context
0923 22:12:36.453729 (+ 234us) server_negotiation.cc:299] Negotiation successful
0923 22:12:36.453794 (+ 65us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":34}
I20260923 22:12:36.453961 5084 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 22:12:36.444966 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.135.254:38865 (local address 127.1.135.195:46993)
0923 22:12:36.445136 (+ 170us) negotiation.cc:107] Waiting for socket to connect
0923 22:12:36.445169 (+ 33us) client_negotiation.cc:174] Beginning negotiation
0923 22:12:36.445629 (+ 460us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0923 22:12:36.445828 (+ 199us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0923 22:12:36.445831 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0923 22:12:36.446434 (+ 603us) client_negotiation.cc:189] Negotiated authn=SASL
0923 22:12:36.446659 (+ 225us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 22:12:36.446664 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 22:12:36.447313 (+ 649us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 22:12:36.447313 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 22:12:36.448218 (+ 905us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 22:12:36.448220 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 22:12:36.448540 (+ 320us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 22:12:36.448541 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 22:12:36.448634 (+ 93us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 22:12:36.449034 (+ 400us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0923 22:12:36.449042 (+ 8us) client_negotiation.cc:656] calling sasl_client_start()
0923 22:12:36.450580 (+ 1538us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0923 22:12:36.452611 (+ 2031us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0923 22:12:36.452617 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0923 22:12:36.452619 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0923 22:12:36.452914 (+ 295us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0923 22:12:36.453156 (+ 242us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0923 22:12:36.453157 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0923 22:12:36.453157 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0923 22:12:36.453251 (+ 94us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0923 22:12:36.453497 (+ 246us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0923 22:12:36.453501 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0923 22:12:36.453602 (+ 101us) client_negotiation.cc:769] Sending connection context
0923 22:12:36.453679 (+ 77us) client_negotiation.cc:240] Negotiation successful
0923 22:12:36.453791 (+ 112us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":134,"thread_start_us":94,"threads_started":1}
I20260923 22:12:36.454411 5082 heartbeater.cc:347] Connected to a master server at 127.1.135.254:38865
I20260923 22:12:36.454484 5082 heartbeater.cc:464] Registering TS with master...
I20260923 22:12:36.454656 5082 heartbeater.cc:511] Master 127.1.135.254:38865 requested a full tablet report, sending...
I20260923 22:12:36.455034 4622 ts_manager.cc:194] Registered new tserver with Master: 6a09006c96974b0fa54c18208fc6ac2b (127.1.135.195:36841)
I20260923 22:12:36.455437 4622 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.1.135.195@KRBTEST.COM'} at 127.1.135.195:46993
I20260923 22:12:36.463937 1567 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
Sep 23 22:12:36 dist-test-slave-s2fj krb5kdc[4578](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790201556, etypes {rep=17 tkt=17 ses=17}, joe-interloper@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for joe-interloper@KRBTEST.COM:
Sep 23 22:12:36 dist-test-slave-s2fj krb5kdc[4578](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790201556, etypes {rep=17 tkt=17 ses=17}, joe-interloper@KRBTEST.COM for kudu/127.1.135.254@KRBTEST.COM
I20260923 22:12:36.486922 4680 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 22:12:36.480116 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46768 (local address 127.1.135.254:38865)
0923 22:12:36.480178 (+ 62us) server_negotiation.cc:206] Beginning negotiation
0923 22:12:36.480181 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0923 22:12:36.480325 (+ 144us) server_negotiation.cc:407] Connection header received
0923 22:12:36.480370 (+ 45us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 22:12:36.480370 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 22:12:36.480379 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 22:12:36.480433 (+ 54us) server_negotiation.cc:226] Negotiated authn=SASL
0923 22:12:36.481034 (+ 601us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 22:12:36.481508 (+ 474us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 22:12:36.482374 (+ 866us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 22:12:36.482545 (+ 171us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 22:12:36.482594 (+ 49us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 22:12:36.484742 (+ 2148us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0923 22:12:36.484744 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0923 22:12:36.484744 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0923 22:12:36.484750 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0923 22:12:36.486072 (+ 1322us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0923 22:12:36.486348 (+ 276us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0923 22:12:36.486348 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0923 22:12:36.486349 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0923 22:12:36.486395 (+ 46us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0923 22:12:36.486533 (+ 138us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0923 22:12:36.486534 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0923 22:12:36.486534 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0923 22:12:36.486650 (+ 116us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0923 22:12:36.486720 (+ 70us) server_negotiation.cc:1035] Waiting for connection context
0923 22:12:36.486807 (+ 87us) server_negotiation.cc:299] Negotiation successful
0923 22:12:36.486862 (+ 55us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":36}
W20260923 22:12:36.487159 4622 server_base.cc:1116] Unauthorized access attempt to method kudu.master.MasterService.ConnectToMaster from {username='joe-interloper', principal='joe-interloper@KRBTEST.COM'} at 127.0.0.1:46768
I20260923 22:12:36.487653 1567 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskRiQe_a/build/release/bin/kudu with pid 4685
I20260923 22:12:36.491905 1567 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskRiQe_a/build/release/bin/kudu with pid 4819
I20260923 22:12:36.495654 1567 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskRiQe_a/build/release/bin/kudu with pid 4953
I20260923 22:12:36.499769 1567 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskRiQe_a/build/release/bin/kudu with pid 4591
2026-09-23T22:12:36Z chronyd exiting
[ OK ] SecurityITest.TestUnauthorizedClientKerberosCredentials (1811 ms)
[ RUN ] SecurityITest.TestAuthorizedSuperuser
Loading random data
Initializing database '/tmp/dist-test-taskRiQe_a/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790201539513534-1567-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Sep 23 22:12:36 dist-test-slave-s2fj krb5kdc[5094](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Sep 23 22:12:36 dist-test-slave-s2fj krb5kdc[5094](info): set up 2 sockets
Sep 23 22:12:36 dist-test-slave-s2fj krb5kdc[5094](info): commencing operation
krb5kdc: starting...
W20260923 22:12:37.520035 1567 mini_kdc.cc:121] Time spent starting KDC: real 1.001s user 0.000s sys 0.005s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskRiQe_a/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790201539513534-1567-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskRiQe_a/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790201539513534-1567-0/krb5kdc/test-user.keytab.
Sep 23 22:12:37 dist-test-slave-s2fj krb5kdc[5094](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790201557, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-09-23T22:12:37Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-09-23T22:12:37Z Disabled control of system clock
WARNING: no policy specified for kudu/127.1.135.254@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.1.135.254@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.1.135.254 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskRiQe_a/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790201539513534-1567-0/krb5kdc/kudu_127.1.135.254.keytab.
Entry for principal kudu/127.1.135.254 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskRiQe_a/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790201539513534-1567-0/krb5kdc/kudu_127.1.135.254.keytab.
I20260923 22:12:37.631124 1567 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskRiQe_a/build/release/bin/kudu
/tmp/dist-test-taskRiQe_a/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskRiQe_a/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790201539513534-1567-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskRiQe_a/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790201539513534-1567-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskRiQe_a/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790201539513534-1567-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskRiQe_a/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790201539513534-1567-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.1.135.254:35103
--webserver_interface=127.1.135.254
--webserver_port=0
--keytab_file=/tmp/dist-test-taskRiQe_a/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790201539513534-1567-0/krb5kdc/kudu_127.1.135.254.keytab
--principal=kudu/127.1.135.254
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.1.135.212:43913
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.1.135.254:35103
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskRiQe_a/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790201539513534-1567-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskRiQe_a/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790201539513534-1567-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskRiQe_a/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790201539513534-1567-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260923 22:12:37.708139 5107 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260923 22:12:37.708387 5107 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260923 22:12:37.708436 5107 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260923 22:12:37.709539 5107 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260923 22:12:37.709605 5107 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260923 22:12:37.709628 5107 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260923 22:12:37.709647 5107 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260923 22:12:37.709676 5107 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260923 22:12:37.711092 5107 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.1.135.212:43913
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskRiQe_a/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790201539513534-1567-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskRiQe_a/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790201539513534-1567-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.1.135.254:35103
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskRiQe_a/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790201539513534-1567-0/krb5kdc/kudu_127.1.135.254.keytab
--principal=kudu/127.1.135.254
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.1.135.254:35103
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskRiQe_a/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790201539513534-1567-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.1.135.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.5107
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskRiQe_a/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790201539513534-1567-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 23 Sep 2026 20:53:37 UTC on ccycloud.quasar-rkviyx.root.comops.site
build id 4668083
I20260923 22:12:37.711359 5107 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260923 22:12:37.711568 5107 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260923 22:12:37.716066 5115 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260923 22:12:37.716110 5113 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260923 22:12:37.716089 5112 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260923 22:12:37.716486 5107 server_base.cc:1034] running on GCE node
I20260923 22:12:37.716675 5107 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260923 22:12:37.716899 5107 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260923 22:12:37.718029 5107 hybrid_clock.cc:648] HybridClock initialized: now 1790201557718017 us; error 27 us; skew 500 ppm
Sep 23 22:12:37 dist-test-slave-s2fj krb5kdc[5094](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790201557, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.135.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260923 22:12:37.725921 5107 init.cc:377] Logged in from keytab as kudu/127.1.135.254@KRBTEST.COM (short username kudu)
I20260923 22:12:37.726370 5107 webserver.cc:460] Webserver started at http://127.1.135.254:42369/ using document root <none> and password file <none>
I20260923 22:12:37.726591 5107 fs_manager.cc:359] Metadata directory not provided
I20260923 22:12:37.726656 5107 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260923 22:12:37.726807 5107 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260923 22:12:37.727802 5107 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskRiQe_a/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790201539513534-1567-0/minicluster-data/master-0/data/instance:
uuid: "1ec9d8e79ea345f5ba8fb445a3e51d51"
format_stamp: "Formatted at 2026-09-23 22:12:37 on dist-test-slave-s2fj"
server_key: "dd8d06880475ebf1c7dd3b1cd067babb"
server_key_iv: "206fe88a535ad49bbf990293c284baab"
server_key_version: "encryptionkey@0"
I20260923 22:12:37.728209 5107 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskRiQe_a/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790201539513534-1567-0/minicluster-data/master-0/wal/instance:
uuid: "1ec9d8e79ea345f5ba8fb445a3e51d51"
format_stamp: "Formatted at 2026-09-23 22:12:37 on dist-test-slave-s2fj"
server_key: "dd8d06880475ebf1c7dd3b1cd067babb"
server_key_iv: "206fe88a535ad49bbf990293c284baab"
server_key_version: "encryptionkey@0"
I20260923 22:12:37.729866 5107 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.002s
I20260923 22:12:37.730547 5122 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260923 22:12:37.730710 5107 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.001s
I20260923 22:12:37.730811 5107 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskRiQe_a/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790201539513534-1567-0/minicluster-data/master-0/data,/tmp/dist-test-taskRiQe_a/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790201539513534-1567-0/minicluster-data/master-0/wal
uuid: "1ec9d8e79ea345f5ba8fb445a3e51d51"
format_stamp: "Formatted at 2026-09-23 22:12:37 on dist-test-slave-s2fj"
server_key: "dd8d06880475ebf1c7dd3b1cd067babb"
server_key_iv: "206fe88a535ad49bbf990293c284baab"
server_key_version: "encryptionkey@0"
I20260923 22:12:37.730895 5107 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskRiQe_a/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790201539513534-1567-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskRiQe_a/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790201539513534-1567-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskRiQe_a/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790201539513534-1567-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260923 22:12:37.748585 5107 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260923 22:12:37.750746 5107 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260923 22:12:37.750900 5107 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260923 22:12:37.754804 5107 rpc_server.cc:307] RPC server started. Bound to: 127.1.135.254:35103
I20260923 22:12:37.754843 5174 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.1.135.254:35103 every 8 connection(s)
I20260923 22:12:37.755303 5107 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskRiQe_a/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790201539513534-1567-0/minicluster-data/master-0/data/info.pb
I20260923 22:12:37.755717 1567 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskRiQe_a/build/release/bin/kudu as pid 5107
I20260923 22:12:37.755823 1567 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskRiQe_a/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790201539513534-1567-0/minicluster-data/master-0/wal/instance
I20260923 22:12:37.755956 5175 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260923 22:12:37.756047 1567 external_mini_cluster.cc:1428] Setting key f7a72ca22e5fc1dbedf71136fa4d9091
I20260923 22:12:37.758594 5175 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 1ec9d8e79ea345f5ba8fb445a3e51d51: Bootstrap starting.
I20260923 22:12:37.759182 5175 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 1ec9d8e79ea345f5ba8fb445a3e51d51: Neither blocks nor log segments found. Creating new log.
I20260923 22:12:37.759454 5175 log.cc:824] T 00000000000000000000000000000000 P 1ec9d8e79ea345f5ba8fb445a3e51d51: Log is configured to *not* fsync() on all Append() calls
I20260923 22:12:37.760151 5175 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 1ec9d8e79ea345f5ba8fb445a3e51d51: No bootstrap required, opened a new log
I20260923 22:12:37.761849 5175 raft_consensus.cc:348] T 00000000000000000000000000000000 P 1ec9d8e79ea345f5ba8fb445a3e51d51 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1ec9d8e79ea345f5ba8fb445a3e51d51" member_type: VOTER last_known_addr { host: "127.1.135.254" port: 35103 } }
I20260923 22:12:37.762015 5175 raft_consensus.cc:374] T 00000000000000000000000000000000 P 1ec9d8e79ea345f5ba8fb445a3e51d51 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260923 22:12:37.762048 5175 raft_consensus.cc:729] T 00000000000000000000000000000000 P 1ec9d8e79ea345f5ba8fb445a3e51d51 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 1ec9d8e79ea345f5ba8fb445a3e51d51, State: Initialized, Role: FOLLOWER
I20260923 22:12:37.762161 5175 consensus_queue.cc:260] T 00000000000000000000000000000000 P 1ec9d8e79ea345f5ba8fb445a3e51d51 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1ec9d8e79ea345f5ba8fb445a3e51d51" member_type: VOTER last_known_addr { host: "127.1.135.254" port: 35103 } }
I20260923 22:12:37.762243 5175 raft_consensus.cc:388] T 00000000000000000000000000000000 P 1ec9d8e79ea345f5ba8fb445a3e51d51 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260923 22:12:37.762274 5175 raft_consensus.cc:482] T 00000000000000000000000000000000 P 1ec9d8e79ea345f5ba8fb445a3e51d51 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260923 22:12:37.762319 5175 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 1ec9d8e79ea345f5ba8fb445a3e51d51 [term 0 FOLLOWER]: Advancing to term 1
Sep 23 22:12:37 dist-test-slave-s2fj krb5kdc[5094](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790201557, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.1.135.254@KRBTEST.COM
I20260923 22:12:37.762866 5175 raft_consensus.cc:504] T 00000000000000000000000000000000 P 1ec9d8e79ea345f5ba8fb445a3e51d51 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1ec9d8e79ea345f5ba8fb445a3e51d51" member_type: VOTER last_known_addr { host: "127.1.135.254" port: 35103 } }
I20260923 22:12:37.762971 5175 leader_election.cc:302] T 00000000000000000000000000000000 P 1ec9d8e79ea345f5ba8fb445a3e51d51 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 1ec9d8e79ea345f5ba8fb445a3e51d51; no voters:
I20260923 22:12:37.763123 5175 leader_election.cc:288] T 00000000000000000000000000000000 P 1ec9d8e79ea345f5ba8fb445a3e51d51 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260923 22:12:37.763150 5180 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 1ec9d8e79ea345f5ba8fb445a3e51d51 [term 1 FOLLOWER]: Leader election won for term 1
I20260923 22:12:37.763339 5180 raft_consensus.cc:686] T 00000000000000000000000000000000 P 1ec9d8e79ea345f5ba8fb445a3e51d51 [term 1 LEADER]: Becoming Leader. State: Replica: 1ec9d8e79ea345f5ba8fb445a3e51d51, State: Running, Role: LEADER
I20260923 22:12:37.763417 5180 consensus_queue.cc:237] T 00000000000000000000000000000000 P 1ec9d8e79ea345f5ba8fb445a3e51d51 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1ec9d8e79ea345f5ba8fb445a3e51d51" member_type: VOTER last_known_addr { host: "127.1.135.254" port: 35103 } }
I20260923 22:12:37.763450 5175 sys_catalog.cc:564] T 00000000000000000000000000000000 P 1ec9d8e79ea345f5ba8fb445a3e51d51 [sys.catalog]: configured and running, proceeding with master startup.
I20260923 22:12:37.765399 5181 sys_catalog.cc:455] T 00000000000000000000000000000000 P 1ec9d8e79ea345f5ba8fb445a3e51d51 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "1ec9d8e79ea345f5ba8fb445a3e51d51" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1ec9d8e79ea345f5ba8fb445a3e51d51" member_type: VOTER last_known_addr { host: "127.1.135.254" port: 35103 } } }
I20260923 22:12:37.765381 5182 sys_catalog.cc:455] T 00000000000000000000000000000000 P 1ec9d8e79ea345f5ba8fb445a3e51d51 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 1ec9d8e79ea345f5ba8fb445a3e51d51. Latest consensus state: current_term: 1 leader_uuid: "1ec9d8e79ea345f5ba8fb445a3e51d51" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1ec9d8e79ea345f5ba8fb445a3e51d51" member_type: VOTER last_known_addr { host: "127.1.135.254" port: 35103 } } }
I20260923 22:12:37.765648 5181 sys_catalog.cc:458] T 00000000000000000000000000000000 P 1ec9d8e79ea345f5ba8fb445a3e51d51 [sys.catalog]: This master's current role is: LEADER
I20260923 22:12:37.765655 5182 sys_catalog.cc:458] T 00000000000000000000000000000000 P 1ec9d8e79ea345f5ba8fb445a3e51d51 [sys.catalog]: This master's current role is: LEADER
W20260923 22:12:37.766114 5194 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 1ec9d8e79ea345f5ba8fb445a3e51d51: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260923 22:12:37.766180 5194 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260923 22:12:37.766268 5196 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260923 22:12:37.766430 5196 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260923 22:12:37.767035 5178 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 22:12:37.756585 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52276 (local address 127.1.135.254:35103)
0923 22:12:37.756768 (+ 183us) server_negotiation.cc:206] Beginning negotiation
0923 22:12:37.756776 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0923 22:12:37.756799 (+ 23us) server_negotiation.cc:407] Connection header received
0923 22:12:37.757307 (+ 508us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 22:12:37.757313 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 22:12:37.757357 (+ 44us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 22:12:37.757511 (+ 154us) server_negotiation.cc:226] Negotiated authn=SASL
0923 22:12:37.758112 (+ 601us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 22:12:37.759033 (+ 921us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 22:12:37.759921 (+ 888us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 22:12:37.760456 (+ 535us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 22:12:37.760584 (+ 128us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 22:12:37.762906 (+ 2322us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0923 22:12:37.762910 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0923 22:12:37.762911 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0923 22:12:37.762923 (+ 12us) server_negotiation.cc:922] calling sasl_server_start()
0923 22:12:37.764989 (+ 2066us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0923 22:12:37.765485 (+ 496us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0923 22:12:37.765488 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0923 22:12:37.765488 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0923 22:12:37.765590 (+ 102us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0923 22:12:37.765840 (+ 250us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0923 22:12:37.765841 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0923 22:12:37.765842 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0923 22:12:37.766147 (+ 305us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0923 22:12:37.766637 (+ 490us) server_negotiation.cc:1035] Waiting for connection context
0923 22:12:37.766707 (+ 70us) server_negotiation.cc:299] Negotiation successful
0923 22:12:37.766842 (+ 135us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":137,"thread_start_us":82,"threads_started":1}
I20260923 22:12:37.767752 5196 catalog_manager.cc:1347] Generated new cluster ID: 5612114ebbf94eac8115229ae34e139e
I20260923 22:12:37.767798 5196 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260923 22:12:37.778491 5196 catalog_manager.cc:1370] Generated new certificate authority record
I20260923 22:12:37.779166 5196 catalog_manager.cc:1504] Loading token signing keys...
I20260923 22:12:37.782747 5196 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 1ec9d8e79ea345f5ba8fb445a3e51d51: Generated new TSK 0
I20260923 22:12:37.782905 5196 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.1.135.193@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.1.135.193@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.1.135.193 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskRiQe_a/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790201539513534-1567-0/krb5kdc/kudu_127.1.135.193.keytab.
Entry for principal kudu/127.1.135.193 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskRiQe_a/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790201539513534-1567-0/krb5kdc/kudu_127.1.135.193.keytab.
I20260923 22:12:37.818114 1567 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskRiQe_a/build/release/bin/kudu
/tmp/dist-test-taskRiQe_a/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskRiQe_a/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790201539513534-1567-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskRiQe_a/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790201539513534-1567-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskRiQe_a/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790201539513534-1567-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskRiQe_a/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790201539513534-1567-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.1.135.193:0
--local_ip_for_outbound_sockets=127.1.135.193
--webserver_interface=127.1.135.193
--webserver_port=0
--tserver_master_addrs=127.1.135.254:35103
--keytab_file=/tmp/dist-test-taskRiQe_a/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790201539513534-1567-0/krb5kdc/kudu_127.1.135.193.keytab
--principal=kudu/127.1.135.193
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.1.135.212:43913
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskRiQe_a/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790201539513534-1567-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskRiQe_a/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790201539513534-1567-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskRiQe_a/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790201539513534-1567-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260923 22:12:37.906128 5201 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260923 22:12:37.906324 5201 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260923 22:12:37.906347 5201 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260923 22:12:37.907465 5201 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260923 22:12:37.907506 5201 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260923 22:12:37.907532 5201 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.1.135.193
I20260923 22:12:37.909058 5201 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.1.135.212:43913
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskRiQe_a/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790201539513534-1567-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskRiQe_a/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790201539513534-1567-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskRiQe_a/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790201539513534-1567-0/krb5kdc/kudu_127.1.135.193.keytab
--principal=kudu/127.1.135.193
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.1.135.193:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskRiQe_a/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790201539513534-1567-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.1.135.193
--webserver_port=0
--tserver_master_addrs=127.1.135.254:35103
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.5201
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.1.135.193
--log_dir=/tmp/dist-test-taskRiQe_a/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790201539513534-1567-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 23 Sep 2026 20:53:37 UTC on ccycloud.quasar-rkviyx.root.comops.site
build id 4668083
I20260923 22:12:37.909286 5201 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260923 22:12:37.909526 5201 file_cache.cc:493] Constructed file cache file cache with capacity 419430
I20260923 22:12:37.914471 5201 server_base.cc:1034] running on GCE node
W20260923 22:12:37.914402 5209 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260923 22:12:37.914397 5207 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260923 22:12:37.914397 5206 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260923 22:12:37.914989 5201 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260923 22:12:37.915279 5201 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260923 22:12:37.916419 5201 hybrid_clock.cc:648] HybridClock initialized: now 1790201557916408 us; error 28 us; skew 500 ppm
Sep 23 22:12:37 dist-test-slave-s2fj krb5kdc[5094](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790201557, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.135.193@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260923 22:12:37.918967 5201 init.cc:377] Logged in from keytab as kudu/127.1.135.193@KRBTEST.COM (short username kudu)
I20260923 22:12:37.919409 5201 webserver.cc:460] Webserver started at http://127.1.135.193:38091/ using document root <none> and password file <none>
I20260923 22:12:37.919625 5201 fs_manager.cc:359] Metadata directory not provided
I20260923 22:12:37.919668 5201 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260923 22:12:37.919811 5201 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260923 22:12:37.920949 5201 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskRiQe_a/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790201539513534-1567-0/minicluster-data/ts-0/data/instance:
uuid: "9d8b23503a3c4c0cafbb7af8a311cccd"
format_stamp: "Formatted at 2026-09-23 22:12:37 on dist-test-slave-s2fj"
server_key: "05431904da7d4cc293d09b03fc43e07c"
server_key_iv: "d14ce33323f03db283adab9edc3ef120"
server_key_version: "encryptionkey@0"
I20260923 22:12:37.921312 5201 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskRiQe_a/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790201539513534-1567-0/minicluster-data/ts-0/wal/instance:
uuid: "9d8b23503a3c4c0cafbb7af8a311cccd"
format_stamp: "Formatted at 2026-09-23 22:12:37 on dist-test-slave-s2fj"
server_key: "05431904da7d4cc293d09b03fc43e07c"
server_key_iv: "d14ce33323f03db283adab9edc3ef120"
server_key_version: "encryptionkey@0"
I20260923 22:12:37.922654 5201 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.002s
I20260923 22:12:37.923302 5216 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260923 22:12:37.923487 5201 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.001s
I20260923 22:12:37.923565 5201 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskRiQe_a/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790201539513534-1567-0/minicluster-data/ts-0/data,/tmp/dist-test-taskRiQe_a/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790201539513534-1567-0/minicluster-data/ts-0/wal
uuid: "9d8b23503a3c4c0cafbb7af8a311cccd"
format_stamp: "Formatted at 2026-09-23 22:12:37 on dist-test-slave-s2fj"
server_key: "05431904da7d4cc293d09b03fc43e07c"
server_key_iv: "d14ce33323f03db283adab9edc3ef120"
server_key_version: "encryptionkey@0"
I20260923 22:12:37.923615 5201 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskRiQe_a/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790201539513534-1567-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskRiQe_a/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790201539513534-1567-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskRiQe_a/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790201539513534-1567-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260923 22:12:37.945519 5201 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260923 22:12:37.947696 5201 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260923 22:12:37.947808 5201 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260923 22:12:37.947973 5201 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260923 22:12:37.948293 5201 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260923 22:12:37.948329 5201 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260923 22:12:37.948350 5201 ts_tablet_manager.cc:610] Registered 0 tablets
I20260923 22:12:37.948398 5201 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260923 22:12:37.953827 5201 rpc_server.cc:307] RPC server started. Bound to: 127.1.135.193:33839
I20260923 22:12:37.953902 5329 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.1.135.193:33839 every 8 connection(s)
I20260923 22:12:37.954203 5201 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskRiQe_a/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790201539513534-1567-0/minicluster-data/ts-0/data/info.pb
Sep 23 22:12:37 dist-test-slave-s2fj krb5kdc[5094](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790201557, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.135.193@KRBTEST.COM for kudu/127.1.135.254@KRBTEST.COM
I20260923 22:12:37.962670 5178 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 22:12:37.954921 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.135.193:47147 (local address 127.1.135.254:35103)
0923 22:12:37.954982 (+ 61us) server_negotiation.cc:206] Beginning negotiation
0923 22:12:37.954984 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0923 22:12:37.955207 (+ 223us) server_negotiation.cc:407] Connection header received
0923 22:12:37.956183 (+ 976us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 22:12:37.956184 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 22:12:37.956200 (+ 16us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 22:12:37.956251 (+ 51us) server_negotiation.cc:226] Negotiated authn=SASL
0923 22:12:37.956807 (+ 556us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 22:12:37.957290 (+ 483us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 22:12:37.958200 (+ 910us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 22:12:37.958391 (+ 191us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 22:12:37.958440 (+ 49us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 22:12:37.960409 (+ 1969us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0923 22:12:37.960410 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0923 22:12:37.960410 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0923 22:12:37.960417 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0923 22:12:37.961592 (+ 1175us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0923 22:12:37.961942 (+ 350us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0923 22:12:37.961943 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0923 22:12:37.961943 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0923 22:12:37.961989 (+ 46us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0923 22:12:37.962187 (+ 198us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0923 22:12:37.962188 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0923 22:12:37.962188 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0923 22:12:37.962283 (+ 95us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0923 22:12:37.962333 (+ 50us) server_negotiation.cc:1035] Waiting for connection context
0923 22:12:37.962549 (+ 216us) server_negotiation.cc:299] Negotiation successful
0923 22:12:37.962609 (+ 60us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":37}
I20260923 22:12:37.962872 5332 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 22:12:37.954886 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.135.254:35103 (local address 127.1.135.193:47147)
0923 22:12:37.955057 (+ 171us) negotiation.cc:107] Waiting for socket to connect
0923 22:12:37.955097 (+ 40us) client_negotiation.cc:174] Beginning negotiation
0923 22:12:37.956014 (+ 917us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0923 22:12:37.956314 (+ 300us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0923 22:12:37.956318 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0923 22:12:37.956552 (+ 234us) client_negotiation.cc:189] Negotiated authn=SASL
0923 22:12:37.956728 (+ 176us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 22:12:37.956732 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 22:12:37.957365 (+ 633us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 22:12:37.957365 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 22:12:37.958127 (+ 762us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 22:12:37.958128 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 22:12:37.958459 (+ 331us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 22:12:37.958460 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 22:12:37.958552 (+ 92us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 22:12:37.958958 (+ 406us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0923 22:12:37.958966 (+ 8us) client_negotiation.cc:656] calling sasl_client_start()
0923 22:12:37.960331 (+ 1365us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0923 22:12:37.961664 (+ 1333us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0923 22:12:37.961669 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0923 22:12:37.961670 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0923 22:12:37.961880 (+ 210us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0923 22:12:37.962046 (+ 166us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0923 22:12:37.962047 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0923 22:12:37.962047 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0923 22:12:37.962136 (+ 89us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0923 22:12:37.962336 (+ 200us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0923 22:12:37.962340 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0923 22:12:37.962444 (+ 104us) client_negotiation.cc:769] Sending connection context
0923 22:12:37.962521 (+ 77us) client_negotiation.cc:240] Negotiation successful
0923 22:12:37.962629 (+ 108us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":127,"thread_start_us":84,"threads_started":1}
I20260923 22:12:37.963493 1567 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskRiQe_a/build/release/bin/kudu as pid 5201
I20260923 22:12:37.963596 1567 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskRiQe_a/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790201539513534-1567-0/minicluster-data/ts-0/wal/instance
I20260923 22:12:37.963579 5330 heartbeater.cc:347] Connected to a master server at 127.1.135.254:35103
I20260923 22:12:37.963666 5330 heartbeater.cc:464] Registering TS with master...
I20260923 22:12:37.963793 1567 external_mini_cluster.cc:1428] Setting key 2f69332ef05766e8b9fab129d669ca56
I20260923 22:12:37.963871 5330 heartbeater.cc:511] Master 127.1.135.254:35103 requested a full tablet report, sending...
I20260923 22:12:37.964434 5138 ts_manager.cc:194] Registered new tserver with Master: 9d8b23503a3c4c0cafbb7af8a311cccd (127.1.135.193:33839)
I20260923 22:12:37.965147 5138 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.1.135.193@KRBTEST.COM'} at 127.1.135.193:47147
WARNING: no policy specified for kudu/127.1.135.194@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.1.135.194@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.1.135.194 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskRiQe_a/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790201539513534-1567-0/krb5kdc/kudu_127.1.135.194.keytab.
Entry for principal kudu/127.1.135.194 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskRiQe_a/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790201539513534-1567-0/krb5kdc/kudu_127.1.135.194.keytab.
I20260923 22:12:37.987082 1567 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskRiQe_a/build/release/bin/kudu
/tmp/dist-test-taskRiQe_a/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskRiQe_a/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790201539513534-1567-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskRiQe_a/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790201539513534-1567-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskRiQe_a/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790201539513534-1567-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskRiQe_a/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790201539513534-1567-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.1.135.194:0
--local_ip_for_outbound_sockets=127.1.135.194
--webserver_interface=127.1.135.194
--webserver_port=0
--tserver_master_addrs=127.1.135.254:35103
--keytab_file=/tmp/dist-test-taskRiQe_a/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790201539513534-1567-0/krb5kdc/kudu_127.1.135.194.keytab
--principal=kudu/127.1.135.194
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.1.135.212:43913
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskRiQe_a/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790201539513534-1567-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskRiQe_a/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790201539513534-1567-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskRiQe_a/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790201539513534-1567-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260923 22:12:38.064811 5335 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260923 22:12:38.065018 5335 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260923 22:12:38.065057 5335 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260923 22:12:38.066231 5335 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260923 22:12:38.066293 5335 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260923 22:12:38.066337 5335 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.1.135.194
I20260923 22:12:38.067759 5335 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.1.135.212:43913
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskRiQe_a/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790201539513534-1567-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskRiQe_a/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790201539513534-1567-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskRiQe_a/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790201539513534-1567-0/krb5kdc/kudu_127.1.135.194.keytab
--principal=kudu/127.1.135.194
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.1.135.194:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskRiQe_a/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790201539513534-1567-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.1.135.194
--webserver_port=0
--tserver_master_addrs=127.1.135.254:35103
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.5335
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.1.135.194
--log_dir=/tmp/dist-test-taskRiQe_a/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790201539513534-1567-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 23 Sep 2026 20:53:37 UTC on ccycloud.quasar-rkviyx.root.comops.site
build id 4668083
I20260923 22:12:38.068017 5335 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260923 22:12:38.068264 5335 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260923 22:12:38.072938 5341 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260923 22:12:38.072921 5343 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260923 22:12:38.072930 5340 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260923 22:12:38.073128 5335 server_base.cc:1034] running on GCE node
I20260923 22:12:38.073376 5335 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260923 22:12:38.073592 5335 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260923 22:12:38.074739 5335 hybrid_clock.cc:648] HybridClock initialized: now 1790201558074725 us; error 31 us; skew 500 ppm
Sep 23 22:12:38 dist-test-slave-s2fj krb5kdc[5094](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790201558, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.135.194@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260923 22:12:38.077105 5335 init.cc:377] Logged in from keytab as kudu/127.1.135.194@KRBTEST.COM (short username kudu)
I20260923 22:12:38.077479 5335 webserver.cc:460] Webserver started at http://127.1.135.194:35073/ using document root <none> and password file <none>
I20260923 22:12:38.077694 5335 fs_manager.cc:359] Metadata directory not provided
I20260923 22:12:38.077756 5335 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260923 22:12:38.077893 5335 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260923 22:12:38.078933 5335 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskRiQe_a/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790201539513534-1567-0/minicluster-data/ts-1/data/instance:
uuid: "ec347bb35538475cadf201dc7afc07b8"
format_stamp: "Formatted at 2026-09-23 22:12:38 on dist-test-slave-s2fj"
server_key: "0c53aa207c02e81df9ef35cd8614f9a1"
server_key_iv: "6719e617b78169f162db2f73aded02d0"
server_key_version: "encryptionkey@0"
I20260923 22:12:38.079290 5335 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskRiQe_a/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790201539513534-1567-0/minicluster-data/ts-1/wal/instance:
uuid: "ec347bb35538475cadf201dc7afc07b8"
format_stamp: "Formatted at 2026-09-23 22:12:38 on dist-test-slave-s2fj"
server_key: "0c53aa207c02e81df9ef35cd8614f9a1"
server_key_iv: "6719e617b78169f162db2f73aded02d0"
server_key_version: "encryptionkey@0"
I20260923 22:12:38.080636 5335 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260923 22:12:38.081275 5350 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260923 22:12:38.081432 5335 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260923 22:12:38.081522 5335 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskRiQe_a/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790201539513534-1567-0/minicluster-data/ts-1/data,/tmp/dist-test-taskRiQe_a/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790201539513534-1567-0/minicluster-data/ts-1/wal
uuid: "ec347bb35538475cadf201dc7afc07b8"
format_stamp: "Formatted at 2026-09-23 22:12:38 on dist-test-slave-s2fj"
server_key: "0c53aa207c02e81df9ef35cd8614f9a1"
server_key_iv: "6719e617b78169f162db2f73aded02d0"
server_key_version: "encryptionkey@0"
I20260923 22:12:38.081598 5335 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskRiQe_a/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790201539513534-1567-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskRiQe_a/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790201539513534-1567-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskRiQe_a/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790201539513534-1567-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260923 22:12:38.104841 5335 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260923 22:12:38.106570 5335 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260923 22:12:38.106729 5335 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260923 22:12:38.106946 5335 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260923 22:12:38.107228 5335 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260923 22:12:38.107287 5335 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260923 22:12:38.107321 5335 ts_tablet_manager.cc:610] Registered 0 tablets
I20260923 22:12:38.107347 5335 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260923 22:12:38.113116 5335 rpc_server.cc:307] RPC server started. Bound to: 127.1.135.194:38395
I20260923 22:12:38.113161 5463 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.1.135.194:38395 every 8 connection(s)
I20260923 22:12:38.113445 5335 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskRiQe_a/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790201539513534-1567-0/minicluster-data/ts-1/data/info.pb
Sep 23 22:12:38 dist-test-slave-s2fj krb5kdc[5094](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790201558, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.135.194@KRBTEST.COM for kudu/127.1.135.254@KRBTEST.COM
I20260923 22:12:38.121874 1567 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskRiQe_a/build/release/bin/kudu as pid 5335
I20260923 22:12:38.121979 1567 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskRiQe_a/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790201539513534-1567-0/minicluster-data/ts-1/wal/instance
I20260923 22:12:38.122005 5178 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 22:12:38.114100 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.135.194:57857 (local address 127.1.135.254:35103)
0923 22:12:38.114153 (+ 53us) server_negotiation.cc:206] Beginning negotiation
0923 22:12:38.114155 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0923 22:12:38.114369 (+ 214us) server_negotiation.cc:407] Connection header received
0923 22:12:38.115289 (+ 920us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 22:12:38.115290 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 22:12:38.115299 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 22:12:38.115350 (+ 51us) server_negotiation.cc:226] Negotiated authn=SASL
0923 22:12:38.115882 (+ 532us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 22:12:38.116421 (+ 539us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 22:12:38.117331 (+ 910us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 22:12:38.117506 (+ 175us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 22:12:38.117558 (+ 52us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 22:12:38.119377 (+ 1819us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0923 22:12:38.119378 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0923 22:12:38.119379 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0923 22:12:38.119386 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0923 22:12:38.120767 (+ 1381us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0923 22:12:38.121144 (+ 377us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0923 22:12:38.121145 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0923 22:12:38.121146 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0923 22:12:38.121194 (+ 48us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0923 22:12:38.121395 (+ 201us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0923 22:12:38.121395 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0923 22:12:38.121396 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0923 22:12:38.121487 (+ 91us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0923 22:12:38.121537 (+ 50us) server_negotiation.cc:1035] Waiting for connection context
0923 22:12:38.121820 (+ 283us) server_negotiation.cc:299] Negotiation successful
0923 22:12:38.121907 (+ 87us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":26}
I20260923 22:12:38.122177 1567 external_mini_cluster.cc:1428] Setting key 2679800a5628c237d3c51fe7ac3ed38b
I20260923 22:12:38.122231 5466 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 22:12:38.114075 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.135.254:35103 (local address 127.1.135.194:57857)
0923 22:12:38.114232 (+ 157us) negotiation.cc:107] Waiting for socket to connect
0923 22:12:38.114267 (+ 35us) client_negotiation.cc:174] Beginning negotiation
0923 22:12:38.115198 (+ 931us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0923 22:12:38.115398 (+ 200us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0923 22:12:38.115401 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0923 22:12:38.115625 (+ 224us) client_negotiation.cc:189] Negotiated authn=SASL
0923 22:12:38.115808 (+ 183us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 22:12:38.115811 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 22:12:38.116493 (+ 682us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 22:12:38.116494 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 22:12:38.117261 (+ 767us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 22:12:38.117262 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 22:12:38.117573 (+ 311us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 22:12:38.117574 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 22:12:38.117662 (+ 88us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 22:12:38.118036 (+ 374us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0923 22:12:38.118043 (+ 7us) client_negotiation.cc:656] calling sasl_client_start()
0923 22:12:38.119275 (+ 1232us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0923 22:12:38.120844 (+ 1569us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0923 22:12:38.120862 (+ 18us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0923 22:12:38.120863 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0923 22:12:38.121080 (+ 217us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0923 22:12:38.121254 (+ 174us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0923 22:12:38.121255 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0923 22:12:38.121255 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0923 22:12:38.121342 (+ 87us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0923 22:12:38.121540 (+ 198us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0923 22:12:38.121544 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0923 22:12:38.121641 (+ 97us) client_negotiation.cc:769] Sending connection context
0923 22:12:38.121779 (+ 138us) client_negotiation.cc:240] Negotiation successful
0923 22:12:38.121982 (+ 203us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":122,"thread_start_us":86,"threads_started":1}
I20260923 22:12:38.122720 5464 heartbeater.cc:347] Connected to a master server at 127.1.135.254:35103
I20260923 22:12:38.122809 5464 heartbeater.cc:464] Registering TS with master...
I20260923 22:12:38.123003 5464 heartbeater.cc:511] Master 127.1.135.254:35103 requested a full tablet report, sending...
I20260923 22:12:38.123356 5138 ts_manager.cc:194] Registered new tserver with Master: ec347bb35538475cadf201dc7afc07b8 (127.1.135.194:38395)
I20260923 22:12:38.123719 5138 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.1.135.194@KRBTEST.COM'} at 127.1.135.194:57857
WARNING: no policy specified for kudu/127.1.135.195@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.1.135.195@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.1.135.195 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskRiQe_a/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790201539513534-1567-0/krb5kdc/kudu_127.1.135.195.keytab.
Entry for principal kudu/127.1.135.195 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskRiQe_a/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790201539513534-1567-0/krb5kdc/kudu_127.1.135.195.keytab.
I20260923 22:12:38.146268 1567 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskRiQe_a/build/release/bin/kudu
/tmp/dist-test-taskRiQe_a/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskRiQe_a/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790201539513534-1567-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskRiQe_a/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790201539513534-1567-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskRiQe_a/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790201539513534-1567-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskRiQe_a/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790201539513534-1567-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.1.135.195:0
--local_ip_for_outbound_sockets=127.1.135.195
--webserver_interface=127.1.135.195
--webserver_port=0
--tserver_master_addrs=127.1.135.254:35103
--keytab_file=/tmp/dist-test-taskRiQe_a/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790201539513534-1567-0/krb5kdc/kudu_127.1.135.195.keytab
--principal=kudu/127.1.135.195
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.1.135.212:43913
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskRiQe_a/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790201539513534-1567-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskRiQe_a/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790201539513534-1567-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskRiQe_a/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790201539513534-1567-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260923 22:12:38.224128 5469 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260923 22:12:38.224318 5469 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260923 22:12:38.224339 5469 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260923 22:12:38.225419 5469 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260923 22:12:38.225454 5469 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260923 22:12:38.225481 5469 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.1.135.195
I20260923 22:12:38.226948 5469 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.1.135.212:43913
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskRiQe_a/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790201539513534-1567-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskRiQe_a/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790201539513534-1567-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskRiQe_a/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790201539513534-1567-0/krb5kdc/kudu_127.1.135.195.keytab
--principal=kudu/127.1.135.195
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.1.135.195:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskRiQe_a/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790201539513534-1567-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.1.135.195
--webserver_port=0
--tserver_master_addrs=127.1.135.254:35103
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.5469
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.1.135.195
--log_dir=/tmp/dist-test-taskRiQe_a/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790201539513534-1567-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 23 Sep 2026 20:53:37 UTC on ccycloud.quasar-rkviyx.root.comops.site
build id 4668083
I20260923 22:12:38.227150 5469 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260923 22:12:38.227344 5469 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260923 22:12:38.231899 5474 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260923 22:12:38.231904 5475 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260923 22:12:38.232227 5469 server_base.cc:1034] running on GCE node
W20260923 22:12:38.232024 5477 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260923 22:12:38.232558 5469 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260923 22:12:38.232781 5469 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260923 22:12:38.233917 5469 hybrid_clock.cc:648] HybridClock initialized: now 1790201558233905 us; error 27 us; skew 500 ppm
Sep 23 22:12:38 dist-test-slave-s2fj krb5kdc[5094](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790201558, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.135.195@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260923 22:12:38.236284 5469 init.cc:377] Logged in from keytab as kudu/127.1.135.195@KRBTEST.COM (short username kudu)
I20260923 22:12:38.236732 5469 webserver.cc:460] Webserver started at http://127.1.135.195:42117/ using document root <none> and password file <none>
I20260923 22:12:38.236949 5469 fs_manager.cc:359] Metadata directory not provided
I20260923 22:12:38.237025 5469 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260923 22:12:38.237188 5469 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260923 22:12:38.238166 5469 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskRiQe_a/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790201539513534-1567-0/minicluster-data/ts-2/data/instance:
uuid: "fe39069904a64e74a92da5ccc5a14a11"
format_stamp: "Formatted at 2026-09-23 22:12:38 on dist-test-slave-s2fj"
server_key: "2669b971d61a1802833fe3b93bc0222a"
server_key_iv: "d3b541aaa6e4ce26159255643f97730c"
server_key_version: "encryptionkey@0"
I20260923 22:12:38.238510 5469 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskRiQe_a/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790201539513534-1567-0/minicluster-data/ts-2/wal/instance:
uuid: "fe39069904a64e74a92da5ccc5a14a11"
format_stamp: "Formatted at 2026-09-23 22:12:38 on dist-test-slave-s2fj"
server_key: "2669b971d61a1802833fe3b93bc0222a"
server_key_iv: "d3b541aaa6e4ce26159255643f97730c"
server_key_version: "encryptionkey@0"
I20260923 22:12:38.239784 5469 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.002s
I20260923 22:12:38.240532 5484 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260923 22:12:38.240696 5469 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.000s
I20260923 22:12:38.240794 5469 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskRiQe_a/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790201539513534-1567-0/minicluster-data/ts-2/data,/tmp/dist-test-taskRiQe_a/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790201539513534-1567-0/minicluster-data/ts-2/wal
uuid: "fe39069904a64e74a92da5ccc5a14a11"
format_stamp: "Formatted at 2026-09-23 22:12:38 on dist-test-slave-s2fj"
server_key: "2669b971d61a1802833fe3b93bc0222a"
server_key_iv: "d3b541aaa6e4ce26159255643f97730c"
server_key_version: "encryptionkey@0"
I20260923 22:12:38.240875 5469 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskRiQe_a/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790201539513534-1567-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskRiQe_a/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790201539513534-1567-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskRiQe_a/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790201539513534-1567-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260923 22:12:38.269443 5469 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260923 22:12:38.271773 5469 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260923 22:12:38.271947 5469 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260923 22:12:38.272222 5469 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260923 22:12:38.272527 5469 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260923 22:12:38.272583 5469 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260923 22:12:38.272624 5469 ts_tablet_manager.cc:610] Registered 0 tablets
I20260923 22:12:38.272651 5469 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260923 22:12:38.278326 5469 rpc_server.cc:307] RPC server started. Bound to: 127.1.135.195:36069
I20260923 22:12:38.278367 5597 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.1.135.195:36069 every 8 connection(s)
I20260923 22:12:38.278680 5469 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskRiQe_a/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790201539513534-1567-0/minicluster-data/ts-2/data/info.pb
I20260923 22:12:38.280949 1567 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskRiQe_a/build/release/bin/kudu as pid 5469
I20260923 22:12:38.281044 1567 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskRiQe_a/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790201539513534-1567-0/minicluster-data/ts-2/wal/instance
I20260923 22:12:38.281260 1567 external_mini_cluster.cc:1428] Setting key 0c43935bfc303228a915c99311ea0800
Sep 23 22:12:38 dist-test-slave-s2fj krb5kdc[5094](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790201558, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.135.195@KRBTEST.COM for kudu/127.1.135.254@KRBTEST.COM
I20260923 22:12:38.287760 5178 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 22:12:38.279361 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.135.195:45983 (local address 127.1.135.254:35103)
0923 22:12:38.279439 (+ 78us) server_negotiation.cc:206] Beginning negotiation
0923 22:12:38.279441 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0923 22:12:38.279576 (+ 135us) server_negotiation.cc:407] Connection header received
0923 22:12:38.280545 (+ 969us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 22:12:38.280546 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 22:12:38.280557 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 22:12:38.280603 (+ 46us) server_negotiation.cc:226] Negotiated authn=SASL
0923 22:12:38.281290 (+ 687us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 22:12:38.282028 (+ 738us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 22:12:38.283150 (+ 1122us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 22:12:38.283400 (+ 250us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 22:12:38.283453 (+ 53us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 22:12:38.285581 (+ 2128us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0923 22:12:38.285582 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0923 22:12:38.285583 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0923 22:12:38.285588 (+ 5us) server_negotiation.cc:922] calling sasl_server_start()
0923 22:12:38.286708 (+ 1120us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0923 22:12:38.287041 (+ 333us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0923 22:12:38.287041 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0923 22:12:38.287042 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0923 22:12:38.287087 (+ 45us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0923 22:12:38.287288 (+ 201us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0923 22:12:38.287289 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0923 22:12:38.287289 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0923 22:12:38.287378 (+ 89us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0923 22:12:38.287426 (+ 48us) server_negotiation.cc:1035] Waiting for connection context
0923 22:12:38.287641 (+ 215us) server_negotiation.cc:299] Negotiation successful
0923 22:12:38.287696 (+ 55us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":48}
I20260923 22:12:38.287985 5600 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 22:12:38.279319 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.135.254:35103 (local address 127.1.135.195:45983)
0923 22:12:38.279463 (+ 144us) negotiation.cc:107] Waiting for socket to connect
0923 22:12:38.279496 (+ 33us) client_negotiation.cc:174] Beginning negotiation
0923 22:12:38.280458 (+ 962us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0923 22:12:38.280709 (+ 251us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0923 22:12:38.280712 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0923 22:12:38.280950 (+ 238us) client_negotiation.cc:189] Negotiated authn=SASL
0923 22:12:38.281199 (+ 249us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 22:12:38.281204 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 22:12:38.282112 (+ 908us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 22:12:38.282113 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 22:12:38.283076 (+ 963us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 22:12:38.283078 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 22:12:38.283467 (+ 389us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 22:12:38.283468 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 22:12:38.283567 (+ 99us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 22:12:38.284099 (+ 532us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0923 22:12:38.284108 (+ 9us) client_negotiation.cc:656] calling sasl_client_start()
0923 22:12:38.285504 (+ 1396us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0923 22:12:38.286776 (+ 1272us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0923 22:12:38.286781 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0923 22:12:38.286783 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0923 22:12:38.286979 (+ 196us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0923 22:12:38.287142 (+ 163us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0923 22:12:38.287142 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0923 22:12:38.287143 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0923 22:12:38.287235 (+ 92us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0923 22:12:38.287431 (+ 196us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0923 22:12:38.287435 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0923 22:12:38.287537 (+ 102us) client_negotiation.cc:769] Sending connection context
0923 22:12:38.287615 (+ 78us) client_negotiation.cc:240] Negotiation successful
0923 22:12:38.287737 (+ 122us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":110,"thread_start_us":89,"threads_started":1}
I20260923 22:12:38.288542 5598 heartbeater.cc:347] Connected to a master server at 127.1.135.254:35103
I20260923 22:12:38.288645 5598 heartbeater.cc:464] Registering TS with master...
I20260923 22:12:38.288813 5598 heartbeater.cc:511] Master 127.1.135.254:35103 requested a full tablet report, sending...
I20260923 22:12:38.289162 5138 ts_manager.cc:194] Registered new tserver with Master: fe39069904a64e74a92da5ccc5a14a11 (127.1.135.195:36069)
I20260923 22:12:38.289536 5138 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.1.135.195@KRBTEST.COM'} at 127.1.135.195:45983
I20260923 22:12:38.292794 1567 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
Sep 23 22:12:38 dist-test-slave-s2fj krb5kdc[5094](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790201558, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
Sep 23 22:12:38 dist-test-slave-s2fj krb5kdc[5094](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790201558, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.1.135.193@KRBTEST.COM
I20260923 22:12:38.315006 5605 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 22:12:38.307430 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39040 (local address 127.1.135.193:33839)
0923 22:12:38.307588 (+ 158us) server_negotiation.cc:206] Beginning negotiation
0923 22:12:38.307592 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0923 22:12:38.307600 (+ 8us) server_negotiation.cc:407] Connection header received
0923 22:12:38.307621 (+ 21us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 22:12:38.307623 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 22:12:38.307636 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 22:12:38.307720 (+ 84us) server_negotiation.cc:226] Negotiated authn=SASL
0923 22:12:38.308341 (+ 621us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 22:12:38.309054 (+ 713us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 22:12:38.309810 (+ 756us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 22:12:38.310116 (+ 306us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 22:12:38.310207 (+ 91us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 22:12:38.312422 (+ 2215us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0923 22:12:38.312431 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0923 22:12:38.312432 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0923 22:12:38.312442 (+ 10us) server_negotiation.cc:922] calling sasl_server_start()
0923 22:12:38.313780 (+ 1338us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0923 22:12:38.314145 (+ 365us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0923 22:12:38.314148 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0923 22:12:38.314148 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0923 22:12:38.314217 (+ 69us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0923 22:12:38.314433 (+ 216us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0923 22:12:38.314434 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0923 22:12:38.314435 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0923 22:12:38.314560 (+ 125us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0923 22:12:38.314730 (+ 170us) server_negotiation.cc:1035] Waiting for connection context
0923 22:12:38.314819 (+ 89us) server_negotiation.cc:299] Negotiation successful
0923 22:12:38.314896 (+ 77us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":129,"thread_start_us":75,"threads_started":1}
Sep 23 22:12:38 dist-test-slave-s2fj krb5kdc[5094](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790201558, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.1.135.254@KRBTEST.COM
I20260923 22:12:38.322412 5178 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 22:12:38.316197 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52292 (local address 127.1.135.254:35103)
0923 22:12:38.316264 (+ 67us) server_negotiation.cc:206] Beginning negotiation
0923 22:12:38.316266 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0923 22:12:38.316330 (+ 64us) server_negotiation.cc:407] Connection header received
0923 22:12:38.316379 (+ 49us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 22:12:38.316379 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 22:12:38.316387 (+ 8us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 22:12:38.316433 (+ 46us) server_negotiation.cc:226] Negotiated authn=SASL
0923 22:12:38.316849 (+ 416us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 22:12:38.317292 (+ 443us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 22:12:38.317901 (+ 609us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 22:12:38.318075 (+ 174us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 22:12:38.318122 (+ 47us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 22:12:38.320258 (+ 2136us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0923 22:12:38.320259 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0923 22:12:38.320259 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0923 22:12:38.320264 (+ 5us) server_negotiation.cc:922] calling sasl_server_start()
0923 22:12:38.321465 (+ 1201us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0923 22:12:38.321744 (+ 279us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0923 22:12:38.321745 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0923 22:12:38.321745 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0923 22:12:38.321789 (+ 44us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0923 22:12:38.321923 (+ 134us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0923 22:12:38.321924 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0923 22:12:38.321924 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0923 22:12:38.322034 (+ 110us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0923 22:12:38.322096 (+ 62us) server_negotiation.cc:1035] Waiting for connection context
0923 22:12:38.322240 (+ 144us) server_negotiation.cc:299] Negotiation successful
0923 22:12:38.322314 (+ 74us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":48}
W20260923 22:12:38.322613 5138 server_base.cc:1116] Unauthorized access attempt to method kudu.master.MasterService.TSHeartbeat from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:52292
I20260923 22:12:38.322902 1567 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskRiQe_a/build/release/bin/kudu with pid 5201
I20260923 22:12:38.327626 1567 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskRiQe_a/build/release/bin/kudu with pid 5335
I20260923 22:12:38.332013 1567 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskRiQe_a/build/release/bin/kudu with pid 5469
I20260923 22:12:38.336447 1567 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskRiQe_a/build/release/bin/kudu with pid 5107
2026-09-23T22:12:38Z chronyd exiting
[ OK ] SecurityITest.TestAuthorizedSuperuser (1836 ms)
[ RUN ] SecurityITest.TestDisableWebUI
Loading random data
Initializing database '/tmp/dist-test-taskRiQe_a/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790201539513534-1567-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Sep 23 22:12:38 dist-test-slave-s2fj krb5kdc[5610](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Sep 23 22:12:38 dist-test-slave-s2fj krb5kdc[5610](info): set up 2 sockets
Sep 23 22:12:38 dist-test-slave-s2fj krb5kdc[5610](info): commencing operation
krb5kdc: starting...
W20260923 22:12:39.363910 1567 mini_kdc.cc:121] Time spent starting KDC: real 1.008s user 0.002s sys 0.003s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskRiQe_a/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790201539513534-1567-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskRiQe_a/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790201539513534-1567-0/krb5kdc/test-user.keytab.
Sep 23 22:12:39 dist-test-slave-s2fj krb5kdc[5610](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790201559, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-09-23T22:12:39Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-09-23T22:12:39Z Disabled control of system clock
WARNING: no policy specified for kudu/127.1.135.254@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.1.135.254@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.1.135.254 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskRiQe_a/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790201539513534-1567-0/krb5kdc/kudu_127.1.135.254.keytab.
Entry for principal kudu/127.1.135.254 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskRiQe_a/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790201539513534-1567-0/krb5kdc/kudu_127.1.135.254.keytab.
I20260923 22:12:39.476593 1567 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskRiQe_a/build/release/bin/kudu
/tmp/dist-test-taskRiQe_a/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskRiQe_a/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790201539513534-1567-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskRiQe_a/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790201539513534-1567-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskRiQe_a/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790201539513534-1567-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskRiQe_a/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790201539513534-1567-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.1.135.254:40585
--webserver_interface=127.1.135.254
--webserver_port=0
--keytab_file=/tmp/dist-test-taskRiQe_a/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790201539513534-1567-0/krb5kdc/kudu_127.1.135.254.keytab
--principal=kudu/127.1.135.254
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.1.135.212:39009
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.1.135.254:40585
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--webserver_enabled=0 with env {KRB5CCNAME=/tmp/dist-test-taskRiQe_a/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790201539513534-1567-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskRiQe_a/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790201539513534-1567-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskRiQe_a/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790201539513534-1567-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260923 22:12:39.554576 5623 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260923 22:12:39.554759 5623 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260923 22:12:39.554780 5623 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260923 22:12:39.555800 5623 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260923 22:12:39.555835 5623 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260923 22:12:39.555850 5623 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260923 22:12:39.555861 5623 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260923 22:12:39.555872 5623 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260923 22:12:39.557395 5623 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.1.135.212:39009
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskRiQe_a/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790201539513534-1567-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskRiQe_a/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790201539513534-1567-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.1.135.254:40585
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskRiQe_a/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790201539513534-1567-0/krb5kdc/kudu_127.1.135.254.keytab
--principal=kudu/127.1.135.254
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.1.135.254:40585
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--webserver_enabled=false
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskRiQe_a/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790201539513534-1567-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.1.135.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.5623
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskRiQe_a/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790201539513534-1567-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 23 Sep 2026 20:53:37 UTC on ccycloud.quasar-rkviyx.root.comops.site
build id 4668083
I20260923 22:12:39.557564 5623 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260923 22:12:39.557721 5623 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260923 22:12:39.562521 5629 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260923 22:12:39.562500 5628 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260923 22:12:39.562517 5623 server_base.cc:1034] running on GCE node
W20260923 22:12:39.562500 5631 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260923 22:12:39.562960 5623 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260923 22:12:39.563239 5623 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260923 22:12:39.564396 5623 hybrid_clock.cc:648] HybridClock initialized: now 1790201559564376 us; error 32 us; skew 500 ppm
Sep 23 22:12:39 dist-test-slave-s2fj krb5kdc[5610](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790201559, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.135.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260923 22:12:39.566798 5623 init.cc:377] Logged in from keytab as kudu/127.1.135.254@KRBTEST.COM (short username kudu)
I20260923 22:12:39.567137 5623 fs_manager.cc:359] Metadata directory not provided
I20260923 22:12:39.567191 5623 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260923 22:12:39.567278 5623 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260923 22:12:39.568288 5623 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskRiQe_a/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790201539513534-1567-0/minicluster-data/master-0/data/instance:
uuid: "8c7f315fc0a14568b2c45267e7193357"
format_stamp: "Formatted at 2026-09-23 22:12:39 on dist-test-slave-s2fj"
server_key: "82dbb99cf367ac37d5cdfd15776a2a7c"
server_key_iv: "3ec856405a665153c104e6da647a84a8"
server_key_version: "encryptionkey@0"
I20260923 22:12:39.568579 5623 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskRiQe_a/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790201539513534-1567-0/minicluster-data/master-0/wal/instance:
uuid: "8c7f315fc0a14568b2c45267e7193357"
format_stamp: "Formatted at 2026-09-23 22:12:39 on dist-test-slave-s2fj"
server_key: "82dbb99cf367ac37d5cdfd15776a2a7c"
server_key_iv: "3ec856405a665153c104e6da647a84a8"
server_key_version: "encryptionkey@0"
I20260923 22:12:39.569751 5623 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.001s sys 0.000s
I20260923 22:12:39.570420 5637 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260923 22:12:39.570616 5623 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260923 22:12:39.570708 5623 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskRiQe_a/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790201539513534-1567-0/minicluster-data/master-0/data,/tmp/dist-test-taskRiQe_a/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790201539513534-1567-0/minicluster-data/master-0/wal
uuid: "8c7f315fc0a14568b2c45267e7193357"
format_stamp: "Formatted at 2026-09-23 22:12:39 on dist-test-slave-s2fj"
server_key: "82dbb99cf367ac37d5cdfd15776a2a7c"
server_key_iv: "3ec856405a665153c104e6da647a84a8"
server_key_version: "encryptionkey@0"
I20260923 22:12:39.570762 5623 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskRiQe_a/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790201539513534-1567-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskRiQe_a/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790201539513534-1567-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskRiQe_a/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790201539513534-1567-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260923 22:12:39.579483 5623 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260923 22:12:39.581588 5623 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260923 22:12:39.581732 5623 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260923 22:12:39.585404 5623 rpc_server.cc:307] RPC server started. Bound to: 127.1.135.254:40585
I20260923 22:12:39.585428 5689 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.1.135.254:40585 every 8 connection(s)
I20260923 22:12:39.585690 5623 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskRiQe_a/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790201539513534-1567-0/minicluster-data/master-0/data/info.pb
I20260923 22:12:39.586575 5690 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260923 22:12:39.588798 5690 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 8c7f315fc0a14568b2c45267e7193357: Bootstrap starting.
I20260923 22:12:39.589355 5690 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 8c7f315fc0a14568b2c45267e7193357: Neither blocks nor log segments found. Creating new log.
I20260923 22:12:39.589643 5690 log.cc:824] T 00000000000000000000000000000000 P 8c7f315fc0a14568b2c45267e7193357: Log is configured to *not* fsync() on all Append() calls
I20260923 22:12:39.590272 5690 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 8c7f315fc0a14568b2c45267e7193357: No bootstrap required, opened a new log
I20260923 22:12:39.590634 1567 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskRiQe_a/build/release/bin/kudu as pid 5623
I20260923 22:12:39.590715 1567 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskRiQe_a/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790201539513534-1567-0/minicluster-data/master-0/wal/instance
I20260923 22:12:39.590926 1567 external_mini_cluster.cc:1428] Setting key a8f193b6d94d861dffe7d73f5d400056
I20260923 22:12:39.592190 5690 raft_consensus.cc:348] T 00000000000000000000000000000000 P 8c7f315fc0a14568b2c45267e7193357 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8c7f315fc0a14568b2c45267e7193357" member_type: VOTER last_known_addr { host: "127.1.135.254" port: 40585 } }
I20260923 22:12:39.592339 5690 raft_consensus.cc:374] T 00000000000000000000000000000000 P 8c7f315fc0a14568b2c45267e7193357 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260923 22:12:39.592387 5690 raft_consensus.cc:729] T 00000000000000000000000000000000 P 8c7f315fc0a14568b2c45267e7193357 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 8c7f315fc0a14568b2c45267e7193357, State: Initialized, Role: FOLLOWER
I20260923 22:12:39.592521 5690 consensus_queue.cc:260] T 00000000000000000000000000000000 P 8c7f315fc0a14568b2c45267e7193357 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8c7f315fc0a14568b2c45267e7193357" member_type: VOTER last_known_addr { host: "127.1.135.254" port: 40585 } }
I20260923 22:12:39.592617 5690 raft_consensus.cc:388] T 00000000000000000000000000000000 P 8c7f315fc0a14568b2c45267e7193357 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260923 22:12:39.592657 5690 raft_consensus.cc:482] T 00000000000000000000000000000000 P 8c7f315fc0a14568b2c45267e7193357 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260923 22:12:39.592706 5690 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 8c7f315fc0a14568b2c45267e7193357 [term 0 FOLLOWER]: Advancing to term 1
I20260923 22:12:39.593196 5690 raft_consensus.cc:504] T 00000000000000000000000000000000 P 8c7f315fc0a14568b2c45267e7193357 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8c7f315fc0a14568b2c45267e7193357" member_type: VOTER last_known_addr { host: "127.1.135.254" port: 40585 } }
I20260923 22:12:39.593314 5690 leader_election.cc:302] T 00000000000000000000000000000000 P 8c7f315fc0a14568b2c45267e7193357 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 8c7f315fc0a14568b2c45267e7193357; no voters:
I20260923 22:12:39.593498 5690 leader_election.cc:288] T 00000000000000000000000000000000 P 8c7f315fc0a14568b2c45267e7193357 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260923 22:12:39.593533 5695 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 8c7f315fc0a14568b2c45267e7193357 [term 1 FOLLOWER]: Leader election won for term 1
I20260923 22:12:39.593662 5695 raft_consensus.cc:686] T 00000000000000000000000000000000 P 8c7f315fc0a14568b2c45267e7193357 [term 1 LEADER]: Becoming Leader. State: Replica: 8c7f315fc0a14568b2c45267e7193357, State: Running, Role: LEADER
I20260923 22:12:39.593736 5695 consensus_queue.cc:237] T 00000000000000000000000000000000 P 8c7f315fc0a14568b2c45267e7193357 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8c7f315fc0a14568b2c45267e7193357" member_type: VOTER last_known_addr { host: "127.1.135.254" port: 40585 } }
I20260923 22:12:39.593789 5690 sys_catalog.cc:564] T 00000000000000000000000000000000 P 8c7f315fc0a14568b2c45267e7193357 [sys.catalog]: configured and running, proceeding with master startup.
I20260923 22:12:39.594017 5695 sys_catalog.cc:455] T 00000000000000000000000000000000 P 8c7f315fc0a14568b2c45267e7193357 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 8c7f315fc0a14568b2c45267e7193357. Latest consensus state: current_term: 1 leader_uuid: "8c7f315fc0a14568b2c45267e7193357" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8c7f315fc0a14568b2c45267e7193357" member_type: VOTER last_known_addr { host: "127.1.135.254" port: 40585 } } }
I20260923 22:12:39.594112 5695 sys_catalog.cc:458] T 00000000000000000000000000000000 P 8c7f315fc0a14568b2c45267e7193357 [sys.catalog]: This master's current role is: LEADER
I20260923 22:12:39.594026 5697 sys_catalog.cc:455] T 00000000000000000000000000000000 P 8c7f315fc0a14568b2c45267e7193357 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "8c7f315fc0a14568b2c45267e7193357" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8c7f315fc0a14568b2c45267e7193357" member_type: VOTER last_known_addr { host: "127.1.135.254" port: 40585 } } }
I20260923 22:12:39.594266 5697 sys_catalog.cc:458] T 00000000000000000000000000000000 P 8c7f315fc0a14568b2c45267e7193357 [sys.catalog]: This master's current role is: LEADER
W20260923 22:12:39.595530 5710 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 8c7f315fc0a14568b2c45267e7193357: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260923 22:12:39.595587 5710 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260923 22:12:39.595760 5700 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260923 22:12:39.595933 5700 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260923 22:12:39.597431 5700 catalog_manager.cc:1347] Generated new cluster ID: fa2c512f34d64199ac4ee44af2abeb13
I20260923 22:12:39.597481 5700 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
Sep 23 22:12:39 dist-test-slave-s2fj krb5kdc[5610](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790201559, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.1.135.254@KRBTEST.COM
I20260923 22:12:39.602887 5694 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 22:12:39.591456 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40060 (local address 127.1.135.254:40585)
0923 22:12:39.591628 (+ 172us) server_negotiation.cc:206] Beginning negotiation
0923 22:12:39.591635 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0923 22:12:39.591657 (+ 22us) server_negotiation.cc:407] Connection header received
0923 22:12:39.592162 (+ 505us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 22:12:39.592168 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 22:12:39.592212 (+ 44us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 22:12:39.592339 (+ 127us) server_negotiation.cc:226] Negotiated authn=SASL
0923 22:12:39.593093 (+ 754us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 22:12:39.594424 (+ 1331us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 22:12:39.595204 (+ 780us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 22:12:39.595961 (+ 757us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 22:12:39.596392 (+ 431us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 22:12:39.598916 (+ 2524us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0923 22:12:39.598920 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0923 22:12:39.598921 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0923 22:12:39.598931 (+ 10us) server_negotiation.cc:922] calling sasl_server_start()
0923 22:12:39.600849 (+ 1918us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0923 22:12:39.601254 (+ 405us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0923 22:12:39.601257 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0923 22:12:39.601258 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0923 22:12:39.601372 (+ 114us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0923 22:12:39.601585 (+ 213us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0923 22:12:39.601586 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0923 22:12:39.601586 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0923 22:12:39.601866 (+ 280us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0923 22:12:39.602450 (+ 584us) server_negotiation.cc:1035] Waiting for connection context
0923 22:12:39.602534 (+ 84us) server_negotiation.cc:299] Negotiation successful
0923 22:12:39.602690 (+ 156us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":133,"thread_start_us":79,"threads_started":1}
I20260923 22:12:39.630900 5700 catalog_manager.cc:1370] Generated new certificate authority record
I20260923 22:12:39.631457 5700 catalog_manager.cc:1504] Loading token signing keys...
I20260923 22:12:39.634222 5700 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 8c7f315fc0a14568b2c45267e7193357: Generated new TSK 0
I20260923 22:12:39.634405 5700 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.1.135.193@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.1.135.193@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.1.135.193 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskRiQe_a/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790201539513534-1567-0/krb5kdc/kudu_127.1.135.193.keytab.
Entry for principal kudu/127.1.135.193 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskRiQe_a/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790201539513534-1567-0/krb5kdc/kudu_127.1.135.193.keytab.
I20260923 22:12:39.664659 1567 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskRiQe_a/build/release/bin/kudu
/tmp/dist-test-taskRiQe_a/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskRiQe_a/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790201539513534-1567-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskRiQe_a/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790201539513534-1567-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskRiQe_a/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790201539513534-1567-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskRiQe_a/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790201539513534-1567-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.1.135.193:0
--local_ip_for_outbound_sockets=127.1.135.193
--webserver_interface=127.1.135.193
--webserver_port=0
--tserver_master_addrs=127.1.135.254:40585
--keytab_file=/tmp/dist-test-taskRiQe_a/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790201539513534-1567-0/krb5kdc/kudu_127.1.135.193.keytab
--principal=kudu/127.1.135.193
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.1.135.212:39009
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--webserver_enabled=0 with env {KRB5CCNAME=/tmp/dist-test-taskRiQe_a/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790201539513534-1567-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskRiQe_a/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790201539513534-1567-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskRiQe_a/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790201539513534-1567-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260923 22:12:39.743672 5716 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260923 22:12:39.743873 5716 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260923 22:12:39.743896 5716 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260923 22:12:39.745065 5716 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260923 22:12:39.745106 5716 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260923 22:12:39.745133 5716 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.1.135.193
I20260923 22:12:39.746605 5716 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.1.135.212:39009
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskRiQe_a/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790201539513534-1567-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskRiQe_a/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790201539513534-1567-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskRiQe_a/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790201539513534-1567-0/krb5kdc/kudu_127.1.135.193.keytab
--principal=kudu/127.1.135.193
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.1.135.193:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--webserver_enabled=false
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskRiQe_a/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790201539513534-1567-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.1.135.193
--webserver_port=0
--tserver_master_addrs=127.1.135.254:40585
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.5716
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.1.135.193
--log_dir=/tmp/dist-test-taskRiQe_a/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790201539513534-1567-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 23 Sep 2026 20:53:37 UTC on ccycloud.quasar-rkviyx.root.comops.site
build id 4668083
I20260923 22:12:39.746852 5716 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260923 22:12:39.747082 5716 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260923 22:12:39.751788 5721 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260923 22:12:39.751910 5716 server_base.cc:1034] running on GCE node
W20260923 22:12:39.751798 5724 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260923 22:12:39.751966 5722 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260923 22:12:39.752241 5716 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260923 22:12:39.752488 5716 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260923 22:12:39.753651 5716 hybrid_clock.cc:648] HybridClock initialized: now 1790201559753638 us; error 43 us; skew 500 ppm
Sep 23 22:12:39 dist-test-slave-s2fj krb5kdc[5610](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790201559, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.135.193@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260923 22:12:39.756203 5716 init.cc:377] Logged in from keytab as kudu/127.1.135.193@KRBTEST.COM (short username kudu)
I20260923 22:12:39.756582 5716 fs_manager.cc:359] Metadata directory not provided
I20260923 22:12:39.756654 5716 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260923 22:12:39.756803 5716 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260923 22:12:39.757827 5716 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskRiQe_a/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790201539513534-1567-0/minicluster-data/ts-0/data/instance:
uuid: "9392cee28d9b4fe984b60718919f874d"
format_stamp: "Formatted at 2026-09-23 22:12:39 on dist-test-slave-s2fj"
server_key: "6e9de4617348fdf12284ffa97f4892b6"
server_key_iv: "8f1e375084879ab4d22c5b8898a24205"
server_key_version: "encryptionkey@0"
I20260923 22:12:39.758126 5716 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskRiQe_a/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790201539513534-1567-0/minicluster-data/ts-0/wal/instance:
uuid: "9392cee28d9b4fe984b60718919f874d"
format_stamp: "Formatted at 2026-09-23 22:12:39 on dist-test-slave-s2fj"
server_key: "6e9de4617348fdf12284ffa97f4892b6"
server_key_iv: "8f1e375084879ab4d22c5b8898a24205"
server_key_version: "encryptionkey@0"
I20260923 22:12:39.759402 5716 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260923 22:12:39.760155 5730 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260923 22:12:39.760336 5716 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260923 22:12:39.760417 5716 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskRiQe_a/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790201539513534-1567-0/minicluster-data/ts-0/data,/tmp/dist-test-taskRiQe_a/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790201539513534-1567-0/minicluster-data/ts-0/wal
uuid: "9392cee28d9b4fe984b60718919f874d"
format_stamp: "Formatted at 2026-09-23 22:12:39 on dist-test-slave-s2fj"
server_key: "6e9de4617348fdf12284ffa97f4892b6"
server_key_iv: "8f1e375084879ab4d22c5b8898a24205"
server_key_version: "encryptionkey@0"
I20260923 22:12:39.760469 5716 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskRiQe_a/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790201539513534-1567-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskRiQe_a/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790201539513534-1567-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskRiQe_a/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790201539513534-1567-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260923 22:12:39.775215 5716 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260923 22:12:39.777060 5716 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260923 22:12:39.777173 5716 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260923 22:12:39.777346 5716 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260923 22:12:39.777667 5716 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260923 22:12:39.777702 5716 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260923 22:12:39.777724 5716 ts_tablet_manager.cc:610] Registered 0 tablets
I20260923 22:12:39.777771 5716 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260923 22:12:39.783782 5716 rpc_server.cc:307] RPC server started. Bound to: 127.1.135.193:38845
I20260923 22:12:39.783855 5843 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.1.135.193:38845 every 8 connection(s)
I20260923 22:12:39.784157 5716 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskRiQe_a/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790201539513534-1567-0/minicluster-data/ts-0/data/info.pb
I20260923 22:12:39.789014 1567 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskRiQe_a/build/release/bin/kudu as pid 5716
I20260923 22:12:39.789115 1567 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskRiQe_a/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790201539513534-1567-0/minicluster-data/ts-0/wal/instance
I20260923 22:12:39.789316 1567 external_mini_cluster.cc:1428] Setting key 44b7ce4b5962d7db08aed5835562b89c
Sep 23 22:12:39 dist-test-slave-s2fj krb5kdc[5610](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790201559, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.135.193@KRBTEST.COM for kudu/127.1.135.254@KRBTEST.COM
I20260923 22:12:39.794546 5694 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 22:12:39.784905 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.135.193:48099 (local address 127.1.135.254:40585)
0923 22:12:39.784954 (+ 49us) server_negotiation.cc:206] Beginning negotiation
0923 22:12:39.784956 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0923 22:12:39.785284 (+ 328us) server_negotiation.cc:407] Connection header received
0923 22:12:39.785729 (+ 445us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 22:12:39.785730 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 22:12:39.785741 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 22:12:39.785804 (+ 63us) server_negotiation.cc:226] Negotiated authn=SASL
0923 22:12:39.786963 (+ 1159us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 22:12:39.787469 (+ 506us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 22:12:39.788883 (+ 1414us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 22:12:39.789214 (+ 331us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 22:12:39.789290 (+ 76us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 22:12:39.791848 (+ 2558us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0923 22:12:39.791849 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0923 22:12:39.791849 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0923 22:12:39.791858 (+ 9us) server_negotiation.cc:922] calling sasl_server_start()
0923 22:12:39.793214 (+ 1356us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0923 22:12:39.793594 (+ 380us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0923 22:12:39.793595 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0923 22:12:39.793595 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0923 22:12:39.793652 (+ 57us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0923 22:12:39.793901 (+ 249us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0923 22:12:39.793902 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0923 22:12:39.793902 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0923 22:12:39.794009 (+ 107us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0923 22:12:39.794080 (+ 71us) server_negotiation.cc:1035] Waiting for connection context
0923 22:12:39.794381 (+ 301us) server_negotiation.cc:299] Negotiation successful
0923 22:12:39.794461 (+ 80us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":23}
I20260923 22:12:39.794731 5846 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 22:12:39.785007 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.135.254:40585 (local address 127.1.135.193:48099)
0923 22:12:39.785175 (+ 168us) negotiation.cc:107] Waiting for socket to connect
0923 22:12:39.785211 (+ 36us) client_negotiation.cc:174] Beginning negotiation
0923 22:12:39.785640 (+ 429us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0923 22:12:39.785918 (+ 278us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0923 22:12:39.785922 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0923 22:12:39.786350 (+ 428us) client_negotiation.cc:189] Negotiated authn=SASL
0923 22:12:39.786848 (+ 498us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 22:12:39.786853 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 22:12:39.787553 (+ 700us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 22:12:39.787554 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 22:12:39.788650 (+ 1096us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 22:12:39.788652 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 22:12:39.789295 (+ 643us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 22:12:39.789296 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 22:12:39.789646 (+ 350us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 22:12:39.790194 (+ 548us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0923 22:12:39.790201 (+ 7us) client_negotiation.cc:656] calling sasl_client_start()
0923 22:12:39.791741 (+ 1540us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0923 22:12:39.793303 (+ 1562us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0923 22:12:39.793309 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0923 22:12:39.793310 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0923 22:12:39.793513 (+ 203us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0923 22:12:39.793739 (+ 226us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0923 22:12:39.793740 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0923 22:12:39.793740 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0923 22:12:39.793839 (+ 99us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0923 22:12:39.794070 (+ 231us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0923 22:12:39.794075 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0923 22:12:39.794235 (+ 160us) client_negotiation.cc:769] Sending connection context
0923 22:12:39.794360 (+ 125us) client_negotiation.cc:240] Negotiation successful
0923 22:12:39.794501 (+ 141us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":125,"thread_start_us":80,"threads_started":1}
I20260923 22:12:39.795221 5844 heartbeater.cc:347] Connected to a master server at 127.1.135.254:40585
I20260923 22:12:39.795349 5844 heartbeater.cc:464] Registering TS with master...
I20260923 22:12:39.795504 5844 heartbeater.cc:511] Master 127.1.135.254:40585 requested a full tablet report, sending...
I20260923 22:12:39.795972 5654 ts_manager.cc:194] Registered new tserver with Master: 9392cee28d9b4fe984b60718919f874d (127.1.135.193:38845)
I20260923 22:12:39.796591 5654 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.1.135.193@KRBTEST.COM'} at 127.1.135.193:48099
WARNING: no policy specified for kudu/127.1.135.194@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.1.135.194@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.1.135.194 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskRiQe_a/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790201539513534-1567-0/krb5kdc/kudu_127.1.135.194.keytab.
Entry for principal kudu/127.1.135.194 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskRiQe_a/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790201539513534-1567-0/krb5kdc/kudu_127.1.135.194.keytab.
I20260923 22:12:39.812672 1567 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskRiQe_a/build/release/bin/kudu
/tmp/dist-test-taskRiQe_a/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskRiQe_a/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790201539513534-1567-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskRiQe_a/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790201539513534-1567-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskRiQe_a/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790201539513534-1567-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskRiQe_a/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790201539513534-1567-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.1.135.194:0
--local_ip_for_outbound_sockets=127.1.135.194
--webserver_interface=127.1.135.194
--webserver_port=0
--tserver_master_addrs=127.1.135.254:40585
--keytab_file=/tmp/dist-test-taskRiQe_a/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790201539513534-1567-0/krb5kdc/kudu_127.1.135.194.keytab
--principal=kudu/127.1.135.194
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.1.135.212:39009
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--webserver_enabled=0 with env {KRB5CCNAME=/tmp/dist-test-taskRiQe_a/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790201539513534-1567-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskRiQe_a/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790201539513534-1567-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskRiQe_a/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790201539513534-1567-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260923 22:12:39.890437 5849 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260923 22:12:39.890617 5849 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260923 22:12:39.890640 5849 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260923 22:12:39.891666 5849 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260923 22:12:39.891705 5849 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260923 22:12:39.891731 5849 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.1.135.194
I20260923 22:12:39.893146 5849 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.1.135.212:39009
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskRiQe_a/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790201539513534-1567-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskRiQe_a/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790201539513534-1567-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskRiQe_a/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790201539513534-1567-0/krb5kdc/kudu_127.1.135.194.keytab
--principal=kudu/127.1.135.194
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.1.135.194:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--webserver_enabled=false
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskRiQe_a/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790201539513534-1567-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.1.135.194
--webserver_port=0
--tserver_master_addrs=127.1.135.254:40585
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.5849
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.1.135.194
--log_dir=/tmp/dist-test-taskRiQe_a/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790201539513534-1567-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 23 Sep 2026 20:53:37 UTC on ccycloud.quasar-rkviyx.root.comops.site
build id 4668083
I20260923 22:12:39.893313 5849 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260923 22:12:39.893467 5849 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260923 22:12:39.898275 5855 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260923 22:12:39.898329 5857 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260923 22:12:39.898320 5849 server_base.cc:1034] running on GCE node
W20260923 22:12:39.898422 5854 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260923 22:12:39.898671 5849 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260923 22:12:39.898860 5849 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260923 22:12:39.899992 5849 hybrid_clock.cc:648] HybridClock initialized: now 1790201559899981 us; error 26 us; skew 500 ppm
Sep 23 22:12:39 dist-test-slave-s2fj krb5kdc[5610](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790201559, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.135.194@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260923 22:12:39.902335 5849 init.cc:377] Logged in from keytab as kudu/127.1.135.194@KRBTEST.COM (short username kudu)
I20260923 22:12:39.902675 5849 fs_manager.cc:359] Metadata directory not provided
I20260923 22:12:39.902742 5849 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260923 22:12:39.902884 5849 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260923 22:12:39.903844 5849 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskRiQe_a/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790201539513534-1567-0/minicluster-data/ts-1/data/instance:
uuid: "61281de877044380a0093894dbb77b5e"
format_stamp: "Formatted at 2026-09-23 22:12:39 on dist-test-slave-s2fj"
server_key: "d4bcf32976b188f7f48dff90b8b96a64"
server_key_iv: "8162c779e4c3b7bec0871e55fd0ac200"
server_key_version: "encryptionkey@0"
I20260923 22:12:39.904222 5849 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskRiQe_a/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790201539513534-1567-0/minicluster-data/ts-1/wal/instance:
uuid: "61281de877044380a0093894dbb77b5e"
format_stamp: "Formatted at 2026-09-23 22:12:39 on dist-test-slave-s2fj"
server_key: "d4bcf32976b188f7f48dff90b8b96a64"
server_key_iv: "8162c779e4c3b7bec0871e55fd0ac200"
server_key_version: "encryptionkey@0"
I20260923 22:12:39.905449 5849 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.001s sys 0.000s
I20260923 22:12:39.906662 5863 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260923 22:12:39.906819 5849 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260923 22:12:39.906896 5849 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskRiQe_a/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790201539513534-1567-0/minicluster-data/ts-1/data,/tmp/dist-test-taskRiQe_a/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790201539513534-1567-0/minicluster-data/ts-1/wal
uuid: "61281de877044380a0093894dbb77b5e"
format_stamp: "Formatted at 2026-09-23 22:12:39 on dist-test-slave-s2fj"
server_key: "d4bcf32976b188f7f48dff90b8b96a64"
server_key_iv: "8162c779e4c3b7bec0871e55fd0ac200"
server_key_version: "encryptionkey@0"
I20260923 22:12:39.906946 5849 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskRiQe_a/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790201539513534-1567-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskRiQe_a/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790201539513534-1567-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskRiQe_a/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790201539513534-1567-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260923 22:12:39.926352 5849 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260923 22:12:39.928118 5849 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260923 22:12:39.928231 5849 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260923 22:12:39.928398 5849 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260923 22:12:39.928699 5849 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260923 22:12:39.928732 5849 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260923 22:12:39.928753 5849 ts_tablet_manager.cc:610] Registered 0 tablets
I20260923 22:12:39.928807 5849 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260923 22:12:39.934147 5849 rpc_server.cc:307] RPC server started. Bound to: 127.1.135.194:39347
I20260923 22:12:39.934186 5976 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.1.135.194:39347 every 8 connection(s)
I20260923 22:12:39.934448 5849 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskRiQe_a/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790201539513534-1567-0/minicluster-data/ts-1/data/info.pb
I20260923 22:12:39.936813 1567 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskRiQe_a/build/release/bin/kudu as pid 5849
I20260923 22:12:39.936900 1567 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskRiQe_a/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790201539513534-1567-0/minicluster-data/ts-1/wal/instance
I20260923 22:12:39.937085 1567 external_mini_cluster.cc:1428] Setting key fe96d9035c9ba2dddea7d5ba9293404e
Sep 23 22:12:39 dist-test-slave-s2fj krb5kdc[5610](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790201559, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.135.194@KRBTEST.COM for kudu/127.1.135.254@KRBTEST.COM
I20260923 22:12:39.943646 5694 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 22:12:39.935151 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.135.194:60159 (local address 127.1.135.254:40585)
0923 22:12:39.935231 (+ 80us) server_negotiation.cc:206] Beginning negotiation
0923 22:12:39.935233 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0923 22:12:39.935338 (+ 105us) server_negotiation.cc:407] Connection header received
0923 22:12:39.935783 (+ 445us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 22:12:39.935784 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 22:12:39.935793 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 22:12:39.935843 (+ 50us) server_negotiation.cc:226] Negotiated authn=SASL
0923 22:12:39.936709 (+ 866us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 22:12:39.937377 (+ 668us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 22:12:39.938727 (+ 1350us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 22:12:39.938908 (+ 181us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 22:12:39.938959 (+ 51us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 22:12:39.941074 (+ 2115us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0923 22:12:39.941075 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0923 22:12:39.941076 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0923 22:12:39.941083 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0923 22:12:39.942379 (+ 1296us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0923 22:12:39.942751 (+ 372us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0923 22:12:39.942752 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0923 22:12:39.942753 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0923 22:12:39.942805 (+ 52us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0923 22:12:39.943041 (+ 236us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0923 22:12:39.943042 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0923 22:12:39.943042 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0923 22:12:39.943154 (+ 112us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0923 22:12:39.943229 (+ 75us) server_negotiation.cc:1035] Waiting for connection context
0923 22:12:39.943480 (+ 251us) server_negotiation.cc:299] Negotiation successful
0923 22:12:39.943561 (+ 81us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":47}
I20260923 22:12:39.943856 5979 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 22:12:39.935097 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.135.254:40585 (local address 127.1.135.194:60159)
0923 22:12:39.935242 (+ 145us) negotiation.cc:107] Waiting for socket to connect
0923 22:12:39.935275 (+ 33us) client_negotiation.cc:174] Beginning negotiation
0923 22:12:39.935707 (+ 432us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0923 22:12:39.935892 (+ 185us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0923 22:12:39.935895 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0923 22:12:39.936155 (+ 260us) client_negotiation.cc:189] Negotiated authn=SASL
0923 22:12:39.936604 (+ 449us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 22:12:39.936630 (+ 26us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 22:12:39.937469 (+ 839us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 22:12:39.937470 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 22:12:39.938641 (+ 1171us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 22:12:39.938644 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 22:12:39.938983 (+ 339us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 22:12:39.938984 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 22:12:39.939134 (+ 150us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 22:12:39.939662 (+ 528us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0923 22:12:39.939669 (+ 7us) client_negotiation.cc:656] calling sasl_client_start()
0923 22:12:39.940989 (+ 1320us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0923 22:12:39.942458 (+ 1469us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0923 22:12:39.942464 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0923 22:12:39.942466 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0923 22:12:39.942667 (+ 201us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0923 22:12:39.942879 (+ 212us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0923 22:12:39.942879 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0923 22:12:39.942880 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0923 22:12:39.942980 (+ 100us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0923 22:12:39.943220 (+ 240us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0923 22:12:39.943226 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0923 22:12:39.943354 (+ 128us) client_negotiation.cc:769] Sending connection context
0923 22:12:39.943447 (+ 93us) client_negotiation.cc:240] Negotiation successful
0923 22:12:39.943588 (+ 141us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":111,"thread_start_us":89,"threads_started":1}
I20260923 22:12:39.944324 5977 heartbeater.cc:347] Connected to a master server at 127.1.135.254:40585
I20260923 22:12:39.944442 5977 heartbeater.cc:464] Registering TS with master...
I20260923 22:12:39.944590 5977 heartbeater.cc:511] Master 127.1.135.254:40585 requested a full tablet report, sending...
WARNING: no policy specified for kudu/127.1.135.195@KRBTEST.COM; defaulting to no policy
I20260923 22:12:39.944916 5654 ts_manager.cc:194] Registered new tserver with Master: 61281de877044380a0093894dbb77b5e (127.1.135.194:39347)
I20260923 22:12:39.945281 5654 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.1.135.194@KRBTEST.COM'} at 127.1.135.194:60159
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.1.135.195@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.1.135.195 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskRiQe_a/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790201539513534-1567-0/krb5kdc/kudu_127.1.135.195.keytab.
Entry for principal kudu/127.1.135.195 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskRiQe_a/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790201539513534-1567-0/krb5kdc/kudu_127.1.135.195.keytab.
I20260923 22:12:39.959883 1567 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskRiQe_a/build/release/bin/kudu
/tmp/dist-test-taskRiQe_a/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskRiQe_a/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790201539513534-1567-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskRiQe_a/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790201539513534-1567-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskRiQe_a/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790201539513534-1567-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskRiQe_a/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790201539513534-1567-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.1.135.195:0
--local_ip_for_outbound_sockets=127.1.135.195
--webserver_interface=127.1.135.195
--webserver_port=0
--tserver_master_addrs=127.1.135.254:40585
--keytab_file=/tmp/dist-test-taskRiQe_a/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790201539513534-1567-0/krb5kdc/kudu_127.1.135.195.keytab
--principal=kudu/127.1.135.195
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.1.135.212:39009
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--webserver_enabled=0 with env {KRB5CCNAME=/tmp/dist-test-taskRiQe_a/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790201539513534-1567-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskRiQe_a/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790201539513534-1567-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskRiQe_a/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790201539513534-1567-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260923 22:12:40.038275 5982 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260923 22:12:40.038450 5982 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260923 22:12:40.038472 5982 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260923 22:12:40.039480 5982 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260923 22:12:40.039511 5982 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260923 22:12:40.039536 5982 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.1.135.195
I20260923 22:12:40.040927 5982 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.1.135.212:39009
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskRiQe_a/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790201539513534-1567-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskRiQe_a/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790201539513534-1567-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskRiQe_a/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790201539513534-1567-0/krb5kdc/kudu_127.1.135.195.keytab
--principal=kudu/127.1.135.195
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.1.135.195:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--webserver_enabled=false
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskRiQe_a/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790201539513534-1567-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.1.135.195
--webserver_port=0
--tserver_master_addrs=127.1.135.254:40585
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.5982
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.1.135.195
--log_dir=/tmp/dist-test-taskRiQe_a/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790201539513534-1567-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 23 Sep 2026 20:53:37 UTC on ccycloud.quasar-rkviyx.root.comops.site
build id 4668083
I20260923 22:12:40.041097 5982 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260923 22:12:40.041260 5982 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260923 22:12:40.045917 5990 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260923 22:12:40.045979 5988 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260923 22:12:40.045943 5987 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260923 22:12:40.046022 5982 server_base.cc:1034] running on GCE node
I20260923 22:12:40.046231 5982 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260923 22:12:40.046471 5982 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260923 22:12:40.047626 5982 hybrid_clock.cc:648] HybridClock initialized: now 1790201560047617 us; error 30 us; skew 500 ppm
Sep 23 22:12:40 dist-test-slave-s2fj krb5kdc[5610](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790201560, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.135.195@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260923 22:12:40.050127 5982 init.cc:377] Logged in from keytab as kudu/127.1.135.195@KRBTEST.COM (short username kudu)
I20260923 22:12:40.050483 5982 fs_manager.cc:359] Metadata directory not provided
I20260923 22:12:40.050546 5982 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260923 22:12:40.050673 5982 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260923 22:12:40.051643 5982 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskRiQe_a/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790201539513534-1567-0/minicluster-data/ts-2/data/instance:
uuid: "8aa43ecf7d6e42e7b34e2d32e5b4d11e"
format_stamp: "Formatted at 2026-09-23 22:12:40 on dist-test-slave-s2fj"
server_key: "9a96670c2f8439332d5f46764144e7ed"
server_key_iv: "43f605443718fab5074fa7cd9cf52b44"
server_key_version: "encryptionkey@0"
I20260923 22:12:40.052016 5982 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskRiQe_a/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790201539513534-1567-0/minicluster-data/ts-2/wal/instance:
uuid: "8aa43ecf7d6e42e7b34e2d32e5b4d11e"
format_stamp: "Formatted at 2026-09-23 22:12:40 on dist-test-slave-s2fj"
server_key: "9a96670c2f8439332d5f46764144e7ed"
server_key_iv: "43f605443718fab5074fa7cd9cf52b44"
server_key_version: "encryptionkey@0"
I20260923 22:12:40.053246 5982 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.001s sys 0.000s
I20260923 22:12:40.053911 5996 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260923 22:12:40.054095 5982 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260923 22:12:40.054178 5982 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskRiQe_a/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790201539513534-1567-0/minicluster-data/ts-2/data,/tmp/dist-test-taskRiQe_a/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790201539513534-1567-0/minicluster-data/ts-2/wal
uuid: "8aa43ecf7d6e42e7b34e2d32e5b4d11e"
format_stamp: "Formatted at 2026-09-23 22:12:40 on dist-test-slave-s2fj"
server_key: "9a96670c2f8439332d5f46764144e7ed"
server_key_iv: "43f605443718fab5074fa7cd9cf52b44"
server_key_version: "encryptionkey@0"
I20260923 22:12:40.054234 5982 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskRiQe_a/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790201539513534-1567-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskRiQe_a/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790201539513534-1567-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskRiQe_a/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790201539513534-1567-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260923 22:12:40.062173 5982 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260923 22:12:40.064409 5982 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260923 22:12:40.064510 5982 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260923 22:12:40.064705 5982 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260923 22:12:40.064940 5982 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260923 22:12:40.064970 5982 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260923 22:12:40.064990 5982 ts_tablet_manager.cc:610] Registered 0 tablets
I20260923 22:12:40.065002 5982 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260923 22:12:40.070289 5982 rpc_server.cc:307] RPC server started. Bound to: 127.1.135.195:41699
I20260923 22:12:40.070343 6109 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.1.135.195:41699 every 8 connection(s)
I20260923 22:12:40.070587 5982 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskRiQe_a/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790201539513534-1567-0/minicluster-data/ts-2/data/info.pb
I20260923 22:12:40.074431 1567 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskRiQe_a/build/release/bin/kudu as pid 5982
I20260923 22:12:40.074553 1567 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskRiQe_a/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790201539513534-1567-0/minicluster-data/ts-2/wal/instance
I20260923 22:12:40.074776 1567 external_mini_cluster.cc:1428] Setting key b0bc4d2605ae131907756c5c6b6ecdc7
Sep 23 22:12:40 dist-test-slave-s2fj krb5kdc[5610](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790201560, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.135.195@KRBTEST.COM for kudu/127.1.135.254@KRBTEST.COM
I20260923 22:12:40.079263 5694 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 22:12:40.071224 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.135.195:43717 (local address 127.1.135.254:40585)
0923 22:12:40.071270 (+ 46us) server_negotiation.cc:206] Beginning negotiation
0923 22:12:40.071273 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0923 22:12:40.071459 (+ 186us) server_negotiation.cc:407] Connection header received
0923 22:12:40.071897 (+ 438us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 22:12:40.071898 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 22:12:40.071907 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 22:12:40.071958 (+ 51us) server_negotiation.cc:226] Negotiated authn=SASL
0923 22:12:40.072812 (+ 854us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 22:12:40.073291 (+ 479us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 22:12:40.074305 (+ 1014us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 22:12:40.074629 (+ 324us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 22:12:40.074739 (+ 110us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 22:12:40.076880 (+ 2141us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0923 22:12:40.076882 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0923 22:12:40.076882 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0923 22:12:40.076888 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0923 22:12:40.078269 (+ 1381us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0923 22:12:40.078578 (+ 309us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0923 22:12:40.078579 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0923 22:12:40.078579 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0923 22:12:40.078625 (+ 46us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0923 22:12:40.078825 (+ 200us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0923 22:12:40.078826 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0923 22:12:40.078826 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0923 22:12:40.078915 (+ 89us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0923 22:12:40.078964 (+ 49us) server_negotiation.cc:1035] Waiting for connection context
0923 22:12:40.079147 (+ 183us) server_negotiation.cc:299] Negotiation successful
0923 22:12:40.079204 (+ 57us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":18}
I20260923 22:12:40.079468 6112 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 22:12:40.071215 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.135.254:40585 (local address 127.1.135.195:43717)
0923 22:12:40.071354 (+ 139us) negotiation.cc:107] Waiting for socket to connect
0923 22:12:40.071386 (+ 32us) client_negotiation.cc:174] Beginning negotiation
0923 22:12:40.071825 (+ 439us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0923 22:12:40.072007 (+ 182us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0923 22:12:40.072010 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0923 22:12:40.072308 (+ 298us) client_negotiation.cc:189] Negotiated authn=SASL
0923 22:12:40.072742 (+ 434us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 22:12:40.072748 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 22:12:40.073350 (+ 602us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 22:12:40.073351 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 22:12:40.074196 (+ 845us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 22:12:40.074198 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 22:12:40.074751 (+ 553us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 22:12:40.074752 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 22:12:40.074873 (+ 121us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 22:12:40.075364 (+ 491us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0923 22:12:40.075373 (+ 9us) client_negotiation.cc:656] calling sasl_client_start()
0923 22:12:40.076804 (+ 1431us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0923 22:12:40.078333 (+ 1529us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0923 22:12:40.078337 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0923 22:12:40.078339 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0923 22:12:40.078522 (+ 183us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0923 22:12:40.078676 (+ 154us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0923 22:12:40.078677 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0923 22:12:40.078677 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0923 22:12:40.078778 (+ 101us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0923 22:12:40.078963 (+ 185us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0923 22:12:40.078966 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0923 22:12:40.079062 (+ 96us) client_negotiation.cc:769] Sending connection context
0923 22:12:40.079135 (+ 73us) client_negotiation.cc:240] Negotiation successful
0923 22:12:40.079241 (+ 106us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":104,"thread_start_us":84,"threads_started":1}
I20260923 22:12:40.079955 6110 heartbeater.cc:347] Connected to a master server at 127.1.135.254:40585
I20260923 22:12:40.080047 6110 heartbeater.cc:464] Registering TS with master...
I20260923 22:12:40.080211 6110 heartbeater.cc:511] Master 127.1.135.254:40585 requested a full tablet report, sending...
I20260923 22:12:40.080534 5654 ts_manager.cc:194] Registered new tserver with Master: 8aa43ecf7d6e42e7b34e2d32e5b4d11e (127.1.135.195:41699)
I20260923 22:12:40.080847 5654 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.1.135.195@KRBTEST.COM'} at 127.1.135.195:43717
I20260923 22:12:40.086211 1567 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260923 22:12:40.093176 5694 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 22:12:40.087449 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40068 (local address 127.1.135.254:40585)
0923 22:12:40.087488 (+ 39us) server_negotiation.cc:206] Beginning negotiation
0923 22:12:40.087500 (+ 12us) server_negotiation.cc:399] Waiting for connection header
0923 22:12:40.087616 (+ 116us) server_negotiation.cc:407] Connection header received
0923 22:12:40.087689 (+ 73us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 22:12:40.087690 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 22:12:40.087699 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 22:12:40.087768 (+ 69us) server_negotiation.cc:226] Negotiated authn=SASL
0923 22:12:40.088387 (+ 619us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 22:12:40.088848 (+ 461us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 22:12:40.089543 (+ 695us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 22:12:40.089749 (+ 206us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 22:12:40.089829 (+ 80us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 22:12:40.090986 (+ 1157us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0923 22:12:40.090988 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0923 22:12:40.090988 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0923 22:12:40.090995 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0923 22:12:40.092105 (+ 1110us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0923 22:12:40.092432 (+ 327us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0923 22:12:40.092433 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0923 22:12:40.092434 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0923 22:12:40.092483 (+ 49us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0923 22:12:40.092665 (+ 182us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0923 22:12:40.092667 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0923 22:12:40.092667 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0923 22:12:40.092812 (+ 145us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0923 22:12:40.092908 (+ 96us) server_negotiation.cc:1035] Waiting for connection context
0923 22:12:40.093013 (+ 105us) server_negotiation.cc:299] Negotiation successful
0923 22:12:40.093088 (+ 75us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":16}
I20260923 22:12:40.094583 5654 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:40068:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260923 22:12:40.096233 5654 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260923 22:12:40.101894 6119 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 22:12:40.098193 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.135.193:38845 (local address 127.0.0.1:47858)
0923 22:12:40.098492 (+ 299us) negotiation.cc:107] Waiting for socket to connect
0923 22:12:40.098501 (+ 9us) client_negotiation.cc:174] Beginning negotiation
0923 22:12:40.098555 (+ 54us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0923 22:12:40.098809 (+ 254us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0923 22:12:40.098809 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0923 22:12:40.098811 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0923 22:12:40.098855 (+ 44us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 22:12:40.098857 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 22:12:40.099672 (+ 815us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 22:12:40.099673 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 22:12:40.100647 (+ 974us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 22:12:40.100649 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 22:12:40.101606 (+ 957us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 22:12:40.101607 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 22:12:40.101681 (+ 74us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 22:12:40.101695 (+ 14us) client_negotiation.cc:769] Sending connection context
0923 22:12:40.101769 (+ 74us) client_negotiation.cc:240] Negotiation successful
0923 22:12:40.101816 (+ 47us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":289,"thread_start_us":45,"threads_started":1}
I20260923 22:12:40.101919 6122 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 22:12:40.098352 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47858 (local address 127.1.135.193:38845)
0923 22:12:40.098678 (+ 326us) server_negotiation.cc:206] Beginning negotiation
0923 22:12:40.098683 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0923 22:12:40.098693 (+ 10us) server_negotiation.cc:407] Connection header received
0923 22:12:40.098730 (+ 37us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 22:12:40.098732 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 22:12:40.098747 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 22:12:40.098813 (+ 66us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0923 22:12:40.098911 (+ 98us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 22:12:40.099558 (+ 647us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 22:12:40.100803 (+ 1245us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 22:12:40.101505 (+ 702us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 22:12:40.101599 (+ 94us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 22:12:40.101652 (+ 53us) server_negotiation.cc:1035] Waiting for connection context
0923 22:12:40.101789 (+ 137us) server_negotiation.cc:299] Negotiation successful
0923 22:12:40.101827 (+ 38us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":295,"thread_start_us":72,"threads_started":1}
I20260923 22:12:40.102509 6121 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 22:12:40.098185 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.135.195:41699 (local address 127.0.0.1:46544)
0923 22:12:40.098453 (+ 268us) negotiation.cc:107] Waiting for socket to connect
0923 22:12:40.098492 (+ 39us) client_negotiation.cc:174] Beginning negotiation
0923 22:12:40.098556 (+ 64us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0923 22:12:40.098779 (+ 223us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0923 22:12:40.098782 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0923 22:12:40.098784 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0923 22:12:40.098850 (+ 66us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 22:12:40.098852 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 22:12:40.100369 (+ 1517us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 22:12:40.100370 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 22:12:40.101273 (+ 903us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 22:12:40.101275 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 22:12:40.102109 (+ 834us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 22:12:40.102111 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 22:12:40.102342 (+ 231us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 22:12:40.102348 (+ 6us) client_negotiation.cc:769] Sending connection context
0923 22:12:40.102401 (+ 53us) client_negotiation.cc:240] Negotiation successful
0923 22:12:40.102443 (+ 42us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":246,"mutex_wait_us":20,"thread_start_us":48,"threads_started":1}
I20260923 22:12:40.102532 6120 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 22:12:40.098272 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46544 (local address 127.1.135.195:41699)
0923 22:12:40.098491 (+ 219us) server_negotiation.cc:206] Beginning negotiation
0923 22:12:40.098496 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0923 22:12:40.098544 (+ 48us) server_negotiation.cc:407] Connection header received
0923 22:12:40.098634 (+ 90us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 22:12:40.098636 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 22:12:40.098653 (+ 17us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 22:12:40.099147 (+ 494us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0923 22:12:40.099215 (+ 68us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 22:12:40.100279 (+ 1064us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 22:12:40.101351 (+ 1072us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 22:12:40.102037 (+ 686us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 22:12:40.102110 (+ 73us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 22:12:40.102163 (+ 53us) server_negotiation.cc:1035] Waiting for connection context
0923 22:12:40.102422 (+ 259us) server_negotiation.cc:299] Negotiation successful
0923 22:12:40.102462 (+ 40us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":190,"thread_start_us":92,"threads_started":1}
I20260923 22:12:40.102789 6118 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 22:12:40.098361 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.135.194:39347 (local address 127.0.0.1:54404)
0923 22:12:40.098644 (+ 283us) negotiation.cc:107] Waiting for socket to connect
0923 22:12:40.098651 (+ 7us) client_negotiation.cc:174] Beginning negotiation
0923 22:12:40.098710 (+ 59us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0923 22:12:40.098925 (+ 215us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0923 22:12:40.098926 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0923 22:12:40.098928 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0923 22:12:40.098961 (+ 33us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 22:12:40.098963 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 22:12:40.100330 (+ 1367us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 22:12:40.100331 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 22:12:40.101276 (+ 945us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 22:12:40.101278 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 22:12:40.102450 (+ 1172us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 22:12:40.102451 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 22:12:40.102504 (+ 53us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 22:12:40.102507 (+ 3us) client_negotiation.cc:769] Sending connection context
0923 22:12:40.102552 (+ 45us) client_negotiation.cc:240] Negotiation successful
0923 22:12:40.102686 (+ 134us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":273,"thread_start_us":36,"threads_started":1}
I20260923 22:12:40.102730 5778 tablet_service.cc:1467] Processing CreateTablet for tablet 6ce752740f874c27a225a283eca01839 (DEFAULT_TABLE table=test-table [id=43e0417169ea41e287e962a812a9f8f3]), partition=RANGE (key) PARTITION UNBOUNDED
I20260923 22:12:40.103030 6123 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 22:12:40.098468 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54404 (local address 127.1.135.194:39347)
0923 22:12:40.098635 (+ 167us) server_negotiation.cc:206] Beginning negotiation
0923 22:12:40.098638 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0923 22:12:40.098700 (+ 62us) server_negotiation.cc:407] Connection header received
0923 22:12:40.098777 (+ 77us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 22:12:40.098779 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 22:12:40.098793 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 22:12:40.099030 (+ 237us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0923 22:12:40.099114 (+ 84us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 22:12:40.100222 (+ 1108us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 22:12:40.101366 (+ 1144us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 22:12:40.102305 (+ 939us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 22:12:40.102370 (+ 65us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 22:12:40.102421 (+ 51us) server_negotiation.cc:1035] Waiting for connection context
0923 22:12:40.102904 (+ 483us) server_negotiation.cc:299] Negotiation successful
0923 22:12:40.102943 (+ 39us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":142,"thread_start_us":81,"threads_started":1}
I20260923 22:12:40.103137 5778 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 6ce752740f874c27a225a283eca01839. 1 dirs total, 0 dirs full, 0 dirs failed
I20260923 22:12:40.103534 5911 tablet_service.cc:1467] Processing CreateTablet for tablet 6ce752740f874c27a225a283eca01839 (DEFAULT_TABLE table=test-table [id=43e0417169ea41e287e962a812a9f8f3]), partition=RANGE (key) PARTITION UNBOUNDED
I20260923 22:12:40.103811 5911 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 6ce752740f874c27a225a283eca01839. 1 dirs total, 0 dirs full, 0 dirs failed
I20260923 22:12:40.104897 6044 tablet_service.cc:1467] Processing CreateTablet for tablet 6ce752740f874c27a225a283eca01839 (DEFAULT_TABLE table=test-table [id=43e0417169ea41e287e962a812a9f8f3]), partition=RANGE (key) PARTITION UNBOUNDED
I20260923 22:12:40.105206 6044 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 6ce752740f874c27a225a283eca01839. 1 dirs total, 0 dirs full, 0 dirs failed
I20260923 22:12:40.106355 6124 tablet_bootstrap.cc:492] T 6ce752740f874c27a225a283eca01839 P 9392cee28d9b4fe984b60718919f874d: Bootstrap starting.
I20260923 22:12:40.106655 6125 tablet_bootstrap.cc:492] T 6ce752740f874c27a225a283eca01839 P 61281de877044380a0093894dbb77b5e: Bootstrap starting.
I20260923 22:12:40.107056 6124 tablet_bootstrap.cc:654] T 6ce752740f874c27a225a283eca01839 P 9392cee28d9b4fe984b60718919f874d: Neither blocks nor log segments found. Creating new log.
I20260923 22:12:40.107256 6125 tablet_bootstrap.cc:654] T 6ce752740f874c27a225a283eca01839 P 61281de877044380a0093894dbb77b5e: Neither blocks nor log segments found. Creating new log.
I20260923 22:12:40.107689 6124 log.cc:824] T 6ce752740f874c27a225a283eca01839 P 9392cee28d9b4fe984b60718919f874d: Log is configured to *not* fsync() on all Append() calls
I20260923 22:12:40.107858 6125 log.cc:824] T 6ce752740f874c27a225a283eca01839 P 61281de877044380a0093894dbb77b5e: Log is configured to *not* fsync() on all Append() calls
I20260923 22:12:40.108172 6127 tablet_bootstrap.cc:492] T 6ce752740f874c27a225a283eca01839 P 8aa43ecf7d6e42e7b34e2d32e5b4d11e: Bootstrap starting.
I20260923 22:12:40.108541 6124 tablet_bootstrap.cc:492] T 6ce752740f874c27a225a283eca01839 P 9392cee28d9b4fe984b60718919f874d: No bootstrap required, opened a new log
I20260923 22:12:40.108626 6124 ts_tablet_manager.cc:1397] T 6ce752740f874c27a225a283eca01839 P 9392cee28d9b4fe984b60718919f874d: Time spent bootstrapping tablet: real 0.002s user 0.002s sys 0.000s
I20260923 22:12:40.108771 6127 tablet_bootstrap.cc:654] T 6ce752740f874c27a225a283eca01839 P 8aa43ecf7d6e42e7b34e2d32e5b4d11e: Neither blocks nor log segments found. Creating new log.
I20260923 22:12:40.109444 6127 log.cc:824] T 6ce752740f874c27a225a283eca01839 P 8aa43ecf7d6e42e7b34e2d32e5b4d11e: Log is configured to *not* fsync() on all Append() calls
I20260923 22:12:40.110141 6127 tablet_bootstrap.cc:492] T 6ce752740f874c27a225a283eca01839 P 8aa43ecf7d6e42e7b34e2d32e5b4d11e: No bootstrap required, opened a new log
I20260923 22:12:40.110210 6127 ts_tablet_manager.cc:1397] T 6ce752740f874c27a225a283eca01839 P 8aa43ecf7d6e42e7b34e2d32e5b4d11e: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20260923 22:12:40.110756 6124 raft_consensus.cc:348] T 6ce752740f874c27a225a283eca01839 P 9392cee28d9b4fe984b60718919f874d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9392cee28d9b4fe984b60718919f874d" member_type: VOTER last_known_addr { host: "127.1.135.193" port: 38845 } } peers { permanent_uuid: "61281de877044380a0093894dbb77b5e" member_type: VOTER last_known_addr { host: "127.1.135.194" port: 39347 } } peers { permanent_uuid: "8aa43ecf7d6e42e7b34e2d32e5b4d11e" member_type: VOTER last_known_addr { host: "127.1.135.195" port: 41699 } }
I20260923 22:12:40.110930 6124 raft_consensus.cc:374] T 6ce752740f874c27a225a283eca01839 P 9392cee28d9b4fe984b60718919f874d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260923 22:12:40.110975 6124 raft_consensus.cc:729] T 6ce752740f874c27a225a283eca01839 P 9392cee28d9b4fe984b60718919f874d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 9392cee28d9b4fe984b60718919f874d, State: Initialized, Role: FOLLOWER
I20260923 22:12:40.111088 6124 consensus_queue.cc:260] T 6ce752740f874c27a225a283eca01839 P 9392cee28d9b4fe984b60718919f874d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9392cee28d9b4fe984b60718919f874d" member_type: VOTER last_known_addr { host: "127.1.135.193" port: 38845 } } peers { permanent_uuid: "61281de877044380a0093894dbb77b5e" member_type: VOTER last_known_addr { host: "127.1.135.194" port: 39347 } } peers { permanent_uuid: "8aa43ecf7d6e42e7b34e2d32e5b4d11e" member_type: VOTER last_known_addr { host: "127.1.135.195" port: 41699 } }
I20260923 22:12:40.111359 6124 ts_tablet_manager.cc:1428] T 6ce752740f874c27a225a283eca01839 P 9392cee28d9b4fe984b60718919f874d: Time spent starting tablet: real 0.003s user 0.003s sys 0.000s
I20260923 22:12:40.111382 5844 heartbeater.cc:503] Master 127.1.135.254:40585 was elected leader, sending a full tablet report...
I20260923 22:12:40.111742 6125 tablet_bootstrap.cc:492] T 6ce752740f874c27a225a283eca01839 P 61281de877044380a0093894dbb77b5e: No bootstrap required, opened a new log
I20260923 22:12:40.111806 6125 ts_tablet_manager.cc:1397] T 6ce752740f874c27a225a283eca01839 P 61281de877044380a0093894dbb77b5e: Time spent bootstrapping tablet: real 0.005s user 0.001s sys 0.000s
I20260923 22:12:40.112236 6127 raft_consensus.cc:348] T 6ce752740f874c27a225a283eca01839 P 8aa43ecf7d6e42e7b34e2d32e5b4d11e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9392cee28d9b4fe984b60718919f874d" member_type: VOTER last_known_addr { host: "127.1.135.193" port: 38845 } } peers { permanent_uuid: "61281de877044380a0093894dbb77b5e" member_type: VOTER last_known_addr { host: "127.1.135.194" port: 39347 } } peers { permanent_uuid: "8aa43ecf7d6e42e7b34e2d32e5b4d11e" member_type: VOTER last_known_addr { host: "127.1.135.195" port: 41699 } }
I20260923 22:12:40.112406 6127 raft_consensus.cc:374] T 6ce752740f874c27a225a283eca01839 P 8aa43ecf7d6e42e7b34e2d32e5b4d11e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260923 22:12:40.112447 6127 raft_consensus.cc:729] T 6ce752740f874c27a225a283eca01839 P 8aa43ecf7d6e42e7b34e2d32e5b4d11e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 8aa43ecf7d6e42e7b34e2d32e5b4d11e, State: Initialized, Role: FOLLOWER
I20260923 22:12:40.112581 6127 consensus_queue.cc:260] T 6ce752740f874c27a225a283eca01839 P 8aa43ecf7d6e42e7b34e2d32e5b4d11e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9392cee28d9b4fe984b60718919f874d" member_type: VOTER last_known_addr { host: "127.1.135.193" port: 38845 } } peers { permanent_uuid: "61281de877044380a0093894dbb77b5e" member_type: VOTER last_known_addr { host: "127.1.135.194" port: 39347 } } peers { permanent_uuid: "8aa43ecf7d6e42e7b34e2d32e5b4d11e" member_type: VOTER last_known_addr { host: "127.1.135.195" port: 41699 } }
I20260923 22:12:40.112840 6127 ts_tablet_manager.cc:1428] T 6ce752740f874c27a225a283eca01839 P 8aa43ecf7d6e42e7b34e2d32e5b4d11e: Time spent starting tablet: real 0.003s user 0.000s sys 0.003s
I20260923 22:12:40.112963 6110 heartbeater.cc:503] Master 127.1.135.254:40585 was elected leader, sending a full tablet report...
I20260923 22:12:40.113808 6125 raft_consensus.cc:348] T 6ce752740f874c27a225a283eca01839 P 61281de877044380a0093894dbb77b5e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9392cee28d9b4fe984b60718919f874d" member_type: VOTER last_known_addr { host: "127.1.135.193" port: 38845 } } peers { permanent_uuid: "61281de877044380a0093894dbb77b5e" member_type: VOTER last_known_addr { host: "127.1.135.194" port: 39347 } } peers { permanent_uuid: "8aa43ecf7d6e42e7b34e2d32e5b4d11e" member_type: VOTER last_known_addr { host: "127.1.135.195" port: 41699 } }
I20260923 22:12:40.113950 6125 raft_consensus.cc:374] T 6ce752740f874c27a225a283eca01839 P 61281de877044380a0093894dbb77b5e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260923 22:12:40.113971 6125 raft_consensus.cc:729] T 6ce752740f874c27a225a283eca01839 P 61281de877044380a0093894dbb77b5e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 61281de877044380a0093894dbb77b5e, State: Initialized, Role: FOLLOWER
I20260923 22:12:40.114042 6125 consensus_queue.cc:260] T 6ce752740f874c27a225a283eca01839 P 61281de877044380a0093894dbb77b5e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9392cee28d9b4fe984b60718919f874d" member_type: VOTER last_known_addr { host: "127.1.135.193" port: 38845 } } peers { permanent_uuid: "61281de877044380a0093894dbb77b5e" member_type: VOTER last_known_addr { host: "127.1.135.194" port: 39347 } } peers { permanent_uuid: "8aa43ecf7d6e42e7b34e2d32e5b4d11e" member_type: VOTER last_known_addr { host: "127.1.135.195" port: 41699 } }
I20260923 22:12:40.114213 6125 ts_tablet_manager.cc:1428] T 6ce752740f874c27a225a283eca01839 P 61281de877044380a0093894dbb77b5e: Time spent starting tablet: real 0.002s user 0.003s sys 0.000s
I20260923 22:12:40.114267 5977 heartbeater.cc:503] Master 127.1.135.254:40585 was elected leader, sending a full tablet report...
W20260923 22:12:40.184963 5978 tablet.cc:2369] T 6ce752740f874c27a225a283eca01839 P 61281de877044380a0093894dbb77b5e: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260923 22:12:40.260869 6130 raft_consensus.cc:482] T 6ce752740f874c27a225a283eca01839 P 9392cee28d9b4fe984b60718919f874d [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260923 22:12:40.260987 6130 raft_consensus.cc:504] T 6ce752740f874c27a225a283eca01839 P 9392cee28d9b4fe984b60718919f874d [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9392cee28d9b4fe984b60718919f874d" member_type: VOTER last_known_addr { host: "127.1.135.193" port: 38845 } } peers { permanent_uuid: "61281de877044380a0093894dbb77b5e" member_type: VOTER last_known_addr { host: "127.1.135.194" port: 39347 } } peers { permanent_uuid: "8aa43ecf7d6e42e7b34e2d32e5b4d11e" member_type: VOTER last_known_addr { host: "127.1.135.195" port: 41699 } }
I20260923 22:12:40.261278 6130 leader_election.cc:288] T 6ce752740f874c27a225a283eca01839 P 9392cee28d9b4fe984b60718919f874d [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 61281de877044380a0093894dbb77b5e (127.1.135.194:39347), 8aa43ecf7d6e42e7b34e2d32e5b4d11e (127.1.135.195:41699)
I20260923 22:12:40.264420 6120 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 22:12:40.261541 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.135.193:42275 (local address 127.1.135.195:41699)
0923 22:12:40.261584 (+ 43us) server_negotiation.cc:206] Beginning negotiation
0923 22:12:40.261586 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0923 22:12:40.261766 (+ 180us) server_negotiation.cc:407] Connection header received
0923 22:12:40.261820 (+ 54us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 22:12:40.261821 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 22:12:40.261829 (+ 8us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 22:12:40.261876 (+ 47us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0923 22:12:40.262012 (+ 136us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 22:12:40.262693 (+ 681us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 22:12:40.263649 (+ 956us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 22:12:40.264033 (+ 384us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 22:12:40.264167 (+ 134us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 22:12:40.264183 (+ 16us) server_negotiation.cc:1035] Waiting for connection context
0923 22:12:40.264318 (+ 135us) server_negotiation.cc:299] Negotiation successful
0923 22:12:40.264352 (+ 34us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":19}
I20260923 22:12:40.264436 6133 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 22:12:40.261460 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.135.195:41699 (local address 127.1.135.193:42275)
0923 22:12:40.261700 (+ 240us) negotiation.cc:107] Waiting for socket to connect
0923 22:12:40.261708 (+ 8us) client_negotiation.cc:174] Beginning negotiation
0923 22:12:40.261771 (+ 63us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0923 22:12:40.261894 (+ 123us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0923 22:12:40.261894 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0923 22:12:40.261896 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0923 22:12:40.261951 (+ 55us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 22:12:40.261952 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 22:12:40.262815 (+ 863us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 22:12:40.262816 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 22:12:40.263558 (+ 742us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 22:12:40.263560 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 22:12:40.264168 (+ 608us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 22:12:40.264169 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 22:12:40.264242 (+ 73us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 22:12:40.264245 (+ 3us) client_negotiation.cc:769] Sending connection context
0923 22:12:40.264313 (+ 68us) client_negotiation.cc:240] Negotiation successful
0923 22:12:40.264352 (+ 39us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":230,"mutex_wait_us":107,"thread_start_us":79,"threads_started":1}
I20260923 22:12:40.264554 6123 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 22:12:40.261588 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.135.193:60151 (local address 127.1.135.194:39347)
0923 22:12:40.261640 (+ 52us) server_negotiation.cc:206] Beginning negotiation
0923 22:12:40.261643 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0923 22:12:40.261653 (+ 10us) server_negotiation.cc:407] Connection header received
0923 22:12:40.261691 (+ 38us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 22:12:40.261692 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 22:12:40.261701 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 22:12:40.261780 (+ 79us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0923 22:12:40.261936 (+ 156us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 22:12:40.262701 (+ 765us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 22:12:40.263655 (+ 954us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 22:12:40.264208 (+ 553us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 22:12:40.264298 (+ 90us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 22:12:40.264306 (+ 8us) server_negotiation.cc:1035] Waiting for connection context
0923 22:12:40.264460 (+ 154us) server_negotiation.cc:299] Negotiation successful
0923 22:12:40.264497 (+ 37us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":26}
I20260923 22:12:40.264544 5846 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 22:12:40.261460 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.135.194:39347 (local address 127.1.135.193:60151)
0923 22:12:40.261514 (+ 54us) negotiation.cc:107] Waiting for socket to connect
0923 22:12:40.261526 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0923 22:12:40.261587 (+ 61us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0923 22:12:40.261783 (+ 196us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0923 22:12:40.261784 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0923 22:12:40.261786 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0923 22:12:40.261870 (+ 84us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 22:12:40.261872 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 22:12:40.262766 (+ 894us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 22:12:40.262767 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 22:12:40.263584 (+ 817us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 22:12:40.263587 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 22:12:40.264298 (+ 711us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 22:12:40.264299 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 22:12:40.264382 (+ 83us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 22:12:40.264384 (+ 2us) client_negotiation.cc:769] Sending connection context
0923 22:12:40.264442 (+ 58us) client_negotiation.cc:240] Negotiation successful
0923 22:12:40.264478 (+ 36us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":33,"mutex_wait_us":43}
I20260923 22:12:40.264881 5931 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "6ce752740f874c27a225a283eca01839" candidate_uuid: "9392cee28d9b4fe984b60718919f874d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "61281de877044380a0093894dbb77b5e" is_pre_election: true
I20260923 22:12:40.264883 6064 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "6ce752740f874c27a225a283eca01839" candidate_uuid: "9392cee28d9b4fe984b60718919f874d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8aa43ecf7d6e42e7b34e2d32e5b4d11e" is_pre_election: true
I20260923 22:12:40.265017 6064 raft_consensus.cc:2454] T 6ce752740f874c27a225a283eca01839 P 8aa43ecf7d6e42e7b34e2d32e5b4d11e [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9392cee28d9b4fe984b60718919f874d in term 0.
I20260923 22:12:40.265033 5931 raft_consensus.cc:2454] T 6ce752740f874c27a225a283eca01839 P 61281de877044380a0093894dbb77b5e [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9392cee28d9b4fe984b60718919f874d in term 0.
I20260923 22:12:40.265210 5734 leader_election.cc:302] T 6ce752740f874c27a225a283eca01839 P 9392cee28d9b4fe984b60718919f874d [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8aa43ecf7d6e42e7b34e2d32e5b4d11e, 9392cee28d9b4fe984b60718919f874d; no voters:
I20260923 22:12:40.265333 6130 raft_consensus.cc:2781] T 6ce752740f874c27a225a283eca01839 P 9392cee28d9b4fe984b60718919f874d [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260923 22:12:40.265386 6130 raft_consensus.cc:482] T 6ce752740f874c27a225a283eca01839 P 9392cee28d9b4fe984b60718919f874d [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260923 22:12:40.265408 6130 raft_consensus.cc:3037] T 6ce752740f874c27a225a283eca01839 P 9392cee28d9b4fe984b60718919f874d [term 0 FOLLOWER]: Advancing to term 1
I20260923 22:12:40.266067 6130 raft_consensus.cc:504] T 6ce752740f874c27a225a283eca01839 P 9392cee28d9b4fe984b60718919f874d [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9392cee28d9b4fe984b60718919f874d" member_type: VOTER last_known_addr { host: "127.1.135.193" port: 38845 } } peers { permanent_uuid: "61281de877044380a0093894dbb77b5e" member_type: VOTER last_known_addr { host: "127.1.135.194" port: 39347 } } peers { permanent_uuid: "8aa43ecf7d6e42e7b34e2d32e5b4d11e" member_type: VOTER last_known_addr { host: "127.1.135.195" port: 41699 } }
I20260923 22:12:40.266198 6130 leader_election.cc:288] T 6ce752740f874c27a225a283eca01839 P 9392cee28d9b4fe984b60718919f874d [CANDIDATE]: Term 1 election: Requested vote from peers 61281de877044380a0093894dbb77b5e (127.1.135.194:39347), 8aa43ecf7d6e42e7b34e2d32e5b4d11e (127.1.135.195:41699)
I20260923 22:12:40.266335 5931 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "6ce752740f874c27a225a283eca01839" candidate_uuid: "9392cee28d9b4fe984b60718919f874d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "61281de877044380a0093894dbb77b5e"
I20260923 22:12:40.266357 6064 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "6ce752740f874c27a225a283eca01839" candidate_uuid: "9392cee28d9b4fe984b60718919f874d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8aa43ecf7d6e42e7b34e2d32e5b4d11e"
I20260923 22:12:40.266409 6064 raft_consensus.cc:3037] T 6ce752740f874c27a225a283eca01839 P 8aa43ecf7d6e42e7b34e2d32e5b4d11e [term 0 FOLLOWER]: Advancing to term 1
I20260923 22:12:40.266423 5931 raft_consensus.cc:3037] T 6ce752740f874c27a225a283eca01839 P 61281de877044380a0093894dbb77b5e [term 0 FOLLOWER]: Advancing to term 1
I20260923 22:12:40.267067 6064 raft_consensus.cc:2454] T 6ce752740f874c27a225a283eca01839 P 8aa43ecf7d6e42e7b34e2d32e5b4d11e [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 9392cee28d9b4fe984b60718919f874d in term 1.
I20260923 22:12:40.267153 5931 raft_consensus.cc:2454] T 6ce752740f874c27a225a283eca01839 P 61281de877044380a0093894dbb77b5e [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 9392cee28d9b4fe984b60718919f874d in term 1.
I20260923 22:12:40.267220 5734 leader_election.cc:302] T 6ce752740f874c27a225a283eca01839 P 9392cee28d9b4fe984b60718919f874d [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8aa43ecf7d6e42e7b34e2d32e5b4d11e, 9392cee28d9b4fe984b60718919f874d; no voters:
I20260923 22:12:40.267311 6130 raft_consensus.cc:2781] T 6ce752740f874c27a225a283eca01839 P 9392cee28d9b4fe984b60718919f874d [term 1 FOLLOWER]: Leader election won for term 1
I20260923 22:12:40.267460 6130 raft_consensus.cc:686] T 6ce752740f874c27a225a283eca01839 P 9392cee28d9b4fe984b60718919f874d [term 1 LEADER]: Becoming Leader. State: Replica: 9392cee28d9b4fe984b60718919f874d, State: Running, Role: LEADER
I20260923 22:12:40.267530 6130 consensus_queue.cc:237] T 6ce752740f874c27a225a283eca01839 P 9392cee28d9b4fe984b60718919f874d [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9392cee28d9b4fe984b60718919f874d" member_type: VOTER last_known_addr { host: "127.1.135.193" port: 38845 } } peers { permanent_uuid: "61281de877044380a0093894dbb77b5e" member_type: VOTER last_known_addr { host: "127.1.135.194" port: 39347 } } peers { permanent_uuid: "8aa43ecf7d6e42e7b34e2d32e5b4d11e" member_type: VOTER last_known_addr { host: "127.1.135.195" port: 41699 } }
I20260923 22:12:40.268342 5654 catalog_manager.cc:5495] T 6ce752740f874c27a225a283eca01839 P 9392cee28d9b4fe984b60718919f874d reported cstate change: term changed from 0 to 1, leader changed from <none> to 9392cee28d9b4fe984b60718919f874d (127.1.135.193). New cstate: current_term: 1 leader_uuid: "9392cee28d9b4fe984b60718919f874d" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9392cee28d9b4fe984b60718919f874d" member_type: VOTER last_known_addr { host: "127.1.135.193" port: 38845 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "61281de877044380a0093894dbb77b5e" member_type: VOTER last_known_addr { host: "127.1.135.194" port: 39347 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "8aa43ecf7d6e42e7b34e2d32e5b4d11e" member_type: VOTER last_known_addr { host: "127.1.135.195" port: 41699 } health_report { overall_health: UNKNOWN } } }
I20260923 22:12:40.284386 6122 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 22:12:40.282365 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47866 (local address 127.1.135.193:38845)
0923 22:12:40.282417 (+ 52us) server_negotiation.cc:206] Beginning negotiation
0923 22:12:40.282419 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0923 22:12:40.282427 (+ 8us) server_negotiation.cc:407] Connection header received
0923 22:12:40.282495 (+ 68us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 22:12:40.282496 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 22:12:40.282510 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 22:12:40.282571 (+ 61us) server_negotiation.cc:226] Negotiated authn=TOKEN
0923 22:12:40.282649 (+ 78us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 22:12:40.283085 (+ 436us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 22:12:40.283758 (+ 673us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 22:12:40.283924 (+ 166us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 22:12:40.283972 (+ 48us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 22:12:40.284129 (+ 157us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0923 22:12:40.284218 (+ 89us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0923 22:12:40.284279 (+ 61us) server_negotiation.cc:1035] Waiting for connection context
0923 22:12:40.284311 (+ 32us) server_negotiation.cc:299] Negotiation successful
0923 22:12:40.284326 (+ 15us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":32}
W20260923 22:12:40.284791 5845 tablet.cc:2369] T 6ce752740f874c27a225a283eca01839 P 9392cee28d9b4fe984b60718919f874d: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260923 22:12:40.286080 6064 raft_consensus.cc:1261] T 6ce752740f874c27a225a283eca01839 P 8aa43ecf7d6e42e7b34e2d32e5b4d11e [term 1 FOLLOWER]: Refusing update from remote peer 9392cee28d9b4fe984b60718919f874d: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260923 22:12:40.286080 5931 raft_consensus.cc:1261] T 6ce752740f874c27a225a283eca01839 P 61281de877044380a0093894dbb77b5e [term 1 FOLLOWER]: Refusing update from remote peer 9392cee28d9b4fe984b60718919f874d: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260923 22:12:40.286445 6134 consensus_queue.cc:1035] T 6ce752740f874c27a225a283eca01839 P 9392cee28d9b4fe984b60718919f874d [LEADER]: Connected to new peer: Peer: permanent_uuid: "8aa43ecf7d6e42e7b34e2d32e5b4d11e" member_type: VOTER last_known_addr { host: "127.1.135.195" port: 41699 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260923 22:12:40.286600 6130 consensus_queue.cc:1035] T 6ce752740f874c27a225a283eca01839 P 9392cee28d9b4fe984b60718919f874d [LEADER]: Connected to new peer: Peer: permanent_uuid: "61281de877044380a0093894dbb77b5e" member_type: VOTER last_known_addr { host: "127.1.135.194" port: 39347 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260923 22:12:40.289616 6138 mvcc.cc:205] Tried to move back new op lower bound from 7332665590929297408 to 7332665590856237056. Current Snapshot: MvccSnapshot[applied={T|T < 7332665590929297408 or (T in {7332665590929297408})}]
I20260923 22:12:40.291237 5654 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:40068:
table { table_name: "test-table" } modify_external_catalogs: true
I20260923 22:12:40.291328 5654 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:40068:
table { table_name: "test-table" } modify_external_catalogs: true
I20260923 22:12:40.292064 5654 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 8c7f315fc0a14568b2c45267e7193357: Sending DeleteTablet for 3 replicas of tablet 6ce752740f874c27a225a283eca01839
I20260923 22:12:40.292387 5778 tablet_service.cc:1514] Processing DeleteTablet for tablet 6ce752740f874c27a225a283eca01839 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-23 22:12:40 UTC) from {username='kudu', principal='kudu/127.1.135.254@KRBTEST.COM'} at 127.0.0.1:47858
I20260923 22:12:40.292436 5911 tablet_service.cc:1514] Processing DeleteTablet for tablet 6ce752740f874c27a225a283eca01839 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-23 22:12:40 UTC) from {username='kudu', principal='kudu/127.1.135.254@KRBTEST.COM'} at 127.0.0.1:54404
I20260923 22:12:40.292626 6148 tablet_replica.cc:330] T 6ce752740f874c27a225a283eca01839 P 61281de877044380a0093894dbb77b5e: stopping tablet replica
I20260923 22:12:40.292615 6044 tablet_service.cc:1514] Processing DeleteTablet for tablet 6ce752740f874c27a225a283eca01839 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-23 22:12:40 UTC) from {username='kudu', principal='kudu/127.1.135.254@KRBTEST.COM'} at 127.0.0.1:46544
I20260923 22:12:40.292713 6148 raft_consensus.cc:2229] T 6ce752740f874c27a225a283eca01839 P 61281de877044380a0093894dbb77b5e [term 1 FOLLOWER]: Raft consensus shutting down.
I20260923 22:12:40.292804 6148 raft_consensus.cc:2258] T 6ce752740f874c27a225a283eca01839 P 61281de877044380a0093894dbb77b5e [term 1 FOLLOWER]: Raft consensus is shut down!
I20260923 22:12:40.292829 1567 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskRiQe_a/build/release/bin/kudu with pid 5716
I20260923 22:12:40.293715 6149 tablet_replica.cc:330] T 6ce752740f874c27a225a283eca01839 P 8aa43ecf7d6e42e7b34e2d32e5b4d11e: stopping tablet replica
I20260923 22:12:40.293745 6148 ts_tablet_manager.cc:1905] T 6ce752740f874c27a225a283eca01839 P 61281de877044380a0093894dbb77b5e: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260923 22:12:40.293902 6149 raft_consensus.cc:2229] T 6ce752740f874c27a225a283eca01839 P 8aa43ecf7d6e42e7b34e2d32e5b4d11e [term 1 FOLLOWER]: Raft consensus shutting down.
I20260923 22:12:40.294025 6149 raft_consensus.cc:2258] T 6ce752740f874c27a225a283eca01839 P 8aa43ecf7d6e42e7b34e2d32e5b4d11e [term 1 FOLLOWER]: Raft consensus is shut down!
I20260923 22:12:40.294399 6149 ts_tablet_manager.cc:1905] T 6ce752740f874c27a225a283eca01839 P 8aa43ecf7d6e42e7b34e2d32e5b4d11e: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260923 22:12:40.295481 6148 ts_tablet_manager.cc:1918] T 6ce752740f874c27a225a283eca01839 P 61281de877044380a0093894dbb77b5e: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260923 22:12:40.295563 6148 log.cc:1196] T 6ce752740f874c27a225a283eca01839 P 61281de877044380a0093894dbb77b5e: Deleting WAL directory at /tmp/dist-test-taskRiQe_a/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790201539513534-1567-0/minicluster-data/ts-1/wal/wals/6ce752740f874c27a225a283eca01839
I20260923 22:12:40.295796 6148 ts_tablet_manager.cc:1939] T 6ce752740f874c27a225a283eca01839 P 61281de877044380a0093894dbb77b5e: Deleting consensus metadata
I20260923 22:12:40.296217 5639 catalog_manager.cc:4841] TS 61281de877044380a0093894dbb77b5e (127.1.135.194:39347): tablet 6ce752740f874c27a225a283eca01839 (table test-table [id=43e0417169ea41e287e962a812a9f8f3]) successfully deleted
I20260923 22:12:40.296334 6149 ts_tablet_manager.cc:1918] T 6ce752740f874c27a225a283eca01839 P 8aa43ecf7d6e42e7b34e2d32e5b4d11e: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260923 22:12:40.296391 6149 log.cc:1196] T 6ce752740f874c27a225a283eca01839 P 8aa43ecf7d6e42e7b34e2d32e5b4d11e: Deleting WAL directory at /tmp/dist-test-taskRiQe_a/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790201539513534-1567-0/minicluster-data/ts-2/wal/wals/6ce752740f874c27a225a283eca01839
I20260923 22:12:40.296638 6149 ts_tablet_manager.cc:1939] T 6ce752740f874c27a225a283eca01839 P 8aa43ecf7d6e42e7b34e2d32e5b4d11e: Deleting consensus metadata
I20260923 22:12:40.297005 5641 catalog_manager.cc:4841] TS 8aa43ecf7d6e42e7b34e2d32e5b4d11e (127.1.135.195:41699): tablet 6ce752740f874c27a225a283eca01839 (table test-table [id=43e0417169ea41e287e962a812a9f8f3]) successfully deleted
W20260923 22:12:40.299365 5639 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.1.135.193:38845 (error 108)
I20260923 22:12:40.299660 1567 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskRiQe_a/build/release/bin/kudu with pid 5849
I20260923 22:12:40.299795 6118 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 22:12:40.299660 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.135.193:38845 (local address 127.0.0.1:47872)
0923 22:12:40.299709 (+ 49us) negotiation.cc:107] Waiting for socket to connect
0923 22:12:40.299760 (+ 51us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.135.193:38845: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":32}
W20260923 22:12:40.300344 5639 catalog_manager.cc:4584] TS 9392cee28d9b4fe984b60718919f874d (127.1.135.193:38845): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet 6ce752740f874c27a225a283eca01839: Network error: Client connection negotiation failed: client connection to 127.1.135.193:38845: connect: Connection refused (error 111)
I20260923 22:12:40.304737 1567 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskRiQe_a/build/release/bin/kudu with pid 5982
I20260923 22:12:40.308815 1567 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskRiQe_a/build/release/bin/kudu with pid 5623
2026-09-23T22:12:40Z chronyd exiting
[ OK ] SecurityITest.TestDisableWebUI (1972 ms)
[ RUN ] SecurityITest.TestDisableAuthenticationEncryption
2026-09-23T22:12:40Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-09-23T22:12:40Z Disabled control of system clock
I20260923 22:12:40.335397 1567 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskRiQe_a/build/release/bin/kudu
/tmp/dist-test-taskRiQe_a/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskRiQe_a/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790201539513534-1567-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskRiQe_a/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790201539513534-1567-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskRiQe_a/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790201539513534-1567-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskRiQe_a/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790201539513534-1567-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.1.135.254:34441
--webserver_interface=127.1.135.254
--webserver_port=0
--builtin_ntp_servers=127.1.135.212:42511
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.1.135.254:34441
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--rpc_authentication=disabled
--rpc_encryption=disabled with env {}
W20260923 22:12:40.412853 6154 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260923 22:12:40.413152 6154 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260923 22:12:40.413200 6154 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260923 22:12:40.414634 6154 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260923 22:12:40.414697 6154 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260923 22:12:40.414721 6154 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260923 22:12:40.414752 6154 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260923 22:12:40.414777 6154 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260923 22:12:40.416203 6154 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.1.135.212:42511
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskRiQe_a/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790201539513534-1567-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskRiQe_a/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790201539513534-1567-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.1.135.254:34441
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.1.135.254:34441
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskRiQe_a/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790201539513534-1567-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.1.135.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.6154
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskRiQe_a/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790201539513534-1567-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 23 Sep 2026 20:53:37 UTC on ccycloud.quasar-rkviyx.root.comops.site
build id 4668083
I20260923 22:12:40.416469 6154 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260923 22:12:40.416751 6154 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260923 22:12:40.421301 6162 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260923 22:12:40.421378 6159 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260923 22:12:40.421296 6160 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260923 22:12:40.421700 6154 server_base.cc:1034] running on GCE node
I20260923 22:12:40.421937 6154 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260923 22:12:40.422219 6154 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260923 22:12:40.423369 6154 hybrid_clock.cc:648] HybridClock initialized: now 1790201560423360 us; error 32 us; skew 500 ppm
I20260923 22:12:40.424618 6154 webserver.cc:460] Webserver started at http://127.1.135.254:45559/ using document root <none> and password file <none>
I20260923 22:12:40.424856 6154 fs_manager.cc:359] Metadata directory not provided
I20260923 22:12:40.424919 6154 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260923 22:12:40.425052 6154 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260923 22:12:40.426211 6154 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskRiQe_a/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790201539513534-1567-0/minicluster-data/master-0/data/instance:
uuid: "8ddc15910ca54f8f83d822c7af982a57"
format_stamp: "Formatted at 2026-09-23 22:12:40 on dist-test-slave-s2fj"
server_key: "88ed8fa5de78adeb517d65e6ff233f2a"
server_key_iv: "0589e35b72680256ccaf304717ed0a0a"
server_key_version: "encryptionkey@0"
I20260923 22:12:40.426548 6154 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskRiQe_a/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790201539513534-1567-0/minicluster-data/master-0/wal/instance:
uuid: "8ddc15910ca54f8f83d822c7af982a57"
format_stamp: "Formatted at 2026-09-23 22:12:40 on dist-test-slave-s2fj"
server_key: "88ed8fa5de78adeb517d65e6ff233f2a"
server_key_iv: "0589e35b72680256ccaf304717ed0a0a"
server_key_version: "encryptionkey@0"
I20260923 22:12:40.427770 6154 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.001s
I20260923 22:12:40.428529 6168 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260923 22:12:40.428691 6154 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.000s
I20260923 22:12:40.428783 6154 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskRiQe_a/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790201539513534-1567-0/minicluster-data/master-0/data,/tmp/dist-test-taskRiQe_a/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790201539513534-1567-0/minicluster-data/master-0/wal
uuid: "8ddc15910ca54f8f83d822c7af982a57"
format_stamp: "Formatted at 2026-09-23 22:12:40 on dist-test-slave-s2fj"
server_key: "88ed8fa5de78adeb517d65e6ff233f2a"
server_key_iv: "0589e35b72680256ccaf304717ed0a0a"
server_key_version: "encryptionkey@0"
I20260923 22:12:40.428864 6154 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskRiQe_a/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790201539513534-1567-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskRiQe_a/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790201539513534-1567-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskRiQe_a/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790201539513534-1567-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260923 22:12:40.432015 6154 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260923 22:12:40.432286 6154 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260923 22:12:40.432420 6154 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260923 22:12:40.436098 6154 rpc_server.cc:307] RPC server started. Bound to: 127.1.135.254:34441
I20260923 22:12:40.436163 6220 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.1.135.254:34441 every 8 connection(s)
I20260923 22:12:40.436465 6154 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskRiQe_a/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790201539513534-1567-0/minicluster-data/master-0/data/info.pb
I20260923 22:12:40.437047 6221 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260923 22:12:40.439124 1567 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskRiQe_a/build/release/bin/kudu as pid 6154
I20260923 22:12:40.439232 1567 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskRiQe_a/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790201539513534-1567-0/minicluster-data/master-0/wal/instance
I20260923 22:12:40.439420 1567 external_mini_cluster.cc:1428] Setting key a2c7a58ff45287c17b574fccd5091500
I20260923 22:12:40.439718 6221 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 8ddc15910ca54f8f83d822c7af982a57: Bootstrap starting.
I20260923 22:12:40.440326 6221 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 8ddc15910ca54f8f83d822c7af982a57: Neither blocks nor log segments found. Creating new log.
I20260923 22:12:40.440600 6221 log.cc:824] T 00000000000000000000000000000000 P 8ddc15910ca54f8f83d822c7af982a57: Log is configured to *not* fsync() on all Append() calls
I20260923 22:12:40.441340 6221 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 8ddc15910ca54f8f83d822c7af982a57: No bootstrap required, opened a new log
I20260923 22:12:40.441871 6224 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 22:12:40.439862 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38212 (local address 127.1.135.254:34441)
0923 22:12:40.440050 (+ 188us) server_negotiation.cc:206] Beginning negotiation
0923 22:12:40.440057 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0923 22:12:40.440117 (+ 60us) server_negotiation.cc:407] Connection header received
0923 22:12:40.440563 (+ 446us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 22:12:40.440570 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 22:12:40.440608 (+ 38us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 22:12:40.440755 (+ 147us) server_negotiation.cc:226] Negotiated authn=SASL
0923 22:12:40.441392 (+ 637us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0923 22:12:40.441396 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0923 22:12:40.441398 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0923 22:12:40.441403 (+ 5us) server_negotiation.cc:922] calling sasl_server_start()
0923 22:12:40.441462 (+ 59us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0923 22:12:40.441475 (+ 13us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0923 22:12:40.441548 (+ 73us) server_negotiation.cc:1035] Waiting for connection context
0923 22:12:40.441620 (+ 72us) server_negotiation.cc:299] Negotiation successful
0923 22:12:40.441679 (+ 59us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":111,"thread_start_us":69,"threads_started":1}
I20260923 22:12:40.442975 6221 raft_consensus.cc:348] T 00000000000000000000000000000000 P 8ddc15910ca54f8f83d822c7af982a57 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8ddc15910ca54f8f83d822c7af982a57" member_type: VOTER last_known_addr { host: "127.1.135.254" port: 34441 } }
I20260923 22:12:40.443094 6221 raft_consensus.cc:374] T 00000000000000000000000000000000 P 8ddc15910ca54f8f83d822c7af982a57 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260923 22:12:40.443115 6221 raft_consensus.cc:729] T 00000000000000000000000000000000 P 8ddc15910ca54f8f83d822c7af982a57 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 8ddc15910ca54f8f83d822c7af982a57, State: Initialized, Role: FOLLOWER
I20260923 22:12:40.443245 6221 consensus_queue.cc:260] T 00000000000000000000000000000000 P 8ddc15910ca54f8f83d822c7af982a57 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8ddc15910ca54f8f83d822c7af982a57" member_type: VOTER last_known_addr { host: "127.1.135.254" port: 34441 } }
I20260923 22:12:40.443565 6221 raft_consensus.cc:388] T 00000000000000000000000000000000 P 8ddc15910ca54f8f83d822c7af982a57 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260923 22:12:40.443619 6221 raft_consensus.cc:482] T 00000000000000000000000000000000 P 8ddc15910ca54f8f83d822c7af982a57 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260923 22:12:40.443668 6221 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 8ddc15910ca54f8f83d822c7af982a57 [term 0 FOLLOWER]: Advancing to term 1
I20260923 22:12:40.444199 6221 raft_consensus.cc:504] T 00000000000000000000000000000000 P 8ddc15910ca54f8f83d822c7af982a57 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8ddc15910ca54f8f83d822c7af982a57" member_type: VOTER last_known_addr { host: "127.1.135.254" port: 34441 } }
I20260923 22:12:40.444300 6221 leader_election.cc:302] T 00000000000000000000000000000000 P 8ddc15910ca54f8f83d822c7af982a57 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 8ddc15910ca54f8f83d822c7af982a57; no voters:
I20260923 22:12:40.444458 6221 leader_election.cc:288] T 00000000000000000000000000000000 P 8ddc15910ca54f8f83d822c7af982a57 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260923 22:12:40.444480 6226 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 8ddc15910ca54f8f83d822c7af982a57 [term 1 FOLLOWER]: Leader election won for term 1
I20260923 22:12:40.444612 6226 raft_consensus.cc:686] T 00000000000000000000000000000000 P 8ddc15910ca54f8f83d822c7af982a57 [term 1 LEADER]: Becoming Leader. State: Replica: 8ddc15910ca54f8f83d822c7af982a57, State: Running, Role: LEADER
I20260923 22:12:40.444728 6221 sys_catalog.cc:564] T 00000000000000000000000000000000 P 8ddc15910ca54f8f83d822c7af982a57 [sys.catalog]: configured and running, proceeding with master startup.
I20260923 22:12:40.444707 6226 consensus_queue.cc:237] T 00000000000000000000000000000000 P 8ddc15910ca54f8f83d822c7af982a57 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8ddc15910ca54f8f83d822c7af982a57" member_type: VOTER last_known_addr { host: "127.1.135.254" port: 34441 } }
I20260923 22:12:40.445065 6228 sys_catalog.cc:455] T 00000000000000000000000000000000 P 8ddc15910ca54f8f83d822c7af982a57 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 8ddc15910ca54f8f83d822c7af982a57. Latest consensus state: current_term: 1 leader_uuid: "8ddc15910ca54f8f83d822c7af982a57" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8ddc15910ca54f8f83d822c7af982a57" member_type: VOTER last_known_addr { host: "127.1.135.254" port: 34441 } } }
I20260923 22:12:40.445166 6228 sys_catalog.cc:458] T 00000000000000000000000000000000 P 8ddc15910ca54f8f83d822c7af982a57 [sys.catalog]: This master's current role is: LEADER
I20260923 22:12:40.445345 6227 sys_catalog.cc:455] T 00000000000000000000000000000000 P 8ddc15910ca54f8f83d822c7af982a57 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "8ddc15910ca54f8f83d822c7af982a57" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8ddc15910ca54f8f83d822c7af982a57" member_type: VOTER last_known_addr { host: "127.1.135.254" port: 34441 } } }
I20260923 22:12:40.445374 6233 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260923 22:12:40.445870 6233 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260923 22:12:40.446357 6227 sys_catalog.cc:458] T 00000000000000000000000000000000 P 8ddc15910ca54f8f83d822c7af982a57 [sys.catalog]: This master's current role is: LEADER
I20260923 22:12:40.452921 6233 catalog_manager.cc:1347] Generated new cluster ID: 8c054acd82d44afeb3c9f7a9d1f9bf0b
I20260923 22:12:40.452988 6233 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260923 22:12:40.461373 6233 catalog_manager.cc:1370] Generated new certificate authority record
I20260923 22:12:40.461520 6233 catalog_manager.cc:1504] Loading token signing keys...
I20260923 22:12:40.474078 6233 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 8ddc15910ca54f8f83d822c7af982a57: Generated new TSK 0
I20260923 22:12:40.474259 6233 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260923 22:12:40.484362 1567 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskRiQe_a/build/release/bin/kudu
/tmp/dist-test-taskRiQe_a/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskRiQe_a/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790201539513534-1567-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskRiQe_a/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790201539513534-1567-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskRiQe_a/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790201539513534-1567-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskRiQe_a/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790201539513534-1567-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.1.135.193:0
--local_ip_for_outbound_sockets=127.1.135.193
--webserver_interface=127.1.135.193
--webserver_port=0
--tserver_master_addrs=127.1.135.254:34441
--builtin_ntp_servers=127.1.135.212:42511
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled
--rpc_encryption=disabled with env {}
W20260923 22:12:40.563295 6245 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260923 22:12:40.563479 6245 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260923 22:12:40.563500 6245 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260923 22:12:40.564689 6245 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260923 22:12:40.564729 6245 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260923 22:12:40.564754 6245 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.1.135.193
I20260923 22:12:40.566044 6245 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.1.135.212:42511
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskRiQe_a/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790201539513534-1567-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskRiQe_a/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790201539513534-1567-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.1.135.193:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskRiQe_a/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790201539513534-1567-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.1.135.193
--webserver_port=0
--tserver_master_addrs=127.1.135.254:34441
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.6245
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.1.135.193
--log_dir=/tmp/dist-test-taskRiQe_a/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790201539513534-1567-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 23 Sep 2026 20:53:37 UTC on ccycloud.quasar-rkviyx.root.comops.site
build id 4668083
I20260923 22:12:40.566294 6245 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260923 22:12:40.566574 6245 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260923 22:12:40.571298 6251 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260923 22:12:40.571431 6253 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260923 22:12:40.571309 6250 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260923 22:12:40.571530 6245 server_base.cc:1034] running on GCE node
I20260923 22:12:40.571729 6245 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260923 22:12:40.571931 6245 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260923 22:12:40.573087 6245 hybrid_clock.cc:648] HybridClock initialized: now 1790201560573071 us; error 30 us; skew 500 ppm
I20260923 22:12:40.574216 6245 webserver.cc:460] Webserver started at http://127.1.135.193:45109/ using document root <none> and password file <none>
I20260923 22:12:40.574432 6245 fs_manager.cc:359] Metadata directory not provided
I20260923 22:12:40.574489 6245 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260923 22:12:40.574630 6245 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260923 22:12:40.575583 6245 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskRiQe_a/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790201539513534-1567-0/minicluster-data/ts-0/data/instance:
uuid: "1008f24be17149f69112f72bec7cb889"
format_stamp: "Formatted at 2026-09-23 22:12:40 on dist-test-slave-s2fj"
server_key: "2f79fc8b8910ff29933944923b7b3d5a"
server_key_iv: "49cb3d0e86491e872c704d8a378cf496"
server_key_version: "encryptionkey@0"
I20260923 22:12:40.575927 6245 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskRiQe_a/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790201539513534-1567-0/minicluster-data/ts-0/wal/instance:
uuid: "1008f24be17149f69112f72bec7cb889"
format_stamp: "Formatted at 2026-09-23 22:12:40 on dist-test-slave-s2fj"
server_key: "2f79fc8b8910ff29933944923b7b3d5a"
server_key_iv: "49cb3d0e86491e872c704d8a378cf496"
server_key_version: "encryptionkey@0"
I20260923 22:12:40.577204 6245 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.001s
I20260923 22:12:40.577852 6259 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260923 22:12:40.578011 6245 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.001s
I20260923 22:12:40.578102 6245 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskRiQe_a/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790201539513534-1567-0/minicluster-data/ts-0/data,/tmp/dist-test-taskRiQe_a/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790201539513534-1567-0/minicluster-data/ts-0/wal
uuid: "1008f24be17149f69112f72bec7cb889"
format_stamp: "Formatted at 2026-09-23 22:12:40 on dist-test-slave-s2fj"
server_key: "2f79fc8b8910ff29933944923b7b3d5a"
server_key_iv: "49cb3d0e86491e872c704d8a378cf496"
server_key_version: "encryptionkey@0"
I20260923 22:12:40.578189 6245 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskRiQe_a/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790201539513534-1567-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskRiQe_a/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790201539513534-1567-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskRiQe_a/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790201539513534-1567-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260923 22:12:40.581411 6245 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260923 22:12:40.581636 6245 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260923 22:12:40.581764 6245 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260923 22:12:40.581954 6245 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260923 22:12:40.582222 6245 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260923 22:12:40.582270 6245 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260923 22:12:40.582307 6245 ts_tablet_manager.cc:610] Registered 0 tablets
I20260923 22:12:40.582334 6245 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260923 22:12:40.587776 6245 rpc_server.cc:307] RPC server started. Bound to: 127.1.135.193:44271
I20260923 22:12:40.587807 6372 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.1.135.193:44271 every 8 connection(s)
I20260923 22:12:40.588158 6245 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskRiQe_a/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790201539513534-1567-0/minicluster-data/ts-0/data/info.pb
I20260923 22:12:40.588630 1567 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskRiQe_a/build/release/bin/kudu as pid 6245
I20260923 22:12:40.588704 1567 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskRiQe_a/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790201539513534-1567-0/minicluster-data/ts-0/wal/instance
I20260923 22:12:40.588850 1567 external_mini_cluster.cc:1428] Setting key 0553d6a1a33ad503b9136eb811511770
I20260923 22:12:40.589778 1567 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskRiQe_a/build/release/bin/kudu
/tmp/dist-test-taskRiQe_a/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskRiQe_a/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790201539513534-1567-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskRiQe_a/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790201539513534-1567-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskRiQe_a/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790201539513534-1567-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskRiQe_a/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790201539513534-1567-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.1.135.194:0
--local_ip_for_outbound_sockets=127.1.135.194
--webserver_interface=127.1.135.194
--webserver_port=0
--tserver_master_addrs=127.1.135.254:34441
--builtin_ntp_servers=127.1.135.212:42511
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled
--rpc_encryption=disabled with env {}
I20260923 22:12:40.591914 6224 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 22:12:40.589358 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.135.193:50839 (local address 127.1.135.254:34441)
0923 22:12:40.589399 (+ 41us) server_negotiation.cc:206] Beginning negotiation
0923 22:12:40.589401 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0923 22:12:40.589661 (+ 260us) server_negotiation.cc:407] Connection header received
0923 22:12:40.590252 (+ 591us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 22:12:40.590253 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 22:12:40.590259 (+ 6us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 22:12:40.590300 (+ 41us) server_negotiation.cc:226] Negotiated authn=SASL
0923 22:12:40.591694 (+ 1394us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0923 22:12:40.591696 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0923 22:12:40.591696 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0923 22:12:40.591699 (+ 3us) server_negotiation.cc:922] calling sasl_server_start()
0923 22:12:40.591713 (+ 14us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0923 22:12:40.591717 (+ 4us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0923 22:12:40.591765 (+ 48us) server_negotiation.cc:1035] Waiting for connection context
0923 22:12:40.591857 (+ 92us) server_negotiation.cc:299] Negotiation successful
0923 22:12:40.591867 (+ 10us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":18}
I20260923 22:12:40.592154 6375 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 22:12:40.589359 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.135.254:34441 (local address 127.1.135.193:50839)
0923 22:12:40.589543 (+ 184us) negotiation.cc:107] Waiting for socket to connect
0923 22:12:40.589579 (+ 36us) client_negotiation.cc:174] Beginning negotiation
0923 22:12:40.590133 (+ 554us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0923 22:12:40.590369 (+ 236us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0923 22:12:40.590373 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0923 22:12:40.590382 (+ 9us) client_negotiation.cc:189] Negotiated authn=SASL
0923 22:12:40.591514 (+ 1132us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0923 22:12:40.591514 (+ 0us) client_negotiation.cc:656] calling sasl_client_start()
0923 22:12:40.591548 (+ 34us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0923 22:12:40.591582 (+ 34us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0923 22:12:40.591780 (+ 198us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0923 22:12:40.591785 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0923 22:12:40.591788 (+ 3us) client_negotiation.cc:769] Sending connection context
0923 22:12:40.591866 (+ 78us) client_negotiation.cc:240] Negotiation successful
0923 22:12:40.591928 (+ 62us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":112,"thread_start_us":80,"threads_started":1}
I20260923 22:12:40.592628 6373 heartbeater.cc:347] Connected to a master server at 127.1.135.254:34441
I20260923 22:12:40.592715 6373 heartbeater.cc:464] Registering TS with master...
I20260923 22:12:40.592846 6373 heartbeater.cc:511] Master 127.1.135.254:34441 requested a full tablet report, sending...
I20260923 22:12:40.593294 6185 ts_manager.cc:194] Registered new tserver with Master: 1008f24be17149f69112f72bec7cb889 (127.1.135.193:44271)
W20260923 22:12:40.668632 6376 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260923 22:12:40.668861 6376 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260923 22:12:40.668895 6376 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260923 22:12:40.670018 6376 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260923 22:12:40.670080 6376 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260923 22:12:40.670121 6376 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.1.135.194
I20260923 22:12:40.671412 6376 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.1.135.212:42511
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskRiQe_a/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790201539513534-1567-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskRiQe_a/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790201539513534-1567-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.1.135.194:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskRiQe_a/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790201539513534-1567-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.1.135.194
--webserver_port=0
--tserver_master_addrs=127.1.135.254:34441
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.6376
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.1.135.194
--log_dir=/tmp/dist-test-taskRiQe_a/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790201539513534-1567-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 23 Sep 2026 20:53:37 UTC on ccycloud.quasar-rkviyx.root.comops.site
build id 4668083
I20260923 22:12:40.671684 6376 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260923 22:12:40.671962 6376 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260923 22:12:40.676625 6381 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260923 22:12:40.676635 6384 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260923 22:12:40.676656 6382 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260923 22:12:40.676846 6376 server_base.cc:1034] running on GCE node
I20260923 22:12:40.677018 6376 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260923 22:12:40.677246 6376 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260923 22:12:40.678403 6376 hybrid_clock.cc:648] HybridClock initialized: now 1790201560678391 us; error 29 us; skew 500 ppm
I20260923 22:12:40.679457 6376 webserver.cc:460] Webserver started at http://127.1.135.194:39627/ using document root <none> and password file <none>
I20260923 22:12:40.679677 6376 fs_manager.cc:359] Metadata directory not provided
I20260923 22:12:40.679739 6376 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260923 22:12:40.679867 6376 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260923 22:12:40.680958 6376 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskRiQe_a/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790201539513534-1567-0/minicluster-data/ts-1/data/instance:
uuid: "8563409e59514fa0abc9a50a4f8c3e67"
format_stamp: "Formatted at 2026-09-23 22:12:40 on dist-test-slave-s2fj"
server_key: "635417ba68bbdabec12d973e3467e95f"
server_key_iv: "2ea8042296509a6e33371fe6e0d032d8"
server_key_version: "encryptionkey@0"
I20260923 22:12:40.681295 6376 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskRiQe_a/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790201539513534-1567-0/minicluster-data/ts-1/wal/instance:
uuid: "8563409e59514fa0abc9a50a4f8c3e67"
format_stamp: "Formatted at 2026-09-23 22:12:40 on dist-test-slave-s2fj"
server_key: "635417ba68bbdabec12d973e3467e95f"
server_key_iv: "2ea8042296509a6e33371fe6e0d032d8"
server_key_version: "encryptionkey@0"
I20260923 22:12:40.682552 6376 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.000s
I20260923 22:12:40.683266 6390 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260923 22:12:40.683427 6376 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260923 22:12:40.683516 6376 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskRiQe_a/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790201539513534-1567-0/minicluster-data/ts-1/data,/tmp/dist-test-taskRiQe_a/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790201539513534-1567-0/minicluster-data/ts-1/wal
uuid: "8563409e59514fa0abc9a50a4f8c3e67"
format_stamp: "Formatted at 2026-09-23 22:12:40 on dist-test-slave-s2fj"
server_key: "635417ba68bbdabec12d973e3467e95f"
server_key_iv: "2ea8042296509a6e33371fe6e0d032d8"
server_key_version: "encryptionkey@0"
I20260923 22:12:40.683594 6376 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskRiQe_a/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790201539513534-1567-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskRiQe_a/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790201539513534-1567-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskRiQe_a/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790201539513534-1567-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260923 22:12:40.686794 6376 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260923 22:12:40.687038 6376 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260923 22:12:40.687173 6376 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260923 22:12:40.687376 6376 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260923 22:12:40.687639 6376 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260923 22:12:40.687685 6376 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260923 22:12:40.687727 6376 ts_tablet_manager.cc:610] Registered 0 tablets
I20260923 22:12:40.687753 6376 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260923 22:12:40.693192 6376 rpc_server.cc:307] RPC server started. Bound to: 127.1.135.194:46221
I20260923 22:12:40.693254 6503 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.1.135.194:46221 every 8 connection(s)
I20260923 22:12:40.693535 6376 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskRiQe_a/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790201539513534-1567-0/minicluster-data/ts-1/data/info.pb
I20260923 22:12:40.694656 1567 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskRiQe_a/build/release/bin/kudu as pid 6376
I20260923 22:12:40.694768 1567 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskRiQe_a/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790201539513534-1567-0/minicluster-data/ts-1/wal/instance
I20260923 22:12:40.694933 1567 external_mini_cluster.cc:1428] Setting key 497e3d904291f094eb07bd141e4dc375
I20260923 22:12:40.695838 1567 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskRiQe_a/build/release/bin/kudu
/tmp/dist-test-taskRiQe_a/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskRiQe_a/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790201539513534-1567-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskRiQe_a/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790201539513534-1567-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskRiQe_a/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790201539513534-1567-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskRiQe_a/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790201539513534-1567-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.1.135.195:0
--local_ip_for_outbound_sockets=127.1.135.195
--webserver_interface=127.1.135.195
--webserver_port=0
--tserver_master_addrs=127.1.135.254:34441
--builtin_ntp_servers=127.1.135.212:42511
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled
--rpc_encryption=disabled with env {}
I20260923 22:12:40.696405 6224 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 22:12:40.694159 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.135.194:42681 (local address 127.1.135.254:34441)
0923 22:12:40.694214 (+ 55us) server_negotiation.cc:206] Beginning negotiation
0923 22:12:40.694229 (+ 15us) server_negotiation.cc:399] Waiting for connection header
0923 22:12:40.694392 (+ 163us) server_negotiation.cc:407] Connection header received
0923 22:12:40.694831 (+ 439us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 22:12:40.694833 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 22:12:40.694838 (+ 5us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 22:12:40.694893 (+ 55us) server_negotiation.cc:226] Negotiated authn=SASL
0923 22:12:40.696143 (+ 1250us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0923 22:12:40.696144 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0923 22:12:40.696145 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0923 22:12:40.696150 (+ 5us) server_negotiation.cc:922] calling sasl_server_start()
0923 22:12:40.696166 (+ 16us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0923 22:12:40.696170 (+ 4us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0923 22:12:40.696246 (+ 76us) server_negotiation.cc:1035] Waiting for connection context
0923 22:12:40.696294 (+ 48us) server_negotiation.cc:299] Negotiation successful
0923 22:12:40.696308 (+ 14us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":29}
I20260923 22:12:40.696519 6506 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 22:12:40.694137 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.135.254:34441 (local address 127.1.135.194:42681)
0923 22:12:40.694304 (+ 167us) negotiation.cc:107] Waiting for socket to connect
0923 22:12:40.694332 (+ 28us) client_negotiation.cc:174] Beginning negotiation
0923 22:12:40.694697 (+ 365us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0923 22:12:40.694957 (+ 260us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0923 22:12:40.694961 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0923 22:12:40.694970 (+ 9us) client_negotiation.cc:189] Negotiated authn=SASL
0923 22:12:40.695954 (+ 984us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0923 22:12:40.695954 (+ 0us) client_negotiation.cc:656] calling sasl_client_start()
0923 22:12:40.695992 (+ 38us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0923 22:12:40.696028 (+ 36us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0923 22:12:40.696244 (+ 216us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0923 22:12:40.696247 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0923 22:12:40.696249 (+ 2us) client_negotiation.cc:769] Sending connection context
0923 22:12:40.696300 (+ 51us) client_negotiation.cc:240] Negotiation successful
0923 22:12:40.696344 (+ 44us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":119,"thread_start_us":98,"threads_started":1}
I20260923 22:12:40.696906 6504 heartbeater.cc:347] Connected to a master server at 127.1.135.254:34441
I20260923 22:12:40.696988 6504 heartbeater.cc:464] Registering TS with master...
I20260923 22:12:40.697122 6504 heartbeater.cc:511] Master 127.1.135.254:34441 requested a full tablet report, sending...
I20260923 22:12:40.697454 6185 ts_manager.cc:194] Registered new tserver with Master: 8563409e59514fa0abc9a50a4f8c3e67 (127.1.135.194:46221)
W20260923 22:12:40.773494 6507 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260923 22:12:40.773676 6507 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260923 22:12:40.773696 6507 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260923 22:12:40.774770 6507 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260923 22:12:40.774802 6507 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260923 22:12:40.774824 6507 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.1.135.195
I20260923 22:12:40.775995 6507 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.1.135.212:42511
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskRiQe_a/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790201539513534-1567-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskRiQe_a/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790201539513534-1567-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.1.135.195:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskRiQe_a/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790201539513534-1567-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.1.135.195
--webserver_port=0
--tserver_master_addrs=127.1.135.254:34441
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.6507
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.1.135.195
--log_dir=/tmp/dist-test-taskRiQe_a/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790201539513534-1567-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 23 Sep 2026 20:53:37 UTC on ccycloud.quasar-rkviyx.root.comops.site
build id 4668083
I20260923 22:12:40.776237 6507 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260923 22:12:40.776468 6507 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260923 22:12:40.781054 6515 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260923 22:12:40.781093 6512 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260923 22:12:40.781193 6507 server_base.cc:1034] running on GCE node
W20260923 22:12:40.781329 6513 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260923 22:12:40.781576 6507 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260923 22:12:40.781810 6507 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260923 22:12:40.782965 6507 hybrid_clock.cc:648] HybridClock initialized: now 1790201560782954 us; error 29 us; skew 500 ppm
I20260923 22:12:40.784189 6507 webserver.cc:460] Webserver started at http://127.1.135.195:45153/ using document root <none> and password file <none>
I20260923 22:12:40.784467 6507 fs_manager.cc:359] Metadata directory not provided
I20260923 22:12:40.784509 6507 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260923 22:12:40.784682 6507 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260923 22:12:40.785684 6507 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskRiQe_a/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790201539513534-1567-0/minicluster-data/ts-2/data/instance:
uuid: "c445934bf13441eda87da98ca41c7b72"
format_stamp: "Formatted at 2026-09-23 22:12:40 on dist-test-slave-s2fj"
server_key: "f5e054a7b7569174e29e36e1f033217b"
server_key_iv: "a1b21e4803afeaed1fc47cfcdc9b612f"
server_key_version: "encryptionkey@0"
I20260923 22:12:40.786072 6507 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskRiQe_a/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790201539513534-1567-0/minicluster-data/ts-2/wal/instance:
uuid: "c445934bf13441eda87da98ca41c7b72"
format_stamp: "Formatted at 2026-09-23 22:12:40 on dist-test-slave-s2fj"
server_key: "f5e054a7b7569174e29e36e1f033217b"
server_key_iv: "a1b21e4803afeaed1fc47cfcdc9b612f"
server_key_version: "encryptionkey@0"
I20260923 22:12:40.787341 6507 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.003s
I20260923 22:12:40.787976 6521 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260923 22:12:40.788156 6507 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.001s
I20260923 22:12:40.788257 6507 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskRiQe_a/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790201539513534-1567-0/minicluster-data/ts-2/data,/tmp/dist-test-taskRiQe_a/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790201539513534-1567-0/minicluster-data/ts-2/wal
uuid: "c445934bf13441eda87da98ca41c7b72"
format_stamp: "Formatted at 2026-09-23 22:12:40 on dist-test-slave-s2fj"
server_key: "f5e054a7b7569174e29e36e1f033217b"
server_key_iv: "a1b21e4803afeaed1fc47cfcdc9b612f"
server_key_version: "encryptionkey@0"
I20260923 22:12:40.788324 6507 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskRiQe_a/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790201539513534-1567-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskRiQe_a/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790201539513534-1567-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskRiQe_a/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790201539513534-1567-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260923 22:12:40.791508 6507 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260923 22:12:40.791738 6507 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260923 22:12:40.791862 6507 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260923 22:12:40.792053 6507 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260923 22:12:40.792389 6507 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260923 22:12:40.792438 6507 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260923 22:12:40.792475 6507 ts_tablet_manager.cc:610] Registered 0 tablets
I20260923 22:12:40.792503 6507 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260923 22:12:40.798810 6507 rpc_server.cc:307] RPC server started. Bound to: 127.1.135.195:43723
I20260923 22:12:40.798859 6634 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.1.135.195:43723 every 8 connection(s)
I20260923 22:12:40.799140 6507 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskRiQe_a/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790201539513534-1567-0/minicluster-data/ts-2/data/info.pb
I20260923 22:12:40.800601 1567 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskRiQe_a/build/release/bin/kudu as pid 6507
I20260923 22:12:40.800700 1567 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskRiQe_a/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790201539513534-1567-0/minicluster-data/ts-2/wal/instance
I20260923 22:12:40.800897 1567 external_mini_cluster.cc:1428] Setting key dfca7e8d9d7cbb5ec8b41ccbda190b51
I20260923 22:12:40.802292 6224 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 22:12:40.799777 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.135.195:48335 (local address 127.1.135.254:34441)
0923 22:12:40.799839 (+ 62us) server_negotiation.cc:206] Beginning negotiation
0923 22:12:40.799841 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0923 22:12:40.800100 (+ 259us) server_negotiation.cc:407] Connection header received
0923 22:12:40.800817 (+ 717us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 22:12:40.800819 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 22:12:40.800827 (+ 8us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 22:12:40.800926 (+ 99us) server_negotiation.cc:226] Negotiated authn=SASL
0923 22:12:40.802122 (+ 1196us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0923 22:12:40.802123 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0923 22:12:40.802124 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0923 22:12:40.802127 (+ 3us) server_negotiation.cc:922] calling sasl_server_start()
0923 22:12:40.802141 (+ 14us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0923 22:12:40.802145 (+ 4us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0923 22:12:40.802186 (+ 41us) server_negotiation.cc:1035] Waiting for connection context
0923 22:12:40.802233 (+ 47us) server_negotiation.cc:299] Negotiation successful
0923 22:12:40.802243 (+ 10us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":37}
I20260923 22:12:40.802452 6637 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 22:12:40.799760 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.135.254:34441 (local address 127.1.135.195:48335)
0923 22:12:40.799940 (+ 180us) negotiation.cc:107] Waiting for socket to connect
0923 22:12:40.799997 (+ 57us) client_negotiation.cc:174] Beginning negotiation
0923 22:12:40.800632 (+ 635us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0923 22:12:40.801057 (+ 425us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0923 22:12:40.801075 (+ 18us) client_negotiation.cc:394] Received NEGOTIATE response from server
0923 22:12:40.801085 (+ 10us) client_negotiation.cc:189] Negotiated authn=SASL
0923 22:12:40.801999 (+ 914us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0923 22:12:40.802000 (+ 1us) client_negotiation.cc:656] calling sasl_client_start()
0923 22:12:40.802024 (+ 24us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0923 22:12:40.802050 (+ 26us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0923 22:12:40.802185 (+ 135us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0923 22:12:40.802187 (+ 2us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0923 22:12:40.802189 (+ 2us) client_negotiation.cc:769] Sending connection context
0923 22:12:40.802233 (+ 44us) client_negotiation.cc:240] Negotiation successful
0923 22:12:40.802275 (+ 42us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":121,"thread_start_us":96,"threads_started":1}
I20260923 22:12:40.802806 6635 heartbeater.cc:347] Connected to a master server at 127.1.135.254:34441
I20260923 22:12:40.802899 6635 heartbeater.cc:464] Registering TS with master...
I20260923 22:12:40.803027 6635 heartbeater.cc:511] Master 127.1.135.254:34441 requested a full tablet report, sending...
I20260923 22:12:40.803328 6185 ts_manager.cc:194] Registered new tserver with Master: c445934bf13441eda87da98ca41c7b72 (127.1.135.195:43723)
I20260923 22:12:40.812479 1567 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260923 22:12:40.814641 6224 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 22:12:40.813729 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38224 (local address 127.1.135.254:34441)
0923 22:12:40.813784 (+ 55us) server_negotiation.cc:206] Beginning negotiation
0923 22:12:40.813786 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0923 22:12:40.813923 (+ 137us) server_negotiation.cc:407] Connection header received
0923 22:12:40.813960 (+ 37us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 22:12:40.813961 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 22:12:40.813965 (+ 4us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 22:12:40.814010 (+ 45us) server_negotiation.cc:226] Negotiated authn=SASL
0923 22:12:40.814448 (+ 438us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0923 22:12:40.814449 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0923 22:12:40.814449 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0923 22:12:40.814452 (+ 3us) server_negotiation.cc:922] calling sasl_server_start()
0923 22:12:40.814463 (+ 11us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0923 22:12:40.814466 (+ 3us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0923 22:12:40.814507 (+ 41us) server_negotiation.cc:1035] Waiting for connection context
0923 22:12:40.814537 (+ 30us) server_negotiation.cc:299] Negotiation successful
0923 22:12:40.814547 (+ 10us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":31}
I20260923 22:12:40.815932 6185 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:38224:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260923 22:12:40.817567 6185 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260923 22:12:40.821058 6643 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 22:12:40.819489 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.135.193:44271 (local address 127.0.0.1:46762)
0923 22:12:40.819599 (+ 110us) negotiation.cc:107] Waiting for socket to connect
0923 22:12:40.819635 (+ 36us) client_negotiation.cc:174] Beginning negotiation
0923 22:12:40.819689 (+ 54us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0923 22:12:40.820112 (+ 423us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0923 22:12:40.820113 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0923 22:12:40.820114 (+ 1us) client_negotiation.cc:189] Negotiated authn=SASL
0923 22:12:40.820644 (+ 530us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0923 22:12:40.820644 (+ 0us) client_negotiation.cc:656] calling sasl_client_start()
0923 22:12:40.820659 (+ 15us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0923 22:12:40.820681 (+ 22us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0923 22:12:40.820917 (+ 236us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0923 22:12:40.820923 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0923 22:12:40.820926 (+ 3us) client_negotiation.cc:769] Sending connection context
0923 22:12:40.820972 (+ 46us) client_negotiation.cc:240] Negotiation successful
0923 22:12:40.820985 (+ 13us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":80,"thread_start_us":56,"threads_started":1}
I20260923 22:12:40.821316 6644 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 22:12:40.819491 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.135.195:43723 (local address 127.0.0.1:50126)
0923 22:12:40.819737 (+ 246us) negotiation.cc:107] Waiting for socket to connect
0923 22:12:40.819766 (+ 29us) client_negotiation.cc:174] Beginning negotiation
0923 22:12:40.819813 (+ 47us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0923 22:12:40.820060 (+ 247us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0923 22:12:40.820065 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0923 22:12:40.820099 (+ 34us) client_negotiation.cc:189] Negotiated authn=SASL
0923 22:12:40.820749 (+ 650us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0923 22:12:40.820749 (+ 0us) client_negotiation.cc:656] calling sasl_client_start()
0923 22:12:40.820755 (+ 6us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0923 22:12:40.820766 (+ 11us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0923 22:12:40.821187 (+ 421us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0923 22:12:40.821189 (+ 2us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0923 22:12:40.821190 (+ 1us) client_negotiation.cc:769] Sending connection context
0923 22:12:40.821239 (+ 49us) client_negotiation.cc:240] Negotiation successful
0923 22:12:40.821251 (+ 12us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":224,"thread_start_us":80,"threads_started":1}
I20260923 22:12:40.821327 6648 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 22:12:40.819793 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46762 (local address 127.1.135.193:44271)
0923 22:12:40.819952 (+ 159us) server_negotiation.cc:206] Beginning negotiation
0923 22:12:40.819956 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0923 22:12:40.819964 (+ 8us) server_negotiation.cc:407] Connection header received
0923 22:12:40.819990 (+ 26us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 22:12:40.819993 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 22:12:40.820000 (+ 7us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 22:12:40.820090 (+ 90us) server_negotiation.cc:226] Negotiated authn=SASL
0923 22:12:40.820762 (+ 672us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0923 22:12:40.820768 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0923 22:12:40.820768 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0923 22:12:40.820772 (+ 4us) server_negotiation.cc:922] calling sasl_server_start()
0923 22:12:40.820807 (+ 35us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0923 22:12:40.820830 (+ 23us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0923 22:12:40.821177 (+ 347us) server_negotiation.cc:1035] Waiting for connection context
0923 22:12:40.821211 (+ 34us) server_negotiation.cc:299] Negotiation successful
0923 22:12:40.821228 (+ 17us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":125,"thread_start_us":89,"threads_started":1}
I20260923 22:12:40.821573 6646 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 22:12:40.819635 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50126 (local address 127.1.135.195:43723)
0923 22:12:40.819917 (+ 282us) server_negotiation.cc:206] Beginning negotiation
0923 22:12:40.819921 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0923 22:12:40.819932 (+ 11us) server_negotiation.cc:407] Connection header received
0923 22:12:40.819955 (+ 23us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 22:12:40.819957 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 22:12:40.819967 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 22:12:40.820044 (+ 77us) server_negotiation.cc:226] Negotiated authn=SASL
0923 22:12:40.820823 (+ 779us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0923 22:12:40.820829 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0923 22:12:40.820830 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0923 22:12:40.820833 (+ 3us) server_negotiation.cc:922] calling sasl_server_start()
0923 22:12:40.820864 (+ 31us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0923 22:12:40.820885 (+ 21us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0923 22:12:40.820961 (+ 76us) server_negotiation.cc:1035] Waiting for connection context
0923 22:12:40.821484 (+ 523us) server_negotiation.cc:299] Negotiation successful
0923 22:12:40.821499 (+ 15us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":240,"thread_start_us":61,"threads_started":1}
I20260923 22:12:40.821581 6645 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 22:12:40.819514 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.135.194:46221 (local address 127.0.0.1:45418)
0923 22:12:40.820353 (+ 839us) negotiation.cc:107] Waiting for socket to connect
0923 22:12:40.820363 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0923 22:12:40.820411 (+ 48us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0923 22:12:40.820829 (+ 418us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0923 22:12:40.820830 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0923 22:12:40.820833 (+ 3us) client_negotiation.cc:189] Negotiated authn=SASL
0923 22:12:40.821235 (+ 402us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0923 22:12:40.821236 (+ 1us) client_negotiation.cc:656] calling sasl_client_start()
0923 22:12:40.821242 (+ 6us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0923 22:12:40.821251 (+ 9us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0923 22:12:40.821470 (+ 219us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0923 22:12:40.821471 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0923 22:12:40.821472 (+ 1us) client_negotiation.cc:769] Sending connection context
0923 22:12:40.821512 (+ 40us) client_negotiation.cc:240] Negotiation successful
0923 22:12:40.821524 (+ 12us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":822,"thread_start_us":70,"threads_started":1}
I20260923 22:12:40.821924 6307 tablet_service.cc:1467] Processing CreateTablet for tablet 6356faab492c473683c27ab544e77c74 (DEFAULT_TABLE table=test-table [id=0c214b18ff4e434fa3f1c4a7fac34468]), partition=RANGE (key) PARTITION UNBOUNDED
I20260923 22:12:40.822098 6569 tablet_service.cc:1467] Processing CreateTablet for tablet 6356faab492c473683c27ab544e77c74 (DEFAULT_TABLE table=test-table [id=0c214b18ff4e434fa3f1c4a7fac34468]), partition=RANGE (key) PARTITION UNBOUNDED
I20260923 22:12:40.822353 6307 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 6356faab492c473683c27ab544e77c74. 1 dirs total, 0 dirs full, 0 dirs failed
I20260923 22:12:40.822419 6569 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 6356faab492c473683c27ab544e77c74. 1 dirs total, 0 dirs full, 0 dirs failed
I20260923 22:12:40.821759 6647 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 22:12:40.819709 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45418 (local address 127.1.135.194:46221)
0923 22:12:40.819883 (+ 174us) server_negotiation.cc:206] Beginning negotiation
0923 22:12:40.819888 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0923 22:12:40.820412 (+ 524us) server_negotiation.cc:407] Connection header received
0923 22:12:40.820473 (+ 61us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 22:12:40.820476 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 22:12:40.820486 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 22:12:40.820554 (+ 68us) server_negotiation.cc:226] Negotiated authn=SASL
0923 22:12:40.821324 (+ 770us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0923 22:12:40.821330 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0923 22:12:40.821331 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0923 22:12:40.821335 (+ 4us) server_negotiation.cc:922] calling sasl_server_start()
0923 22:12:40.821364 (+ 29us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0923 22:12:40.821392 (+ 28us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0923 22:12:40.821447 (+ 55us) server_negotiation.cc:1035] Waiting for connection context
0923 22:12:40.821656 (+ 209us) server_negotiation.cc:299] Negotiation successful
0923 22:12:40.821671 (+ 15us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":133,"thread_start_us":94,"threads_started":1}
I20260923 22:12:40.825640 6649 tablet_bootstrap.cc:492] T 6356faab492c473683c27ab544e77c74 P 1008f24be17149f69112f72bec7cb889: Bootstrap starting.
I20260923 22:12:40.826200 6649 tablet_bootstrap.cc:654] T 6356faab492c473683c27ab544e77c74 P 1008f24be17149f69112f72bec7cb889: Neither blocks nor log segments found. Creating new log.
I20260923 22:12:40.826347 6438 tablet_service.cc:1467] Processing CreateTablet for tablet 6356faab492c473683c27ab544e77c74 (DEFAULT_TABLE table=test-table [id=0c214b18ff4e434fa3f1c4a7fac34468]), partition=RANGE (key) PARTITION UNBOUNDED
I20260923 22:12:40.826512 6649 log.cc:824] T 6356faab492c473683c27ab544e77c74 P 1008f24be17149f69112f72bec7cb889: Log is configured to *not* fsync() on all Append() calls
I20260923 22:12:40.826675 6438 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 6356faab492c473683c27ab544e77c74. 1 dirs total, 0 dirs full, 0 dirs failed
I20260923 22:12:40.826676 6650 tablet_bootstrap.cc:492] T 6356faab492c473683c27ab544e77c74 P c445934bf13441eda87da98ca41c7b72: Bootstrap starting.
I20260923 22:12:40.827337 6650 tablet_bootstrap.cc:654] T 6356faab492c473683c27ab544e77c74 P c445934bf13441eda87da98ca41c7b72: Neither blocks nor log segments found. Creating new log.
I20260923 22:12:40.827451 6649 tablet_bootstrap.cc:492] T 6356faab492c473683c27ab544e77c74 P 1008f24be17149f69112f72bec7cb889: No bootstrap required, opened a new log
I20260923 22:12:40.827661 6649 ts_tablet_manager.cc:1397] T 6356faab492c473683c27ab544e77c74 P 1008f24be17149f69112f72bec7cb889: Time spent bootstrapping tablet: real 0.002s user 0.002s sys 0.000s
I20260923 22:12:40.827775 6650 log.cc:824] T 6356faab492c473683c27ab544e77c74 P c445934bf13441eda87da98ca41c7b72: Log is configured to *not* fsync() on all Append() calls
I20260923 22:12:40.828502 6650 tablet_bootstrap.cc:492] T 6356faab492c473683c27ab544e77c74 P c445934bf13441eda87da98ca41c7b72: No bootstrap required, opened a new log
I20260923 22:12:40.828600 6650 ts_tablet_manager.cc:1397] T 6356faab492c473683c27ab544e77c74 P c445934bf13441eda87da98ca41c7b72: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20260923 22:12:40.829409 6653 tablet_bootstrap.cc:492] T 6356faab492c473683c27ab544e77c74 P 8563409e59514fa0abc9a50a4f8c3e67: Bootstrap starting.
I20260923 22:12:40.829917 6653 tablet_bootstrap.cc:654] T 6356faab492c473683c27ab544e77c74 P 8563409e59514fa0abc9a50a4f8c3e67: Neither blocks nor log segments found. Creating new log.
I20260923 22:12:40.830180 6653 log.cc:824] T 6356faab492c473683c27ab544e77c74 P 8563409e59514fa0abc9a50a4f8c3e67: Log is configured to *not* fsync() on all Append() calls
I20260923 22:12:40.830185 6649 raft_consensus.cc:348] T 6356faab492c473683c27ab544e77c74 P 1008f24be17149f69112f72bec7cb889 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1008f24be17149f69112f72bec7cb889" member_type: VOTER last_known_addr { host: "127.1.135.193" port: 44271 } } peers { permanent_uuid: "c445934bf13441eda87da98ca41c7b72" member_type: VOTER last_known_addr { host: "127.1.135.195" port: 43723 } } peers { permanent_uuid: "8563409e59514fa0abc9a50a4f8c3e67" member_type: VOTER last_known_addr { host: "127.1.135.194" port: 46221 } }
I20260923 22:12:40.830345 6649 raft_consensus.cc:374] T 6356faab492c473683c27ab544e77c74 P 1008f24be17149f69112f72bec7cb889 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260923 22:12:40.830399 6649 raft_consensus.cc:729] T 6356faab492c473683c27ab544e77c74 P 1008f24be17149f69112f72bec7cb889 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 1008f24be17149f69112f72bec7cb889, State: Initialized, Role: FOLLOWER
I20260923 22:12:40.830528 6649 consensus_queue.cc:260] T 6356faab492c473683c27ab544e77c74 P 1008f24be17149f69112f72bec7cb889 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1008f24be17149f69112f72bec7cb889" member_type: VOTER last_known_addr { host: "127.1.135.193" port: 44271 } } peers { permanent_uuid: "c445934bf13441eda87da98ca41c7b72" member_type: VOTER last_known_addr { host: "127.1.135.195" port: 43723 } } peers { permanent_uuid: "8563409e59514fa0abc9a50a4f8c3e67" member_type: VOTER last_known_addr { host: "127.1.135.194" port: 46221 } }
I20260923 22:12:40.830883 6653 tablet_bootstrap.cc:492] T 6356faab492c473683c27ab544e77c74 P 8563409e59514fa0abc9a50a4f8c3e67: No bootstrap required, opened a new log
I20260923 22:12:40.830952 6653 ts_tablet_manager.cc:1397] T 6356faab492c473683c27ab544e77c74 P 8563409e59514fa0abc9a50a4f8c3e67: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20260923 22:12:40.831092 6373 heartbeater.cc:503] Master 127.1.135.254:34441 was elected leader, sending a full tablet report...
I20260923 22:12:40.831105 6649 ts_tablet_manager.cc:1428] T 6356faab492c473683c27ab544e77c74 P 1008f24be17149f69112f72bec7cb889: Time spent starting tablet: real 0.003s user 0.000s sys 0.003s
I20260923 22:12:40.831192 6650 raft_consensus.cc:348] T 6356faab492c473683c27ab544e77c74 P c445934bf13441eda87da98ca41c7b72 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1008f24be17149f69112f72bec7cb889" member_type: VOTER last_known_addr { host: "127.1.135.193" port: 44271 } } peers { permanent_uuid: "c445934bf13441eda87da98ca41c7b72" member_type: VOTER last_known_addr { host: "127.1.135.195" port: 43723 } } peers { permanent_uuid: "8563409e59514fa0abc9a50a4f8c3e67" member_type: VOTER last_known_addr { host: "127.1.135.194" port: 46221 } }
I20260923 22:12:40.831319 6650 raft_consensus.cc:374] T 6356faab492c473683c27ab544e77c74 P c445934bf13441eda87da98ca41c7b72 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260923 22:12:40.831350 6650 raft_consensus.cc:729] T 6356faab492c473683c27ab544e77c74 P c445934bf13441eda87da98ca41c7b72 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c445934bf13441eda87da98ca41c7b72, State: Initialized, Role: FOLLOWER
I20260923 22:12:40.831420 6650 consensus_queue.cc:260] T 6356faab492c473683c27ab544e77c74 P c445934bf13441eda87da98ca41c7b72 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1008f24be17149f69112f72bec7cb889" member_type: VOTER last_known_addr { host: "127.1.135.193" port: 44271 } } peers { permanent_uuid: "c445934bf13441eda87da98ca41c7b72" member_type: VOTER last_known_addr { host: "127.1.135.195" port: 43723 } } peers { permanent_uuid: "8563409e59514fa0abc9a50a4f8c3e67" member_type: VOTER last_known_addr { host: "127.1.135.194" port: 46221 } }
I20260923 22:12:40.831640 6650 ts_tablet_manager.cc:1428] T 6356faab492c473683c27ab544e77c74 P c445934bf13441eda87da98ca41c7b72: Time spent starting tablet: real 0.003s user 0.003s sys 0.000s
I20260923 22:12:40.831682 6635 heartbeater.cc:503] Master 127.1.135.254:34441 was elected leader, sending a full tablet report...
I20260923 22:12:40.833380 6653 raft_consensus.cc:348] T 6356faab492c473683c27ab544e77c74 P 8563409e59514fa0abc9a50a4f8c3e67 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1008f24be17149f69112f72bec7cb889" member_type: VOTER last_known_addr { host: "127.1.135.193" port: 44271 } } peers { permanent_uuid: "c445934bf13441eda87da98ca41c7b72" member_type: VOTER last_known_addr { host: "127.1.135.195" port: 43723 } } peers { permanent_uuid: "8563409e59514fa0abc9a50a4f8c3e67" member_type: VOTER last_known_addr { host: "127.1.135.194" port: 46221 } }
I20260923 22:12:40.833508 6653 raft_consensus.cc:374] T 6356faab492c473683c27ab544e77c74 P 8563409e59514fa0abc9a50a4f8c3e67 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260923 22:12:40.833550 6653 raft_consensus.cc:729] T 6356faab492c473683c27ab544e77c74 P 8563409e59514fa0abc9a50a4f8c3e67 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 8563409e59514fa0abc9a50a4f8c3e67, State: Initialized, Role: FOLLOWER
I20260923 22:12:40.833652 6653 consensus_queue.cc:260] T 6356faab492c473683c27ab544e77c74 P 8563409e59514fa0abc9a50a4f8c3e67 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1008f24be17149f69112f72bec7cb889" member_type: VOTER last_known_addr { host: "127.1.135.193" port: 44271 } } peers { permanent_uuid: "c445934bf13441eda87da98ca41c7b72" member_type: VOTER last_known_addr { host: "127.1.135.195" port: 43723 } } peers { permanent_uuid: "8563409e59514fa0abc9a50a4f8c3e67" member_type: VOTER last_known_addr { host: "127.1.135.194" port: 46221 } }
I20260923 22:12:40.833874 6653 ts_tablet_manager.cc:1428] T 6356faab492c473683c27ab544e77c74 P 8563409e59514fa0abc9a50a4f8c3e67: Time spent starting tablet: real 0.003s user 0.003s sys 0.000s
I20260923 22:12:40.833932 6504 heartbeater.cc:503] Master 127.1.135.254:34441 was elected leader, sending a full tablet report...
W20260923 22:12:40.838613 6374 tablet.cc:2369] T 6356faab492c473683c27ab544e77c74 P 1008f24be17149f69112f72bec7cb889: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260923 22:12:40.944190 6505 tablet.cc:2369] T 6356faab492c473683c27ab544e77c74 P 8563409e59514fa0abc9a50a4f8c3e67: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260923 22:12:41.034626 6655 raft_consensus.cc:482] T 6356faab492c473683c27ab544e77c74 P 1008f24be17149f69112f72bec7cb889 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260923 22:12:41.034797 6655 raft_consensus.cc:504] T 6356faab492c473683c27ab544e77c74 P 1008f24be17149f69112f72bec7cb889 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1008f24be17149f69112f72bec7cb889" member_type: VOTER last_known_addr { host: "127.1.135.193" port: 44271 } } peers { permanent_uuid: "c445934bf13441eda87da98ca41c7b72" member_type: VOTER last_known_addr { host: "127.1.135.195" port: 43723 } } peers { permanent_uuid: "8563409e59514fa0abc9a50a4f8c3e67" member_type: VOTER last_known_addr { host: "127.1.135.194" port: 46221 } }
I20260923 22:12:41.035156 6655 leader_election.cc:288] T 6356faab492c473683c27ab544e77c74 P 1008f24be17149f69112f72bec7cb889 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers c445934bf13441eda87da98ca41c7b72 (127.1.135.195:43723), 8563409e59514fa0abc9a50a4f8c3e67 (127.1.135.194:46221)
I20260923 22:12:41.036540 6647 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 22:12:41.035445 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.135.193:50893 (local address 127.1.135.194:46221)
0923 22:12:41.035493 (+ 48us) server_negotiation.cc:206] Beginning negotiation
0923 22:12:41.035495 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0923 22:12:41.035509 (+ 14us) server_negotiation.cc:407] Connection header received
0923 22:12:41.035589 (+ 80us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 22:12:41.035590 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 22:12:41.035596 (+ 6us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 22:12:41.035654 (+ 58us) server_negotiation.cc:226] Negotiated authn=SASL
0923 22:12:41.036292 (+ 638us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0923 22:12:41.036294 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0923 22:12:41.036294 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0923 22:12:41.036298 (+ 4us) server_negotiation.cc:922] calling sasl_server_start()
0923 22:12:41.036316 (+ 18us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0923 22:12:41.036320 (+ 4us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0923 22:12:41.036375 (+ 55us) server_negotiation.cc:1035] Waiting for connection context
0923 22:12:41.036438 (+ 63us) server_negotiation.cc:299] Negotiation successful
0923 22:12:41.036451 (+ 13us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":25}
I20260923 22:12:41.036542 6375 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 22:12:41.035376 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.135.194:46221 (local address 127.1.135.193:50893)
0923 22:12:41.035439 (+ 63us) negotiation.cc:107] Waiting for socket to connect
0923 22:12:41.035452 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0923 22:12:41.035515 (+ 63us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0923 22:12:41.035688 (+ 173us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0923 22:12:41.035689 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0923 22:12:41.035693 (+ 4us) client_negotiation.cc:189] Negotiated authn=SASL
0923 22:12:41.036185 (+ 492us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0923 22:12:41.036185 (+ 0us) client_negotiation.cc:656] calling sasl_client_start()
0923 22:12:41.036192 (+ 7us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0923 22:12:41.036203 (+ 11us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0923 22:12:41.036385 (+ 182us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0923 22:12:41.036386 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0923 22:12:41.036387 (+ 1us) client_negotiation.cc:769] Sending connection context
0923 22:12:41.036437 (+ 50us) client_negotiation.cc:240] Negotiation successful
0923 22:12:41.036450 (+ 13us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":40}
I20260923 22:12:41.036640 6646 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 22:12:41.035510 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.135.193:56415 (local address 127.1.135.195:43723)
0923 22:12:41.035562 (+ 52us) server_negotiation.cc:206] Beginning negotiation
0923 22:12:41.035564 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0923 22:12:41.035691 (+ 127us) server_negotiation.cc:407] Connection header received
0923 22:12:41.035730 (+ 39us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 22:12:41.035731 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 22:12:41.035735 (+ 4us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 22:12:41.035776 (+ 41us) server_negotiation.cc:226] Negotiated authn=SASL
0923 22:12:41.036424 (+ 648us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0923 22:12:41.036425 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0923 22:12:41.036426 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0923 22:12:41.036430 (+ 4us) server_negotiation.cc:922] calling sasl_server_start()
0923 22:12:41.036447 (+ 17us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0923 22:12:41.036452 (+ 5us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0923 22:12:41.036506 (+ 54us) server_negotiation.cc:1035] Waiting for connection context
0923 22:12:41.036556 (+ 50us) server_negotiation.cc:299] Negotiation successful
0923 22:12:41.036569 (+ 13us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":27}
I20260923 22:12:41.036836 6658 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 22:12:41.035376 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.135.195:43723 (local address 127.1.135.193:56415)
0923 22:12:41.035595 (+ 219us) negotiation.cc:107] Waiting for socket to connect
0923 22:12:41.035603 (+ 8us) client_negotiation.cc:174] Beginning negotiation
0923 22:12:41.035671 (+ 68us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0923 22:12:41.035810 (+ 139us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0923 22:12:41.035810 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0923 22:12:41.035813 (+ 3us) client_negotiation.cc:189] Negotiated authn=SASL
0923 22:12:41.036323 (+ 510us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0923 22:12:41.036324 (+ 1us) client_negotiation.cc:656] calling sasl_client_start()
0923 22:12:41.036328 (+ 4us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0923 22:12:41.036337 (+ 9us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0923 22:12:41.036509 (+ 172us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0923 22:12:41.036509 (+ 0us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0923 22:12:41.036510 (+ 1us) client_negotiation.cc:769] Sending connection context
0923 22:12:41.036557 (+ 47us) client_negotiation.cc:240] Negotiation successful
0923 22:12:41.036569 (+ 12us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":201,"mutex_wait_us":66,"thread_start_us":89,"threads_started":1}
I20260923 22:12:41.036924 6458 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "6356faab492c473683c27ab544e77c74" candidate_uuid: "1008f24be17149f69112f72bec7cb889" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8563409e59514fa0abc9a50a4f8c3e67" is_pre_election: true
I20260923 22:12:41.037083 6458 raft_consensus.cc:2454] T 6356faab492c473683c27ab544e77c74 P 8563409e59514fa0abc9a50a4f8c3e67 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 1008f24be17149f69112f72bec7cb889 in term 0.
I20260923 22:12:41.037233 6589 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "6356faab492c473683c27ab544e77c74" candidate_uuid: "1008f24be17149f69112f72bec7cb889" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c445934bf13441eda87da98ca41c7b72" is_pre_election: true
I20260923 22:12:41.037353 6260 leader_election.cc:302] T 6356faab492c473683c27ab544e77c74 P 1008f24be17149f69112f72bec7cb889 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1008f24be17149f69112f72bec7cb889, 8563409e59514fa0abc9a50a4f8c3e67; no voters:
I20260923 22:12:41.037377 6589 raft_consensus.cc:2454] T 6356faab492c473683c27ab544e77c74 P c445934bf13441eda87da98ca41c7b72 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 1008f24be17149f69112f72bec7cb889 in term 0.
I20260923 22:12:41.037503 6655 raft_consensus.cc:2781] T 6356faab492c473683c27ab544e77c74 P 1008f24be17149f69112f72bec7cb889 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260923 22:12:41.037581 6655 raft_consensus.cc:482] T 6356faab492c473683c27ab544e77c74 P 1008f24be17149f69112f72bec7cb889 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260923 22:12:41.037606 6655 raft_consensus.cc:3037] T 6356faab492c473683c27ab544e77c74 P 1008f24be17149f69112f72bec7cb889 [term 0 FOLLOWER]: Advancing to term 1
I20260923 22:12:41.038293 6655 raft_consensus.cc:504] T 6356faab492c473683c27ab544e77c74 P 1008f24be17149f69112f72bec7cb889 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1008f24be17149f69112f72bec7cb889" member_type: VOTER last_known_addr { host: "127.1.135.193" port: 44271 } } peers { permanent_uuid: "c445934bf13441eda87da98ca41c7b72" member_type: VOTER last_known_addr { host: "127.1.135.195" port: 43723 } } peers { permanent_uuid: "8563409e59514fa0abc9a50a4f8c3e67" member_type: VOTER last_known_addr { host: "127.1.135.194" port: 46221 } }
I20260923 22:12:41.038422 6655 leader_election.cc:288] T 6356faab492c473683c27ab544e77c74 P 1008f24be17149f69112f72bec7cb889 [CANDIDATE]: Term 1 election: Requested vote from peers c445934bf13441eda87da98ca41c7b72 (127.1.135.195:43723), 8563409e59514fa0abc9a50a4f8c3e67 (127.1.135.194:46221)
I20260923 22:12:41.038538 6589 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "6356faab492c473683c27ab544e77c74" candidate_uuid: "1008f24be17149f69112f72bec7cb889" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c445934bf13441eda87da98ca41c7b72"
I20260923 22:12:41.038568 6458 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "6356faab492c473683c27ab544e77c74" candidate_uuid: "1008f24be17149f69112f72bec7cb889" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8563409e59514fa0abc9a50a4f8c3e67"
I20260923 22:12:41.038615 6589 raft_consensus.cc:3037] T 6356faab492c473683c27ab544e77c74 P c445934bf13441eda87da98ca41c7b72 [term 0 FOLLOWER]: Advancing to term 1
I20260923 22:12:41.038626 6458 raft_consensus.cc:3037] T 6356faab492c473683c27ab544e77c74 P 8563409e59514fa0abc9a50a4f8c3e67 [term 0 FOLLOWER]: Advancing to term 1
I20260923 22:12:41.039472 6589 raft_consensus.cc:2454] T 6356faab492c473683c27ab544e77c74 P c445934bf13441eda87da98ca41c7b72 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 1008f24be17149f69112f72bec7cb889 in term 1.
I20260923 22:12:41.039481 6458 raft_consensus.cc:2454] T 6356faab492c473683c27ab544e77c74 P 8563409e59514fa0abc9a50a4f8c3e67 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 1008f24be17149f69112f72bec7cb889 in term 1.
I20260923 22:12:41.039647 6263 leader_election.cc:302] T 6356faab492c473683c27ab544e77c74 P 1008f24be17149f69112f72bec7cb889 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1008f24be17149f69112f72bec7cb889, c445934bf13441eda87da98ca41c7b72; no voters:
I20260923 22:12:41.039737 6655 raft_consensus.cc:2781] T 6356faab492c473683c27ab544e77c74 P 1008f24be17149f69112f72bec7cb889 [term 1 FOLLOWER]: Leader election won for term 1
I20260923 22:12:41.039876 6655 raft_consensus.cc:686] T 6356faab492c473683c27ab544e77c74 P 1008f24be17149f69112f72bec7cb889 [term 1 LEADER]: Becoming Leader. State: Replica: 1008f24be17149f69112f72bec7cb889, State: Running, Role: LEADER
I20260923 22:12:41.039969 6655 consensus_queue.cc:237] T 6356faab492c473683c27ab544e77c74 P 1008f24be17149f69112f72bec7cb889 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1008f24be17149f69112f72bec7cb889" member_type: VOTER last_known_addr { host: "127.1.135.193" port: 44271 } } peers { permanent_uuid: "c445934bf13441eda87da98ca41c7b72" member_type: VOTER last_known_addr { host: "127.1.135.195" port: 43723 } } peers { permanent_uuid: "8563409e59514fa0abc9a50a4f8c3e67" member_type: VOTER last_known_addr { host: "127.1.135.194" port: 46221 } }
I20260923 22:12:41.040853 6185 catalog_manager.cc:5495] T 6356faab492c473683c27ab544e77c74 P 1008f24be17149f69112f72bec7cb889 reported cstate change: term changed from 0 to 1, leader changed from <none> to 1008f24be17149f69112f72bec7cb889 (127.1.135.193). New cstate: current_term: 1 leader_uuid: "1008f24be17149f69112f72bec7cb889" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1008f24be17149f69112f72bec7cb889" member_type: VOTER last_known_addr { host: "127.1.135.193" port: 44271 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "c445934bf13441eda87da98ca41c7b72" member_type: VOTER last_known_addr { host: "127.1.135.195" port: 43723 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "8563409e59514fa0abc9a50a4f8c3e67" member_type: VOTER last_known_addr { host: "127.1.135.194" port: 46221 } health_report { overall_health: UNKNOWN } } }
I20260923 22:12:41.046568 6648 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 22:12:41.045582 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46774 (local address 127.1.135.193:44271)
0923 22:12:41.045620 (+ 38us) server_negotiation.cc:206] Beginning negotiation
0923 22:12:41.045621 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0923 22:12:41.045629 (+ 8us) server_negotiation.cc:407] Connection header received
0923 22:12:41.045707 (+ 78us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 22:12:41.045707 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 22:12:41.045716 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 22:12:41.045801 (+ 85us) server_negotiation.cc:226] Negotiated authn=SASL
0923 22:12:41.046373 (+ 572us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0923 22:12:41.046373 (+ 0us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0923 22:12:41.046374 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0923 22:12:41.046377 (+ 3us) server_negotiation.cc:922] calling sasl_server_start()
0923 22:12:41.046389 (+ 12us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0923 22:12:41.046393 (+ 4us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0923 22:12:41.046440 (+ 47us) server_negotiation.cc:1035] Waiting for connection context
0923 22:12:41.046472 (+ 32us) server_negotiation.cc:299] Negotiation successful
0923 22:12:41.046481 (+ 9us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":21}
I20260923 22:12:41.048154 6589 raft_consensus.cc:1261] T 6356faab492c473683c27ab544e77c74 P c445934bf13441eda87da98ca41c7b72 [term 1 FOLLOWER]: Refusing update from remote peer 1008f24be17149f69112f72bec7cb889: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260923 22:12:41.048162 6458 raft_consensus.cc:1261] T 6356faab492c473683c27ab544e77c74 P 8563409e59514fa0abc9a50a4f8c3e67 [term 1 FOLLOWER]: Refusing update from remote peer 1008f24be17149f69112f72bec7cb889: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260923 22:12:41.048445 6655 consensus_queue.cc:1035] T 6356faab492c473683c27ab544e77c74 P 1008f24be17149f69112f72bec7cb889 [LEADER]: Connected to new peer: Peer: permanent_uuid: "8563409e59514fa0abc9a50a4f8c3e67" member_type: VOTER last_known_addr { host: "127.1.135.194" port: 46221 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260923 22:12:41.048571 6659 consensus_queue.cc:1035] T 6356faab492c473683c27ab544e77c74 P 1008f24be17149f69112f72bec7cb889 [LEADER]: Connected to new peer: Peer: permanent_uuid: "c445934bf13441eda87da98ca41c7b72" member_type: VOTER last_known_addr { host: "127.1.135.195" port: 43723 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
W20260923 22:12:41.049600 6636 tablet.cc:2369] T 6356faab492c473683c27ab544e77c74 P c445934bf13441eda87da98ca41c7b72: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260923 22:12:41.050549 6662 mvcc.cc:205] Tried to move back new op lower bound from 7332665594050674688 to 7332665594020261888. Current Snapshot: MvccSnapshot[applied={T|T < 7332665594050674688}]
I20260923 22:12:41.051234 6664 mvcc.cc:205] Tried to move back new op lower bound from 7332665594050674688 to 7332665594020261888. Current Snapshot: MvccSnapshot[applied={T|T < 7332665594050674688 or (T in {7332665594050674688})}]
I20260923 22:12:41.052817 6184 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:38224:
table { table_name: "test-table" } modify_external_catalogs: true
I20260923 22:12:41.052911 6184 catalog_manager.cc:2645] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:38224:
table { table_name: "test-table" } modify_external_catalogs: true
I20260923 22:12:41.053643 6184 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 8ddc15910ca54f8f83d822c7af982a57: Sending DeleteTablet for 3 replicas of tablet 6356faab492c473683c27ab544e77c74
I20260923 22:12:41.053985 6307 tablet_service.cc:1514] Processing DeleteTablet for tablet 6356faab492c473683c27ab544e77c74 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-23 22:12:41 UTC) from {username='slave'} at 127.0.0.1:46762
I20260923 22:12:41.053984 6438 tablet_service.cc:1514] Processing DeleteTablet for tablet 6356faab492c473683c27ab544e77c74 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-23 22:12:41 UTC) from {username='slave'} at 127.0.0.1:45418
I20260923 22:12:41.053984 6569 tablet_service.cc:1514] Processing DeleteTablet for tablet 6356faab492c473683c27ab544e77c74 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-23 22:12:41 UTC) from {username='slave'} at 127.0.0.1:50126
I20260923 22:12:41.054230 6674 tablet_replica.cc:330] T 6356faab492c473683c27ab544e77c74 P c445934bf13441eda87da98ca41c7b72: stopping tablet replica
I20260923 22:12:41.054332 6674 raft_consensus.cc:2229] T 6356faab492c473683c27ab544e77c74 P c445934bf13441eda87da98ca41c7b72 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260923 22:12:41.054433 6674 raft_consensus.cc:2258] T 6356faab492c473683c27ab544e77c74 P c445934bf13441eda87da98ca41c7b72 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260923 22:12:41.054430 6673 tablet_replica.cc:330] T 6356faab492c473683c27ab544e77c74 P 8563409e59514fa0abc9a50a4f8c3e67: stopping tablet replica
I20260923 22:12:41.054484 1567 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskRiQe_a/build/release/bin/kudu with pid 6245
I20260923 22:12:41.054504 6673 raft_consensus.cc:2229] T 6356faab492c473683c27ab544e77c74 P 8563409e59514fa0abc9a50a4f8c3e67 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260923 22:12:41.054585 6673 raft_consensus.cc:2258] T 6356faab492c473683c27ab544e77c74 P 8563409e59514fa0abc9a50a4f8c3e67 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260923 22:12:41.055411 6673 ts_tablet_manager.cc:1905] T 6356faab492c473683c27ab544e77c74 P 8563409e59514fa0abc9a50a4f8c3e67: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260923 22:12:41.055411 6674 ts_tablet_manager.cc:1905] T 6356faab492c473683c27ab544e77c74 P c445934bf13441eda87da98ca41c7b72: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260923 22:12:41.057113 6673 ts_tablet_manager.cc:1918] T 6356faab492c473683c27ab544e77c74 P 8563409e59514fa0abc9a50a4f8c3e67: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260923 22:12:41.057174 6674 ts_tablet_manager.cc:1918] T 6356faab492c473683c27ab544e77c74 P c445934bf13441eda87da98ca41c7b72: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260923 22:12:41.057195 6673 log.cc:1196] T 6356faab492c473683c27ab544e77c74 P 8563409e59514fa0abc9a50a4f8c3e67: Deleting WAL directory at /tmp/dist-test-taskRiQe_a/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790201539513534-1567-0/minicluster-data/ts-1/wal/wals/6356faab492c473683c27ab544e77c74
I20260923 22:12:41.057245 6674 log.cc:1196] T 6356faab492c473683c27ab544e77c74 P c445934bf13441eda87da98ca41c7b72: Deleting WAL directory at /tmp/dist-test-taskRiQe_a/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790201539513534-1567-0/minicluster-data/ts-2/wal/wals/6356faab492c473683c27ab544e77c74
I20260923 22:12:41.057451 6673 ts_tablet_manager.cc:1939] T 6356faab492c473683c27ab544e77c74 P 8563409e59514fa0abc9a50a4f8c3e67: Deleting consensus metadata
I20260923 22:12:41.057510 6674 ts_tablet_manager.cc:1939] T 6356faab492c473683c27ab544e77c74 P c445934bf13441eda87da98ca41c7b72: Deleting consensus metadata
I20260923 22:12:41.057845 6169 catalog_manager.cc:4841] TS 8563409e59514fa0abc9a50a4f8c3e67 (127.1.135.194:46221): tablet 6356faab492c473683c27ab544e77c74 (table test-table [id=0c214b18ff4e434fa3f1c4a7fac34468]) successfully deleted
I20260923 22:12:41.057881 6172 catalog_manager.cc:4841] TS c445934bf13441eda87da98ca41c7b72 (127.1.135.195:43723): tablet 6356faab492c473683c27ab544e77c74 (table test-table [id=0c214b18ff4e434fa3f1c4a7fac34468]) successfully deleted
W20260923 22:12:41.060287 6171 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.1.135.193:44271 (error 108)
I20260923 22:12:41.060500 1567 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskRiQe_a/build/release/bin/kudu with pid 6376
I20260923 22:12:41.060710 6645 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 22:12:41.060582 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.135.193:44271 (local address 127.0.0.1:46780)
0923 22:12:41.060626 (+ 44us) negotiation.cc:107] Waiting for socket to connect
0923 22:12:41.060677 (+ 51us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.135.193:44271: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":26}
W20260923 22:12:41.061138 6171 catalog_manager.cc:4584] TS 1008f24be17149f69112f72bec7cb889 (127.1.135.193:44271): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet 6356faab492c473683c27ab544e77c74: Network error: Client connection negotiation failed: client connection to 127.1.135.193:44271: connect: Connection refused (error 111)
I20260923 22:12:41.066009 1567 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskRiQe_a/build/release/bin/kudu with pid 6507
I20260923 22:12:41.070112 1567 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskRiQe_a/build/release/bin/kudu with pid 6154
2026-09-23T22:12:41Z chronyd exiting
[ OK ] SecurityITest.TestDisableAuthenticationEncryption (760 ms)
[ RUN ] SecurityITest.TestJwtMiniCluster
I20260923 22:12:41.089483 1567 mini_oidc.cc:152] Starting JWKS server
I20260923 22:12:41.090536 1567 webserver.cc:460] Webserver started at https://127.0.0.1:32771/ using document root <none> and password file <none>
I20260923 22:12:41.090638 1567 mini_oidc.cc:189] Starting OIDC Discovery server
I20260923 22:12:41.090816 1567 webserver.cc:460] Webserver started at http://127.0.0.1:46425/ using document root <none> and password file <none>
2026-09-23T22:12:41Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-09-23T22:12:41Z Disabled control of system clock
I20260923 22:12:41.098279 1567 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskRiQe_a/build/release/bin/kudu
/tmp/dist-test-taskRiQe_a/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskRiQe_a/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1790201539513534-1567-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskRiQe_a/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1790201539513534-1567-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskRiQe_a/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1790201539513534-1567-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskRiQe_a/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1790201539513534-1567-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.1.135.254:43611
--webserver_interface=127.1.135.254
--webserver_port=0
--builtin_ntp_servers=127.1.135.212:45929
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.1.135.254:43611
--enable_jwt_token_auth=true
--jwks_url=https://localhost:32771/jwks.json
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--trusted_certificate_file=/tmp/dist-test-taskRiQe_a/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1790201539513534-1567-0/testchainca.cert with env {}
W20260923 22:12:41.176486 6681 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260923 22:12:41.176764 6681 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260923 22:12:41.176800 6681 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260923 22:12:41.178813 6681 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260923 22:12:41.178866 6681 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260923 22:12:41.178886 6681 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260923 22:12:41.178910 6681 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260923 22:12:41.178934 6681 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
W20260923 22:12:41.178958 6681 flags.cc:425] Enabled experimental flag: --enable_jwt_token_auth=true
W20260923 22:12:41.178979 6681 flags.cc:425] Enabled experimental flag: --jwks_url=https://localhost:32771/jwks.json
I20260923 22:12:41.181618 6681 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.1.135.212:45929
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskRiQe_a/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1790201539513534-1567-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskRiQe_a/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1790201539513534-1567-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.1.135.254:43611
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.1.135.254:43611
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--enable_jwt_token_auth=true
--jwks_url=https://localhost:32771/jwks.json
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskRiQe_a/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1790201539513534-1567-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.1.135.254
--webserver_port=0
--trusted_certificate_file=/tmp/dist-test-taskRiQe_a/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1790201539513534-1567-0/testchainca.cert
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.6681
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskRiQe_a/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1790201539513534-1567-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 23 Sep 2026 20:53:37 UTC on ccycloud.quasar-rkviyx.root.comops.site
build id 4668083
I20260923 22:12:41.181862 6681 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260923 22:12:41.182083 6681 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260923 22:12:41.191085 6687 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260923 22:12:41.191134 6689 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260923 22:12:41.191145 6686 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260923 22:12:41.191650 6681 server_base.cc:1034] running on GCE node
I20260923 22:12:41.192057 6681 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260923 22:12:41.192406 6681 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260923 22:12:41.193558 6681 hybrid_clock.cc:648] HybridClock initialized: now 1790201561193536 us; error 35 us; skew 500 ppm
I20260923 22:12:41.195122 6681 webserver.cc:460] Webserver started at http://127.1.135.254:45443/ using document root <none> and password file <none>
I20260923 22:12:41.195413 6681 fs_manager.cc:359] Metadata directory not provided
I20260923 22:12:41.195479 6681 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260923 22:12:41.195604 6681 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260923 22:12:41.197635 6681 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskRiQe_a/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1790201539513534-1567-0/minicluster-data/master-0/data/instance:
uuid: "072f3e3e23f649d6898a4eb08e46fc1b"
format_stamp: "Formatted at 2026-09-23 22:12:41 on dist-test-slave-s2fj"
server_key: "a5521c19540355a4a2894fc4fd64b59e"
server_key_iv: "21ce99802c2dcab3f718a3fbe6da5d60"
server_key_version: "encryptionkey@0"
I20260923 22:12:41.198136 6681 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskRiQe_a/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1790201539513534-1567-0/minicluster-data/master-0/wal/instance:
uuid: "072f3e3e23f649d6898a4eb08e46fc1b"
format_stamp: "Formatted at 2026-09-23 22:12:41 on dist-test-slave-s2fj"
server_key: "a5521c19540355a4a2894fc4fd64b59e"
server_key_iv: "21ce99802c2dcab3f718a3fbe6da5d60"
server_key_version: "encryptionkey@0"
I20260923 22:12:41.200232 6681 fs_manager.cc:691] Time spent creating directory manager: real 0.002s user 0.004s sys 0.000s
I20260923 22:12:41.201375 6695 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260923 22:12:41.201536 6681 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260923 22:12:41.201622 6681 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskRiQe_a/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1790201539513534-1567-0/minicluster-data/master-0/data,/tmp/dist-test-taskRiQe_a/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1790201539513534-1567-0/minicluster-data/master-0/wal
uuid: "072f3e3e23f649d6898a4eb08e46fc1b"
format_stamp: "Formatted at 2026-09-23 22:12:41 on dist-test-slave-s2fj"
server_key: "a5521c19540355a4a2894fc4fd64b59e"
server_key_iv: "21ce99802c2dcab3f718a3fbe6da5d60"
server_key_version: "encryptionkey@0"
I20260923 22:12:41.201696 6681 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskRiQe_a/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1790201539513534-1567-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskRiQe_a/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1790201539513534-1567-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskRiQe_a/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1790201539513534-1567-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260923 22:12:41.216727 6681 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260923 22:12:41.217013 6681 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260923 22:12:41.217149 6681 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260923 22:12:41.221091 6681 rpc_server.cc:307] RPC server started. Bound to: 127.1.135.254:43611
I20260923 22:12:41.221118 6747 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.1.135.254:43611 every 8 connection(s)
I20260923 22:12:41.221448 6681 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskRiQe_a/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1790201539513534-1567-0/minicluster-data/master-0/data/info.pb
I20260923 22:12:41.222327 6748 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260923 22:12:41.224519 6748 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 072f3e3e23f649d6898a4eb08e46fc1b: Bootstrap starting.
I20260923 22:12:41.225054 6748 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 072f3e3e23f649d6898a4eb08e46fc1b: Neither blocks nor log segments found. Creating new log.
I20260923 22:12:41.225308 6748 log.cc:824] T 00000000000000000000000000000000 P 072f3e3e23f649d6898a4eb08e46fc1b: Log is configured to *not* fsync() on all Append() calls
I20260923 22:12:41.225903 6748 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 072f3e3e23f649d6898a4eb08e46fc1b: No bootstrap required, opened a new log
I20260923 22:12:41.226857 1567 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskRiQe_a/build/release/bin/kudu as pid 6681
I20260923 22:12:41.226965 1567 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskRiQe_a/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1790201539513534-1567-0/minicluster-data/master-0/wal/instance
I20260923 22:12:41.227138 1567 external_mini_cluster.cc:1428] Setting key 8f7836337e297f8e88a365eed74e9fb4
I20260923 22:12:41.227666 6748 raft_consensus.cc:348] T 00000000000000000000000000000000 P 072f3e3e23f649d6898a4eb08e46fc1b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "072f3e3e23f649d6898a4eb08e46fc1b" member_type: VOTER last_known_addr { host: "127.1.135.254" port: 43611 } }
I20260923 22:12:41.227854 6748 raft_consensus.cc:374] T 00000000000000000000000000000000 P 072f3e3e23f649d6898a4eb08e46fc1b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260923 22:12:41.227888 6748 raft_consensus.cc:729] T 00000000000000000000000000000000 P 072f3e3e23f649d6898a4eb08e46fc1b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 072f3e3e23f649d6898a4eb08e46fc1b, State: Initialized, Role: FOLLOWER
I20260923 22:12:41.228013 6748 consensus_queue.cc:260] T 00000000000000000000000000000000 P 072f3e3e23f649d6898a4eb08e46fc1b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "072f3e3e23f649d6898a4eb08e46fc1b" member_type: VOTER last_known_addr { host: "127.1.135.254" port: 43611 } }
I20260923 22:12:41.228119 6748 raft_consensus.cc:388] T 00000000000000000000000000000000 P 072f3e3e23f649d6898a4eb08e46fc1b [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260923 22:12:41.228158 6748 raft_consensus.cc:482] T 00000000000000000000000000000000 P 072f3e3e23f649d6898a4eb08e46fc1b [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260923 22:12:41.228209 6748 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 072f3e3e23f649d6898a4eb08e46fc1b [term 0 FOLLOWER]: Advancing to term 1
I20260923 22:12:41.228688 6748 raft_consensus.cc:504] T 00000000000000000000000000000000 P 072f3e3e23f649d6898a4eb08e46fc1b [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "072f3e3e23f649d6898a4eb08e46fc1b" member_type: VOTER last_known_addr { host: "127.1.135.254" port: 43611 } }
I20260923 22:12:41.228785 6748 leader_election.cc:302] T 00000000000000000000000000000000 P 072f3e3e23f649d6898a4eb08e46fc1b [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 072f3e3e23f649d6898a4eb08e46fc1b; no voters:
I20260923 22:12:41.228947 6748 leader_election.cc:288] T 00000000000000000000000000000000 P 072f3e3e23f649d6898a4eb08e46fc1b [CANDIDATE]: Term 1 election: Requested vote from peers
I20260923 22:12:41.228976 6753 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 072f3e3e23f649d6898a4eb08e46fc1b [term 1 FOLLOWER]: Leader election won for term 1
I20260923 22:12:41.229076 6753 raft_consensus.cc:686] T 00000000000000000000000000000000 P 072f3e3e23f649d6898a4eb08e46fc1b [term 1 LEADER]: Becoming Leader. State: Replica: 072f3e3e23f649d6898a4eb08e46fc1b, State: Running, Role: LEADER
I20260923 22:12:41.229190 6748 sys_catalog.cc:564] T 00000000000000000000000000000000 P 072f3e3e23f649d6898a4eb08e46fc1b [sys.catalog]: configured and running, proceeding with master startup.
I20260923 22:12:41.229154 6753 consensus_queue.cc:237] T 00000000000000000000000000000000 P 072f3e3e23f649d6898a4eb08e46fc1b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "072f3e3e23f649d6898a4eb08e46fc1b" member_type: VOTER last_known_addr { host: "127.1.135.254" port: 43611 } }
I20260923 22:12:41.229504 6753 sys_catalog.cc:455] T 00000000000000000000000000000000 P 072f3e3e23f649d6898a4eb08e46fc1b [sys.catalog]: SysCatalogTable state changed. Reason: New leader 072f3e3e23f649d6898a4eb08e46fc1b. Latest consensus state: current_term: 1 leader_uuid: "072f3e3e23f649d6898a4eb08e46fc1b" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "072f3e3e23f649d6898a4eb08e46fc1b" member_type: VOTER last_known_addr { host: "127.1.135.254" port: 43611 } } }
I20260923 22:12:41.229596 6753 sys_catalog.cc:458] T 00000000000000000000000000000000 P 072f3e3e23f649d6898a4eb08e46fc1b [sys.catalog]: This master's current role is: LEADER
I20260923 22:12:41.229514 6754 sys_catalog.cc:455] T 00000000000000000000000000000000 P 072f3e3e23f649d6898a4eb08e46fc1b [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "072f3e3e23f649d6898a4eb08e46fc1b" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "072f3e3e23f649d6898a4eb08e46fc1b" member_type: VOTER last_known_addr { host: "127.1.135.254" port: 43611 } } }
I20260923 22:12:41.229751 6754 sys_catalog.cc:458] T 00000000000000000000000000000000 P 072f3e3e23f649d6898a4eb08e46fc1b [sys.catalog]: This master's current role is: LEADER
I20260923 22:12:41.229822 6759 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260923 22:12:41.230352 6759 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260923 22:12:41.231483 6770 mvcc.cc:205] Tried to move back new op lower bound from 7332665594802708480 to 7332665594795175936. Current Snapshot: MvccSnapshot[applied={T|T < 7332665594802708480}]
I20260923 22:12:41.231951 6759 catalog_manager.cc:1347] Generated new cluster ID: 4918e86790f046f5aea2dd03b7b9a320
I20260923 22:12:41.231998 6759 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260923 22:12:41.233481 6752 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 22:12:41.227696 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55148 (local address 127.1.135.254:43611)
0923 22:12:41.227902 (+ 206us) server_negotiation.cc:206] Beginning negotiation
0923 22:12:41.227910 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0923 22:12:41.227941 (+ 31us) server_negotiation.cc:407] Connection header received
0923 22:12:41.228050 (+ 109us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 22:12:41.228056 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 22:12:41.228184 (+ 128us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 22:12:41.228327 (+ 143us) server_negotiation.cc:226] Negotiated authn=SASL
0923 22:12:41.228534 (+ 207us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 22:12:41.230193 (+ 1659us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 22:12:41.231382 (+ 1189us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 22:12:41.231987 (+ 605us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 22:12:41.232441 (+ 454us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 22:12:41.232826 (+ 385us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0923 22:12:41.232829 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0923 22:12:41.232829 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0923 22:12:41.233013 (+ 184us) server_negotiation.cc:922] calling sasl_server_start()
0923 22:12:41.233052 (+ 39us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0923 22:12:41.233064 (+ 12us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0923 22:12:41.233130 (+ 66us) server_negotiation.cc:1035] Waiting for connection context
0923 22:12:41.233232 (+ 102us) server_negotiation.cc:299] Negotiation successful
0923 22:12:41.233299 (+ 67us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":120,"thread_start_us":78,"threads_started":1}
I20260923 22:12:41.234328 1567 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260923 22:12:41.243983 6759 catalog_manager.cc:1370] Generated new certificate authority record
I20260923 22:12:41.244477 6759 catalog_manager.cc:1504] Loading token signing keys...
I20260923 22:12:41.251492 6759 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 072f3e3e23f649d6898a4eb08e46fc1b: Generated new TSK 0
I20260923 22:12:41.251654 6759 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260923 22:12:41.275787 6752 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 22:12:41.260572 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55152 (local address 127.1.135.254:43611)
0923 22:12:41.260631 (+ 59us) server_negotiation.cc:206] Beginning negotiation
0923 22:12:41.260633 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0923 22:12:41.260774 (+ 141us) server_negotiation.cc:407] Connection header received
0923 22:12:41.260872 (+ 98us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 22:12:41.260873 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 22:12:41.260899 (+ 26us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 22:12:41.260962 (+ 63us) server_negotiation.cc:226] Negotiated authn=JWT
0923 22:12:41.261127 (+ 165us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 22:12:41.261643 (+ 516us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 22:12:41.262432 (+ 789us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 22:12:41.262613 (+ 181us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 22:12:41.262680 (+ 67us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 22:12:41.262893 (+ 213us) server_negotiation.cc:365] Received JWT_EXCHANGE NegotiatePB request
0923 22:12:41.275553 (+ 12660us) server_negotiation.cc:394] Sending RPC error: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'https://localhost:32771/jwks.json': curl error: SSL peer certificate or SSH remote key was not OK: SSL certificate problem: certificate has expired
0923 22:12:41.275710 (+ 157us) negotiation.cc:327] Negotiation complete: Not authorized: Server connection negotiation failed: server connection from 127.0.0.1:55152: Failed to load JWKS: Error downloading JWKS from 'https://localhost:32771/jwks.json': curl error: SSL peer certificate or SSH remote key was not OK: SSL certificate problem: certificate has expired
Metrics: {"server-negotiator.queue_time_us":35}
W20260923 22:12:41.275848 6752 negotiation.cc:344] Unauthorized connection attempt: Server connection negotiation failed: server connection from 127.0.0.1:55152: Failed to load JWKS: Error downloading JWKS from 'https://localhost:32771/jwks.json': curl error: SSL peer certificate or SSH remote key was not OK: SSL certificate problem: certificate has expired
E20260923 22:12:41.276077 6778 webserver.cc:533] Webserver: sslize failed: error:14094415:SSL routines:ssl3_read_bytes:sslv3 alert certificate expired
E20260923 22:12:41.276139 6778 webserver.cc:533] Webserver: error shutting down SSL connection: error:140E0197:SSL routines:SSL_shutdown:shutdown while in init
W20260923 22:12:41.276156 6777 negotiation.cc:337] Failed RPC negotiation. Trace:
0923 22:12:41.260506 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.135.254:43611 (local address 127.0.0.1:55152)
0923 22:12:41.260712 (+ 206us) negotiation.cc:107] Waiting for socket to connect
0923 22:12:41.260721 (+ 9us) client_negotiation.cc:174] Beginning negotiation
0923 22:12:41.260794 (+ 73us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0923 22:12:41.260985 (+ 191us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0923 22:12:41.260986 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0923 22:12:41.260988 (+ 2us) client_negotiation.cc:189] Negotiated authn=JWT
0923 22:12:41.261056 (+ 68us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 22:12:41.261058 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 22:12:41.261735 (+ 677us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 22:12:41.261736 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 22:12:41.262337 (+ 601us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 22:12:41.262338 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 22:12:41.262701 (+ 363us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 22:12:41.262702 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 22:12:41.262766 (+ 64us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 22:12:41.262782 (+ 16us) client_negotiation.cc:252] Sending JWT_EXCHANGE NegotiatePB request
0923 22:12:41.275689 (+ 12907us) client_negotiation.cc:283] Received error response from server: Runtime error: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'https://localhost:32771/jwks.json': curl error: SSL peer certificate or SSH remote key was not OK: SSL certificate problem: certificate has expired
0923 22:12:41.275812 (+ 123us) negotiation.cc:327] Negotiation complete: Runtime error: Client connection negotiation failed: client connection to 127.1.135.254:43611: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'https://localhost:32771/jwks.json': curl error: SSL peer certificate or SSH remote key was not OK: SSL certificate problem: certificate has expired
Metrics: {"client-negotiator.queue_time_us":149,"thread_start_us":97,"threads_started":1}
/data0/somelongdirectorytoavoidrpathissues/src/kudu/src/kudu/integration-tests/security-itest.cc:597: Failure
Failed
Bad status: Runtime error: Could not connect to the cluster: Client connection negotiation failed: client connection to 127.1.135.254:43611: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'https://localhost:32771/jwks.json': curl error: SSL peer certificate or SSH remote key was not OK: SSL certificate problem: certificate has expired
Google Test trace:
/data0/somelongdirectorytoavoidrpathissues/src/kudu/src/kudu/integration-tests/security-itest.cc:593: Valid JWT
I20260923 22:12:41.276762 1567 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskRiQe_a/build/release/bin/kudu with pid 6681
2026-09-23T22:12:41Z chronyd exiting
I20260923 22:12:41.312840 1567 test_util.cc:175] -----------------------------------------------
I20260923 22:12:41.312906 1567 test_util.cc:176] Had fatal failures, leaving test files at /tmp/dist-test-taskRiQe_a/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1790201539513534-1567-0
[ FAILED ] SecurityITest.TestJwtMiniCluster (224 ms)
[ RUN ] SecurityITest.TestJwtMiniClusterWithInvalidCert
I20260923 22:12:41.313578 1567 mini_oidc.cc:152] Starting JWKS server
I20260923 22:12:41.314189 1567 webserver.cc:460] Webserver started at https://127.0.0.1:46007/ using document root <none> and password file <none>
I20260923 22:12:41.314226 1567 mini_oidc.cc:189] Starting OIDC Discovery server
I20260923 22:12:41.314366 1567 webserver.cc:460] Webserver started at http://127.0.0.1:41397/ using document root <none> and password file <none>
2026-09-23T22:12:41Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-09-23T22:12:41Z Disabled control of system clock
I20260923 22:12:41.321398 1567 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskRiQe_a/build/release/bin/kudu
/tmp/dist-test-taskRiQe_a/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskRiQe_a/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1790201539513534-1567-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskRiQe_a/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1790201539513534-1567-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskRiQe_a/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1790201539513534-1567-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskRiQe_a/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1790201539513534-1567-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.1.135.254:41661
--webserver_interface=127.1.135.254
--webserver_port=0
--builtin_ntp_servers=127.1.135.212:44697
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.1.135.254:41661
--enable_jwt_token_auth=true
--jwks_url=https://localhost:46007/jwks.json
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--trusted_certificate_file=/tmp/dist-test-taskRiQe_a/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1790201539513534-1567-0/testchainca.cert with env {}
W20260923 22:12:41.397217 6784 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260923 22:12:41.397456 6784 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260923 22:12:41.397488 6784 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260923 22:12:41.398581 6784 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260923 22:12:41.398648 6784 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260923 22:12:41.398667 6784 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260923 22:12:41.398697 6784 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260923 22:12:41.398734 6784 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
W20260923 22:12:41.398761 6784 flags.cc:425] Enabled experimental flag: --enable_jwt_token_auth=true
W20260923 22:12:41.398788 6784 flags.cc:425] Enabled experimental flag: --jwks_url=https://localhost:46007/jwks.json
I20260923 22:12:41.400240 6784 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.1.135.212:44697
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskRiQe_a/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1790201539513534-1567-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskRiQe_a/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1790201539513534-1567-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.1.135.254:41661
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.1.135.254:41661
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--enable_jwt_token_auth=true
--jwks_url=https://localhost:46007/jwks.json
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskRiQe_a/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1790201539513534-1567-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.1.135.254
--webserver_port=0
--trusted_certificate_file=/tmp/dist-test-taskRiQe_a/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1790201539513534-1567-0/testchainca.cert
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.6784
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskRiQe_a/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1790201539513534-1567-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 23 Sep 2026 20:53:37 UTC on ccycloud.quasar-rkviyx.root.comops.site
build id 4668083
I20260923 22:12:41.400484 6784 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260923 22:12:41.400672 6784 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260923 22:12:41.405164 6789 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260923 22:12:41.405179 6790 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260923 22:12:41.405330 6784 server_base.cc:1034] running on GCE node
W20260923 22:12:41.405151 6792 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260923 22:12:41.405606 6784 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260923 22:12:41.405923 6784 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260923 22:12:41.407073 6784 hybrid_clock.cc:648] HybridClock initialized: now 1790201561407045 us; error 39 us; skew 500 ppm
I20260923 22:12:41.408214 6784 webserver.cc:460] Webserver started at http://127.1.135.254:44485/ using document root <none> and password file <none>
I20260923 22:12:41.408429 6784 fs_manager.cc:359] Metadata directory not provided
I20260923 22:12:41.408489 6784 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260923 22:12:41.408613 6784 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260923 22:12:41.409596 6784 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskRiQe_a/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1790201539513534-1567-0/minicluster-data/master-0/data/instance:
uuid: "d80d75bdd4b64c46985867e346e5921c"
format_stamp: "Formatted at 2026-09-23 22:12:41 on dist-test-slave-s2fj"
server_key: "f2eb1d7aad4d89293bcccc9979f0e124"
server_key_iv: "c64ef83173d1af2631e23d6d033cbe38"
server_key_version: "encryptionkey@0"
I20260923 22:12:41.409936 6784 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskRiQe_a/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1790201539513534-1567-0/minicluster-data/master-0/wal/instance:
uuid: "d80d75bdd4b64c46985867e346e5921c"
format_stamp: "Formatted at 2026-09-23 22:12:41 on dist-test-slave-s2fj"
server_key: "f2eb1d7aad4d89293bcccc9979f0e124"
server_key_iv: "c64ef83173d1af2631e23d6d033cbe38"
server_key_version: "encryptionkey@0"
I20260923 22:12:41.411087 6784 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260923 22:12:41.411701 6798 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260923 22:12:41.411855 6784 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260923 22:12:41.411942 6784 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskRiQe_a/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1790201539513534-1567-0/minicluster-data/master-0/data,/tmp/dist-test-taskRiQe_a/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1790201539513534-1567-0/minicluster-data/master-0/wal
uuid: "d80d75bdd4b64c46985867e346e5921c"
format_stamp: "Formatted at 2026-09-23 22:12:41 on dist-test-slave-s2fj"
server_key: "f2eb1d7aad4d89293bcccc9979f0e124"
server_key_iv: "c64ef83173d1af2631e23d6d033cbe38"
server_key_version: "encryptionkey@0"
I20260923 22:12:41.412027 6784 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskRiQe_a/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1790201539513534-1567-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskRiQe_a/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1790201539513534-1567-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskRiQe_a/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1790201539513534-1567-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260923 22:12:41.425611 6784 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260923 22:12:41.425875 6784 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260923 22:12:41.426011 6784 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260923 22:12:41.429724 6784 rpc_server.cc:307] RPC server started. Bound to: 127.1.135.254:41661
I20260923 22:12:41.429754 6850 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.1.135.254:41661 every 8 connection(s)
I20260923 22:12:41.430054 6784 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskRiQe_a/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1790201539513534-1567-0/minicluster-data/master-0/data/info.pb
I20260923 22:12:41.430966 6851 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260923 22:12:41.433125 6851 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P d80d75bdd4b64c46985867e346e5921c: Bootstrap starting.
I20260923 22:12:41.433722 6851 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P d80d75bdd4b64c46985867e346e5921c: Neither blocks nor log segments found. Creating new log.
I20260923 22:12:41.434013 6851 log.cc:824] T 00000000000000000000000000000000 P d80d75bdd4b64c46985867e346e5921c: Log is configured to *not* fsync() on all Append() calls
I20260923 22:12:41.434644 6851 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P d80d75bdd4b64c46985867e346e5921c: No bootstrap required, opened a new log
I20260923 22:12:41.435215 1567 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskRiQe_a/build/release/bin/kudu as pid 6784
I20260923 22:12:41.435321 1567 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskRiQe_a/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1790201539513534-1567-0/minicluster-data/master-0/wal/instance
I20260923 22:12:41.435521 1567 external_mini_cluster.cc:1428] Setting key d8c137508767a30311e6e6b353dacb0e
I20260923 22:12:41.436618 6851 raft_consensus.cc:348] T 00000000000000000000000000000000 P d80d75bdd4b64c46985867e346e5921c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d80d75bdd4b64c46985867e346e5921c" member_type: VOTER last_known_addr { host: "127.1.135.254" port: 41661 } }
I20260923 22:12:41.436776 6851 raft_consensus.cc:374] T 00000000000000000000000000000000 P d80d75bdd4b64c46985867e346e5921c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260923 22:12:41.436820 6851 raft_consensus.cc:729] T 00000000000000000000000000000000 P d80d75bdd4b64c46985867e346e5921c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d80d75bdd4b64c46985867e346e5921c, State: Initialized, Role: FOLLOWER
I20260923 22:12:41.436942 6851 consensus_queue.cc:260] T 00000000000000000000000000000000 P d80d75bdd4b64c46985867e346e5921c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d80d75bdd4b64c46985867e346e5921c" member_type: VOTER last_known_addr { host: "127.1.135.254" port: 41661 } }
I20260923 22:12:41.437034 6851 raft_consensus.cc:388] T 00000000000000000000000000000000 P d80d75bdd4b64c46985867e346e5921c [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260923 22:12:41.437072 6851 raft_consensus.cc:482] T 00000000000000000000000000000000 P d80d75bdd4b64c46985867e346e5921c [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260923 22:12:41.437124 6851 raft_consensus.cc:3037] T 00000000000000000000000000000000 P d80d75bdd4b64c46985867e346e5921c [term 0 FOLLOWER]: Advancing to term 1
I20260923 22:12:41.437623 6851 raft_consensus.cc:504] T 00000000000000000000000000000000 P d80d75bdd4b64c46985867e346e5921c [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d80d75bdd4b64c46985867e346e5921c" member_type: VOTER last_known_addr { host: "127.1.135.254" port: 41661 } }
I20260923 22:12:41.437744 6851 leader_election.cc:302] T 00000000000000000000000000000000 P d80d75bdd4b64c46985867e346e5921c [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: d80d75bdd4b64c46985867e346e5921c; no voters:
I20260923 22:12:41.437911 6851 leader_election.cc:288] T 00000000000000000000000000000000 P d80d75bdd4b64c46985867e346e5921c [CANDIDATE]: Term 1 election: Requested vote from peers
I20260923 22:12:41.437937 6856 raft_consensus.cc:2781] T 00000000000000000000000000000000 P d80d75bdd4b64c46985867e346e5921c [term 1 FOLLOWER]: Leader election won for term 1
I20260923 22:12:41.438064 6856 raft_consensus.cc:686] T 00000000000000000000000000000000 P d80d75bdd4b64c46985867e346e5921c [term 1 LEADER]: Becoming Leader. State: Replica: d80d75bdd4b64c46985867e346e5921c, State: Running, Role: LEADER
I20260923 22:12:41.438150 6851 sys_catalog.cc:564] T 00000000000000000000000000000000 P d80d75bdd4b64c46985867e346e5921c [sys.catalog]: configured and running, proceeding with master startup.
I20260923 22:12:41.438143 6856 consensus_queue.cc:237] T 00000000000000000000000000000000 P d80d75bdd4b64c46985867e346e5921c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d80d75bdd4b64c46985867e346e5921c" member_type: VOTER last_known_addr { host: "127.1.135.254" port: 41661 } }
I20260923 22:12:41.438459 6856 sys_catalog.cc:455] T 00000000000000000000000000000000 P d80d75bdd4b64c46985867e346e5921c [sys.catalog]: SysCatalogTable state changed. Reason: New leader d80d75bdd4b64c46985867e346e5921c. Latest consensus state: current_term: 1 leader_uuid: "d80d75bdd4b64c46985867e346e5921c" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d80d75bdd4b64c46985867e346e5921c" member_type: VOTER last_known_addr { host: "127.1.135.254" port: 41661 } } }
I20260923 22:12:41.438467 6857 sys_catalog.cc:455] T 00000000000000000000000000000000 P d80d75bdd4b64c46985867e346e5921c [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "d80d75bdd4b64c46985867e346e5921c" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d80d75bdd4b64c46985867e346e5921c" member_type: VOTER last_known_addr { host: "127.1.135.254" port: 41661 } } }
I20260923 22:12:41.438545 6856 sys_catalog.cc:458] T 00000000000000000000000000000000 P d80d75bdd4b64c46985867e346e5921c [sys.catalog]: This master's current role is: LEADER
I20260923 22:12:41.438570 6857 sys_catalog.cc:458] T 00000000000000000000000000000000 P d80d75bdd4b64c46985867e346e5921c [sys.catalog]: This master's current role is: LEADER
W20260923 22:12:41.439939 6872 catalog_manager.cc:1558] T 00000000000000000000000000000000 P d80d75bdd4b64c46985867e346e5921c: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260923 22:12:41.440940 6872 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260923 22:12:41.441016 6862 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260923 22:12:41.441186 6862 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260923 22:12:41.441708 6855 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 22:12:41.435985 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43678 (local address 127.1.135.254:41661)
0923 22:12:41.436233 (+ 248us) server_negotiation.cc:206] Beginning negotiation
0923 22:12:41.436241 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0923 22:12:41.436272 (+ 31us) server_negotiation.cc:407] Connection header received
0923 22:12:41.436603 (+ 331us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 22:12:41.436609 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 22:12:41.436662 (+ 53us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 22:12:41.436792 (+ 130us) server_negotiation.cc:226] Negotiated authn=SASL
0923 22:12:41.437000 (+ 208us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 22:12:41.438471 (+ 1471us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 22:12:41.439621 (+ 1150us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 22:12:41.440239 (+ 618us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 22:12:41.440373 (+ 134us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 22:12:41.440999 (+ 626us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0923 22:12:41.441003 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0923 22:12:41.441004 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0923 22:12:41.441201 (+ 197us) server_negotiation.cc:922] calling sasl_server_start()
0923 22:12:41.441243 (+ 42us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0923 22:12:41.441258 (+ 15us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0923 22:12:41.441323 (+ 65us) server_negotiation.cc:1035] Waiting for connection context
0923 22:12:41.441419 (+ 96us) server_negotiation.cc:299] Negotiation successful
0923 22:12:41.441491 (+ 72us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":156,"thread_start_us":95,"threads_started":1}
I20260923 22:12:41.442502 1567 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260923 22:12:41.443112 6862 catalog_manager.cc:1347] Generated new cluster ID: a0c6ba09fb3f4a8fa04e49411ffac58a
I20260923 22:12:41.443157 6862 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260923 22:12:41.454963 6862 catalog_manager.cc:1370] Generated new certificate authority record
I20260923 22:12:41.455435 6862 catalog_manager.cc:1504] Loading token signing keys...
I20260923 22:12:41.463740 6862 catalog_manager.cc:5868] T 00000000000000000000000000000000 P d80d75bdd4b64c46985867e346e5921c: Generated new TSK 0
I20260923 22:12:41.463900 6862 catalog_manager.cc:1514] Initializing in-progress tserver states...
E20260923 22:12:41.484449 6881 webserver.cc:533] Webserver: sslize failed: error:14094418:SSL routines:ssl3_read_bytes:tlsv1 alert unknown ca
E20260923 22:12:41.484519 6881 webserver.cc:533] Webserver: error shutting down SSL connection: error:140E0197:SSL routines:SSL_shutdown:shutdown while in init
I20260923 22:12:41.484833 6855 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 22:12:41.470779 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43682 (local address 127.1.135.254:41661)
0923 22:12:41.470835 (+ 56us) server_negotiation.cc:206] Beginning negotiation
0923 22:12:41.470837 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0923 22:12:41.470921 (+ 84us) server_negotiation.cc:407] Connection header received
0923 22:12:41.470972 (+ 51us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 22:12:41.470973 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 22:12:41.470985 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 22:12:41.471056 (+ 71us) server_negotiation.cc:226] Negotiated authn=JWT
0923 22:12:41.471153 (+ 97us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 22:12:41.471886 (+ 733us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 22:12:41.472679 (+ 793us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 22:12:41.472982 (+ 303us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 22:12:41.473046 (+ 64us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 22:12:41.473197 (+ 151us) server_negotiation.cc:365] Received JWT_EXCHANGE NegotiatePB request
0923 22:12:41.484638 (+ 11441us) server_negotiation.cc:394] Sending RPC error: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'https://localhost:46007/jwks.json': curl error: SSL peer certificate or SSH remote key was not OK: SSL certificate problem: unable to get local issuer certificate
0923 22:12:41.484769 (+ 131us) negotiation.cc:327] Negotiation complete: Not authorized: Server connection negotiation failed: server connection from 127.0.0.1:43682: Failed to load JWKS: Error downloading JWKS from 'https://localhost:46007/jwks.json': curl error: SSL peer certificate or SSH remote key was not OK: SSL certificate problem: unable to get local issuer certificate
Metrics: {"server-negotiator.queue_time_us":24}
W20260923 22:12:41.484889 6855 negotiation.cc:344] Unauthorized connection attempt: Server connection negotiation failed: server connection from 127.0.0.1:43682: Failed to load JWKS: Error downloading JWKS from 'https://localhost:46007/jwks.json': curl error: SSL peer certificate or SSH remote key was not OK: SSL certificate problem: unable to get local issuer certificate
W20260923 22:12:41.484903 6880 negotiation.cc:337] Failed RPC negotiation. Trace:
0923 22:12:41.470695 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.135.254:41661 (local address 127.0.0.1:43682)
0923 22:12:41.470859 (+ 164us) negotiation.cc:107] Waiting for socket to connect
0923 22:12:41.470867 (+ 8us) client_negotiation.cc:174] Beginning negotiation
0923 22:12:41.470915 (+ 48us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0923 22:12:41.471053 (+ 138us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0923 22:12:41.471053 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0923 22:12:41.471055 (+ 2us) client_negotiation.cc:189] Negotiated authn=JWT
0923 22:12:41.471100 (+ 45us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 22:12:41.471101 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 22:12:41.471959 (+ 858us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 22:12:41.471960 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 22:12:41.472594 (+ 634us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 22:12:41.472596 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 22:12:41.473061 (+ 465us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 22:12:41.473061 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 22:12:41.473100 (+ 39us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 22:12:41.473104 (+ 4us) client_negotiation.cc:252] Sending JWT_EXCHANGE NegotiatePB request
0923 22:12:41.484733 (+ 11629us) client_negotiation.cc:283] Received error response from server: Runtime error: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'https://localhost:46007/jwks.json': curl error: SSL peer certificate or SSH remote key was not OK: SSL certificate problem: unable to get local issuer certificate
0923 22:12:41.484819 (+ 86us) negotiation.cc:327] Negotiation complete: Runtime error: Client connection negotiation failed: client connection to 127.1.135.254:41661: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'https://localhost:46007/jwks.json': curl error: SSL peer certificate or SSH remote key was not OK: SSL certificate problem: unable to get local issuer certificate
Metrics: {"client-negotiator.queue_time_us":85,"thread_start_us":66,"threads_started":1}
I20260923 22:12:41.485174 1567 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskRiQe_a/build/release/bin/kudu with pid 6784
2026-09-23T22:12:41Z chronyd exiting
[ OK ] SecurityITest.TestJwtMiniClusterWithInvalidCert (208 ms)
[ RUN ] SecurityITest.TestJwtMiniClusterWithUntrustedCert
I20260923 22:12:41.522192 1567 mini_oidc.cc:152] Starting JWKS server
I20260923 22:12:41.522826 1567 webserver.cc:460] Webserver started at https://127.0.0.1:35483/ using document root <none> and password file <none>
I20260923 22:12:41.522866 1567 mini_oidc.cc:189] Starting OIDC Discovery server
I20260923 22:12:41.523028 1567 webserver.cc:460] Webserver started at http://127.0.0.1:35789/ using document root <none> and password file <none>
2026-09-23T22:12:41Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-09-23T22:12:41Z Disabled control of system clock
I20260923 22:12:41.530174 1567 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskRiQe_a/build/release/bin/kudu
/tmp/dist-test-taskRiQe_a/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskRiQe_a/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1790201539513534-1567-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskRiQe_a/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1790201539513534-1567-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskRiQe_a/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1790201539513534-1567-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskRiQe_a/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1790201539513534-1567-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.1.135.254:38483
--webserver_interface=127.1.135.254
--webserver_port=0
--builtin_ntp_servers=127.1.135.212:44339
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.1.135.254:38483
--enable_jwt_token_auth=true
--jwks_url=https://localhost:35483/jwks.json
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {}
W20260923 22:12:41.607241 6887 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260923 22:12:41.607435 6887 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260923 22:12:41.607455 6887 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260923 22:12:41.608594 6887 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260923 22:12:41.608632 6887 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260923 22:12:41.608644 6887 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260923 22:12:41.608657 6887 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260923 22:12:41.608668 6887 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
W20260923 22:12:41.608680 6887 flags.cc:425] Enabled experimental flag: --enable_jwt_token_auth=true
W20260923 22:12:41.608691 6887 flags.cc:425] Enabled experimental flag: --jwks_url=https://localhost:35483/jwks.json
I20260923 22:12:41.609941 6887 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.1.135.212:44339
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskRiQe_a/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1790201539513534-1567-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskRiQe_a/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1790201539513534-1567-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.1.135.254:38483
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.1.135.254:38483
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--enable_jwt_token_auth=true
--jwks_url=https://localhost:35483/jwks.json
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskRiQe_a/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1790201539513534-1567-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.1.135.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.6887
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskRiQe_a/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1790201539513534-1567-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 23 Sep 2026 20:53:37 UTC on ccycloud.quasar-rkviyx.root.comops.site
build id 4668083
I20260923 22:12:41.610114 6887 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260923 22:12:41.610369 6887 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260923 22:12:41.614845 6893 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260923 22:12:41.614809 6892 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260923 22:12:41.614848 6895 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260923 22:12:41.615005 6887 server_base.cc:1034] running on GCE node
I20260923 22:12:41.615242 6887 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260923 22:12:41.615525 6887 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260923 22:12:41.616667 6887 hybrid_clock.cc:648] HybridClock initialized: now 1790201561616644 us; error 40 us; skew 500 ppm
I20260923 22:12:41.617828 6887 webserver.cc:460] Webserver started at http://127.1.135.254:34495/ using document root <none> and password file <none>
I20260923 22:12:41.618062 6887 fs_manager.cc:359] Metadata directory not provided
I20260923 22:12:41.618125 6887 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260923 22:12:41.618259 6887 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260923 22:12:41.619257 6887 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskRiQe_a/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1790201539513534-1567-0/minicluster-data/master-0/data/instance:
uuid: "b4c14f721f9c466188629bb73a3cdca7"
format_stamp: "Formatted at 2026-09-23 22:12:41 on dist-test-slave-s2fj"
server_key: "b6699474651d31bb28ef1e3eb5b30697"
server_key_iv: "4c48b6a3849b9d1d2c4d40b2d2b0bc6b"
server_key_version: "encryptionkey@0"
I20260923 22:12:41.619589 6887 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskRiQe_a/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1790201539513534-1567-0/minicluster-data/master-0/wal/instance:
uuid: "b4c14f721f9c466188629bb73a3cdca7"
format_stamp: "Formatted at 2026-09-23 22:12:41 on dist-test-slave-s2fj"
server_key: "b6699474651d31bb28ef1e3eb5b30697"
server_key_iv: "4c48b6a3849b9d1d2c4d40b2d2b0bc6b"
server_key_version: "encryptionkey@0"
I20260923 22:12:41.620842 6887 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260923 22:12:41.621493 6901 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260923 22:12:41.621649 6887 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260923 22:12:41.621748 6887 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskRiQe_a/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1790201539513534-1567-0/minicluster-data/master-0/data,/tmp/dist-test-taskRiQe_a/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1790201539513534-1567-0/minicluster-data/master-0/wal
uuid: "b4c14f721f9c466188629bb73a3cdca7"
format_stamp: "Formatted at 2026-09-23 22:12:41 on dist-test-slave-s2fj"
server_key: "b6699474651d31bb28ef1e3eb5b30697"
server_key_iv: "4c48b6a3849b9d1d2c4d40b2d2b0bc6b"
server_key_version: "encryptionkey@0"
I20260923 22:12:41.621819 6887 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskRiQe_a/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1790201539513534-1567-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskRiQe_a/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1790201539513534-1567-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskRiQe_a/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1790201539513534-1567-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260923 22:12:41.631616 6887 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260923 22:12:41.631886 6887 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260923 22:12:41.632026 6887 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260923 22:12:41.635677 6887 rpc_server.cc:307] RPC server started. Bound to: 127.1.135.254:38483
I20260923 22:12:41.635715 6953 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.1.135.254:38483 every 8 connection(s)
I20260923 22:12:41.636035 6887 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskRiQe_a/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1790201539513534-1567-0/minicluster-data/master-0/data/info.pb
I20260923 22:12:41.636701 6954 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260923 22:12:41.639149 6954 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P b4c14f721f9c466188629bb73a3cdca7: Bootstrap starting.
I20260923 22:12:41.639703 6954 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P b4c14f721f9c466188629bb73a3cdca7: Neither blocks nor log segments found. Creating new log.
I20260923 22:12:41.639973 6954 log.cc:824] T 00000000000000000000000000000000 P b4c14f721f9c466188629bb73a3cdca7: Log is configured to *not* fsync() on all Append() calls
I20260923 22:12:41.640628 6954 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P b4c14f721f9c466188629bb73a3cdca7: No bootstrap required, opened a new log
I20260923 22:12:41.642128 6954 raft_consensus.cc:348] T 00000000000000000000000000000000 P b4c14f721f9c466188629bb73a3cdca7 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b4c14f721f9c466188629bb73a3cdca7" member_type: VOTER last_known_addr { host: "127.1.135.254" port: 38483 } }
I20260923 22:12:41.642262 6954 raft_consensus.cc:374] T 00000000000000000000000000000000 P b4c14f721f9c466188629bb73a3cdca7 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260923 22:12:41.642309 6954 raft_consensus.cc:729] T 00000000000000000000000000000000 P b4c14f721f9c466188629bb73a3cdca7 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b4c14f721f9c466188629bb73a3cdca7, State: Initialized, Role: FOLLOWER
I20260923 22:12:41.642493 6954 consensus_queue.cc:260] T 00000000000000000000000000000000 P b4c14f721f9c466188629bb73a3cdca7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b4c14f721f9c466188629bb73a3cdca7" member_type: VOTER last_known_addr { host: "127.1.135.254" port: 38483 } }
I20260923 22:12:41.642566 6954 raft_consensus.cc:388] T 00000000000000000000000000000000 P b4c14f721f9c466188629bb73a3cdca7 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260923 22:12:41.642611 6954 raft_consensus.cc:482] T 00000000000000000000000000000000 P b4c14f721f9c466188629bb73a3cdca7 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260923 22:12:41.642674 6954 raft_consensus.cc:3037] T 00000000000000000000000000000000 P b4c14f721f9c466188629bb73a3cdca7 [term 0 FOLLOWER]: Advancing to term 1
I20260923 22:12:41.643232 6954 raft_consensus.cc:504] T 00000000000000000000000000000000 P b4c14f721f9c466188629bb73a3cdca7 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b4c14f721f9c466188629bb73a3cdca7" member_type: VOTER last_known_addr { host: "127.1.135.254" port: 38483 } }
I20260923 22:12:41.643339 6954 leader_election.cc:302] T 00000000000000000000000000000000 P b4c14f721f9c466188629bb73a3cdca7 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: b4c14f721f9c466188629bb73a3cdca7; no voters:
I20260923 22:12:41.643513 6954 leader_election.cc:288] T 00000000000000000000000000000000 P b4c14f721f9c466188629bb73a3cdca7 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260923 22:12:41.643532 6957 raft_consensus.cc:2781] T 00000000000000000000000000000000 P b4c14f721f9c466188629bb73a3cdca7 [term 1 FOLLOWER]: Leader election won for term 1
I20260923 22:12:41.643671 6957 raft_consensus.cc:686] T 00000000000000000000000000000000 P b4c14f721f9c466188629bb73a3cdca7 [term 1 LEADER]: Becoming Leader. State: Replica: b4c14f721f9c466188629bb73a3cdca7, State: Running, Role: LEADER
I20260923 22:12:41.643769 6954 sys_catalog.cc:564] T 00000000000000000000000000000000 P b4c14f721f9c466188629bb73a3cdca7 [sys.catalog]: configured and running, proceeding with master startup.
I20260923 22:12:41.643775 6957 consensus_queue.cc:237] T 00000000000000000000000000000000 P b4c14f721f9c466188629bb73a3cdca7 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b4c14f721f9c466188629bb73a3cdca7" member_type: VOTER last_known_addr { host: "127.1.135.254" port: 38483 } }
I20260923 22:12:41.644174 6959 sys_catalog.cc:455] T 00000000000000000000000000000000 P b4c14f721f9c466188629bb73a3cdca7 [sys.catalog]: SysCatalogTable state changed. Reason: New leader b4c14f721f9c466188629bb73a3cdca7. Latest consensus state: current_term: 1 leader_uuid: "b4c14f721f9c466188629bb73a3cdca7" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b4c14f721f9c466188629bb73a3cdca7" member_type: VOTER last_known_addr { host: "127.1.135.254" port: 38483 } } }
I20260923 22:12:41.644270 6959 sys_catalog.cc:458] T 00000000000000000000000000000000 P b4c14f721f9c466188629bb73a3cdca7 [sys.catalog]: This master's current role is: LEADER
I20260923 22:12:41.644321 6958 sys_catalog.cc:455] T 00000000000000000000000000000000 P b4c14f721f9c466188629bb73a3cdca7 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "b4c14f721f9c466188629bb73a3cdca7" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b4c14f721f9c466188629bb73a3cdca7" member_type: VOTER last_known_addr { host: "127.1.135.254" port: 38483 } } }
I20260923 22:12:41.644383 6958 sys_catalog.cc:458] T 00000000000000000000000000000000 P b4c14f721f9c466188629bb73a3cdca7 [sys.catalog]: This master's current role is: LEADER
I20260923 22:12:41.644358 1567 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskRiQe_a/build/release/bin/kudu as pid 6887
I20260923 22:12:41.644464 1567 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskRiQe_a/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1790201539513534-1567-0/minicluster-data/master-0/wal/instance
I20260923 22:12:41.644498 6963 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260923 22:12:41.644652 1567 external_mini_cluster.cc:1428] Setting key 9c43be5e4f371b9102c534149f992cbd
I20260923 22:12:41.644973 6963 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260923 22:12:41.646755 6963 catalog_manager.cc:1347] Generated new cluster ID: f650c5fea3d34ddd9476192e0e5a5ec6
I20260923 22:12:41.646806 6963 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260923 22:12:41.651229 6975 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 22:12:41.645342 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34610 (local address 127.1.135.254:38483)
0923 22:12:41.645521 (+ 179us) server_negotiation.cc:206] Beginning negotiation
0923 22:12:41.645530 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0923 22:12:41.645858 (+ 328us) server_negotiation.cc:407] Connection header received
0923 22:12:41.646056 (+ 198us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 22:12:41.646064 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 22:12:41.646132 (+ 68us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 22:12:41.646245 (+ 113us) server_negotiation.cc:226] Negotiated authn=SASL
0923 22:12:41.646619 (+ 374us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 22:12:41.647976 (+ 1357us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 22:12:41.649085 (+ 1109us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 22:12:41.649742 (+ 657us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 22:12:41.649867 (+ 125us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 22:12:41.650539 (+ 672us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0923 22:12:41.650543 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0923 22:12:41.650543 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0923 22:12:41.650740 (+ 197us) server_negotiation.cc:922] calling sasl_server_start()
0923 22:12:41.650779 (+ 39us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0923 22:12:41.650793 (+ 14us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0923 22:12:41.650864 (+ 71us) server_negotiation.cc:1035] Waiting for connection context
0923 22:12:41.650940 (+ 76us) server_negotiation.cc:299] Negotiation successful
0923 22:12:41.651020 (+ 80us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":104,"thread_start_us":58,"threads_started":1}
I20260923 22:12:41.652185 1567 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260923 22:12:41.656080 6963 catalog_manager.cc:1370] Generated new certificate authority record
I20260923 22:12:41.656627 6963 catalog_manager.cc:1504] Loading token signing keys...
I20260923 22:12:41.660691 6963 catalog_manager.cc:5868] T 00000000000000000000000000000000 P b4c14f721f9c466188629bb73a3cdca7: Generated new TSK 0
I20260923 22:12:41.660847 6963 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260923 22:12:41.672288 6975 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 22:12:41.668513 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34616 (local address 127.1.135.254:38483)
0923 22:12:41.668567 (+ 54us) server_negotiation.cc:206] Beginning negotiation
0923 22:12:41.668569 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0923 22:12:41.668682 (+ 113us) server_negotiation.cc:407] Connection header received
0923 22:12:41.668742 (+ 60us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 22:12:41.668743 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 22:12:41.668752 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 22:12:41.668807 (+ 55us) server_negotiation.cc:226] Negotiated authn=JWT
0923 22:12:41.668930 (+ 123us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 22:12:41.669720 (+ 790us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 22:12:41.670856 (+ 1136us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 22:12:41.671061 (+ 205us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 22:12:41.671110 (+ 49us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 22:12:41.671292 (+ 182us) server_negotiation.cc:365] Received JWT_EXCHANGE NegotiatePB request
0923 22:12:41.672025 (+ 733us) server_negotiation.cc:394] Sending RPC error: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'https://localhost:35483/jwks.json': curl error: Problem with the SSL CA cert (path? access rights?): error setting certificate verify locations:
CAfile: /etc/pki/tls/certs/ca-bundle.crt
CApath: none
0923 22:12:41.672212 (+ 187us) negotiation.cc:327] Negotiation complete: Not authorized: Server connection negotiation failed: server connection from 127.0.0.1:34616: Failed to load JWKS: Error downloading JWKS from 'https://localhost:35483/jwks.json': curl error: Problem with the SSL CA cert (path? access rights?): error setting certificate verify locations:
CAfile: /etc/pki/tls/certs/ca-bundle.crt
CApath: none
Metrics: {"server-negotiator.queue_time_us":33}
W20260923 22:12:41.672358 6983 negotiation.cc:337] Failed RPC negotiation. Trace:
0923 22:12:41.668466 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.135.254:38483 (local address 127.0.0.1:34616)
0923 22:12:41.668610 (+ 144us) negotiation.cc:107] Waiting for socket to connect
0923 22:12:41.668620 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0923 22:12:41.668680 (+ 60us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0923 22:12:41.668820 (+ 140us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0923 22:12:41.668821 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0923 22:12:41.668823 (+ 2us) client_negotiation.cc:189] Negotiated authn=JWT
0923 22:12:41.668869 (+ 46us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 22:12:41.668871 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 22:12:41.669807 (+ 936us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 22:12:41.669808 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 22:12:41.670767 (+ 959us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 22:12:41.670769 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 22:12:41.671132 (+ 363us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 22:12:41.671133 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 22:12:41.671190 (+ 57us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 22:12:41.671202 (+ 12us) client_negotiation.cc:252] Sending JWT_EXCHANGE NegotiatePB request
0923 22:12:41.672173 (+ 971us) client_negotiation.cc:283] Received error response from server: Runtime error: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'https://localhost:35483/jwks.json': curl error: Problem with the SSL CA cert (path? access rights?): error setting certificate verify locations:
CAfile: /etc/pki/tls/certs/ca-bundle.crt
CApath: none
0923 22:12:41.672255 (+ 82us) negotiation.cc:327] Negotiation complete: Runtime error: Client connection negotiation failed: client connection to 127.1.135.254:38483: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'https://localhost:35483/jwks.json': curl error: Problem with the SSL CA cert (path? access rights?): error setting certificate verify locations:
CAfile: /etc/pki/tls/certs/ca-bundle.crt
CApath: none
Metrics: {"client-negotiator.queue_time_us":122,"thread_start_us":103,"threads_started":1}
I20260923 22:12:41.672588 1567 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskRiQe_a/build/release/bin/kudu with pid 6887
W20260923 22:12:41.672344 6975 negotiation.cc:344] Unauthorized connection attempt: Server connection negotiation failed: server connection from 127.0.0.1:34616: Failed to load JWKS: Error downloading JWKS from 'https://localhost:35483/jwks.json': curl error: Problem with the SSL CA cert (path? access rights?): error setting certificate verify locations:
CAfile: /etc/pki/tls/certs/ca-bundle.crt
CApath: none
E20260923 22:12:41.683147 6984 webserver.cc:533] Webserver: sslize failed:
E20260923 22:12:41.683212 6984 webserver.cc:533] Webserver: error shutting down SSL connection: error:140E0197:SSL routines:SSL_shutdown:shutdown while in init
2026-09-23T22:12:41Z chronyd exiting
[ OK ] SecurityITest.TestJwtMiniClusterWithUntrustedCert (187 ms)
[ RUN ] SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS
2026-09-23T22:12:41Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-09-23T22:12:41Z Disabled control of system clock
I20260923 22:12:41.716804 1567 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskRiQe_a/build/release/bin/kudu
/tmp/dist-test-taskRiQe_a/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskRiQe_a/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1790201539513534-1567-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskRiQe_a/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1790201539513534-1567-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskRiQe_a/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1790201539513534-1567-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskRiQe_a/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1790201539513534-1567-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.1.135.254:41941
--webserver_interface=127.1.135.254
--webserver_port=0
--builtin_ntp_servers=127.1.135.212:34401
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.1.135.254:41941
--enable_jwt_token_auth=true
--jwks_url=default.url
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--trusted_certificate_file=/tmp/dist-test-taskRiQe_a/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1790201539513534-1567-0/testchainca.cert with env {}
W20260923 22:12:41.793823 6988 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260923 22:12:41.794008 6988 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260923 22:12:41.794029 6988 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260923 22:12:41.795063 6988 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260923 22:12:41.795094 6988 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260923 22:12:41.795106 6988 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260923 22:12:41.795118 6988 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260923 22:12:41.795130 6988 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
W20260923 22:12:41.795142 6988 flags.cc:425] Enabled experimental flag: --enable_jwt_token_auth=true
W20260923 22:12:41.795153 6988 flags.cc:425] Enabled experimental flag: --jwks_url=default.url
I20260923 22:12:41.796540 6988 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.1.135.212:34401
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskRiQe_a/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1790201539513534-1567-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskRiQe_a/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1790201539513534-1567-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.1.135.254:41941
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.1.135.254:41941
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--enable_jwt_token_auth=true
--jwks_url=default.url
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskRiQe_a/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1790201539513534-1567-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.1.135.254
--webserver_port=0
--trusted_certificate_file=/tmp/dist-test-taskRiQe_a/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1790201539513534-1567-0/testchainca.cert
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.6988
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskRiQe_a/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1790201539513534-1567-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 23 Sep 2026 20:53:37 UTC on ccycloud.quasar-rkviyx.root.comops.site
build id 4668083
I20260923 22:12:41.796727 6988 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260923 22:12:41.796873 6988 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260923 22:12:41.801589 6994 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260923 22:12:41.801735 6988 server_base.cc:1034] running on GCE node
W20260923 22:12:41.801672 6996 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260923 22:12:41.801620 6993 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260923 22:12:41.802060 6988 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260923 22:12:41.802407 6988 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260923 22:12:41.803598 6988 hybrid_clock.cc:648] HybridClock initialized: now 1790201561803562 us; error 55 us; skew 500 ppm
I20260923 22:12:41.804788 6988 webserver.cc:460] Webserver started at http://127.1.135.254:34701/ using document root <none> and password file <none>
I20260923 22:12:41.805019 6988 fs_manager.cc:359] Metadata directory not provided
I20260923 22:12:41.805083 6988 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260923 22:12:41.805224 6988 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260923 22:12:41.806200 6988 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskRiQe_a/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1790201539513534-1567-0/minicluster-data/master-0/data/instance:
uuid: "49f56d566f8249a0b42904768be77f6c"
format_stamp: "Formatted at 2026-09-23 22:12:41 on dist-test-slave-s2fj"
server_key: "c32feaab649f62d2b1357579aa8b0f99"
server_key_iv: "8eadafafd8d24ecec6ca69467c684052"
server_key_version: "encryptionkey@0"
I20260923 22:12:41.806542 6988 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskRiQe_a/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1790201539513534-1567-0/minicluster-data/master-0/wal/instance:
uuid: "49f56d566f8249a0b42904768be77f6c"
format_stamp: "Formatted at 2026-09-23 22:12:41 on dist-test-slave-s2fj"
server_key: "c32feaab649f62d2b1357579aa8b0f99"
server_key_iv: "8eadafafd8d24ecec6ca69467c684052"
server_key_version: "encryptionkey@0"
I20260923 22:12:41.807765 6988 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260923 22:12:41.808499 7002 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260923 22:12:41.808651 6988 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.000s
I20260923 22:12:41.808749 6988 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskRiQe_a/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1790201539513534-1567-0/minicluster-data/master-0/data,/tmp/dist-test-taskRiQe_a/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1790201539513534-1567-0/minicluster-data/master-0/wal
uuid: "49f56d566f8249a0b42904768be77f6c"
format_stamp: "Formatted at 2026-09-23 22:12:41 on dist-test-slave-s2fj"
server_key: "c32feaab649f62d2b1357579aa8b0f99"
server_key_iv: "8eadafafd8d24ecec6ca69467c684052"
server_key_version: "encryptionkey@0"
I20260923 22:12:41.808822 6988 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskRiQe_a/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1790201539513534-1567-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskRiQe_a/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1790201539513534-1567-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskRiQe_a/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1790201539513534-1567-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260923 22:12:41.828696 6988 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260923 22:12:41.828979 6988 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260923 22:12:41.829134 6988 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260923 22:12:41.832839 6988 rpc_server.cc:307] RPC server started. Bound to: 127.1.135.254:41941
I20260923 22:12:41.832872 7054 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.1.135.254:41941 every 8 connection(s)
I20260923 22:12:41.833180 6988 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskRiQe_a/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1790201539513534-1567-0/minicluster-data/master-0/data/info.pb
I20260923 22:12:41.834059 7055 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260923 22:12:41.836223 7055 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 49f56d566f8249a0b42904768be77f6c: Bootstrap starting.
I20260923 22:12:41.836773 7055 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 49f56d566f8249a0b42904768be77f6c: Neither blocks nor log segments found. Creating new log.
I20260923 22:12:41.837046 7055 log.cc:824] T 00000000000000000000000000000000 P 49f56d566f8249a0b42904768be77f6c: Log is configured to *not* fsync() on all Append() calls
I20260923 22:12:41.837670 7055 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 49f56d566f8249a0b42904768be77f6c: No bootstrap required, opened a new log
I20260923 22:12:41.839178 7055 raft_consensus.cc:348] T 00000000000000000000000000000000 P 49f56d566f8249a0b42904768be77f6c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "49f56d566f8249a0b42904768be77f6c" member_type: VOTER last_known_addr { host: "127.1.135.254" port: 41941 } }
I20260923 22:12:41.839315 7055 raft_consensus.cc:374] T 00000000000000000000000000000000 P 49f56d566f8249a0b42904768be77f6c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260923 22:12:41.839357 7055 raft_consensus.cc:729] T 00000000000000000000000000000000 P 49f56d566f8249a0b42904768be77f6c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 49f56d566f8249a0b42904768be77f6c, State: Initialized, Role: FOLLOWER
I20260923 22:12:41.839504 7055 consensus_queue.cc:260] T 00000000000000000000000000000000 P 49f56d566f8249a0b42904768be77f6c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "49f56d566f8249a0b42904768be77f6c" member_type: VOTER last_known_addr { host: "127.1.135.254" port: 41941 } }
I20260923 22:12:41.839562 7055 raft_consensus.cc:388] T 00000000000000000000000000000000 P 49f56d566f8249a0b42904768be77f6c [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260923 22:12:41.839599 7055 raft_consensus.cc:482] T 00000000000000000000000000000000 P 49f56d566f8249a0b42904768be77f6c [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260923 22:12:41.839649 7055 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 49f56d566f8249a0b42904768be77f6c [term 0 FOLLOWER]: Advancing to term 1
I20260923 22:12:41.840159 7055 raft_consensus.cc:504] T 00000000000000000000000000000000 P 49f56d566f8249a0b42904768be77f6c [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "49f56d566f8249a0b42904768be77f6c" member_type: VOTER last_known_addr { host: "127.1.135.254" port: 41941 } }
I20260923 22:12:41.840281 7055 leader_election.cc:302] T 00000000000000000000000000000000 P 49f56d566f8249a0b42904768be77f6c [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 49f56d566f8249a0b42904768be77f6c; no voters:
I20260923 22:12:41.840440 7055 leader_election.cc:288] T 00000000000000000000000000000000 P 49f56d566f8249a0b42904768be77f6c [CANDIDATE]: Term 1 election: Requested vote from peers
I20260923 22:12:41.840466 7058 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 49f56d566f8249a0b42904768be77f6c [term 1 FOLLOWER]: Leader election won for term 1
I20260923 22:12:41.840610 7058 raft_consensus.cc:686] T 00000000000000000000000000000000 P 49f56d566f8249a0b42904768be77f6c [term 1 LEADER]: Becoming Leader. State: Replica: 49f56d566f8249a0b42904768be77f6c, State: Running, Role: LEADER
I20260923 22:12:41.840672 7055 sys_catalog.cc:564] T 00000000000000000000000000000000 P 49f56d566f8249a0b42904768be77f6c [sys.catalog]: configured and running, proceeding with master startup.
I20260923 22:12:41.840719 7058 consensus_queue.cc:237] T 00000000000000000000000000000000 P 49f56d566f8249a0b42904768be77f6c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "49f56d566f8249a0b42904768be77f6c" member_type: VOTER last_known_addr { host: "127.1.135.254" port: 41941 } }
I20260923 22:12:41.840962 1567 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskRiQe_a/build/release/bin/kudu as pid 6988
I20260923 22:12:41.841053 1567 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskRiQe_a/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1790201539513534-1567-0/minicluster-data/master-0/wal/instance
I20260923 22:12:41.841264 1567 external_mini_cluster.cc:1428] Setting key e905c0814eb548f89b1f5f5380a125b3
I20260923 22:12:41.841867 7060 sys_catalog.cc:455] T 00000000000000000000000000000000 P 49f56d566f8249a0b42904768be77f6c [sys.catalog]: SysCatalogTable state changed. Reason: New leader 49f56d566f8249a0b42904768be77f6c. Latest consensus state: current_term: 1 leader_uuid: "49f56d566f8249a0b42904768be77f6c" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "49f56d566f8249a0b42904768be77f6c" member_type: VOTER last_known_addr { host: "127.1.135.254" port: 41941 } } }
I20260923 22:12:41.841945 7060 sys_catalog.cc:458] T 00000000000000000000000000000000 P 49f56d566f8249a0b42904768be77f6c [sys.catalog]: This master's current role is: LEADER
I20260923 22:12:41.842130 7059 sys_catalog.cc:455] T 00000000000000000000000000000000 P 49f56d566f8249a0b42904768be77f6c [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "49f56d566f8249a0b42904768be77f6c" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "49f56d566f8249a0b42904768be77f6c" member_type: VOTER last_known_addr { host: "127.1.135.254" port: 41941 } } }
I20260923 22:12:41.842336 7059 sys_catalog.cc:458] T 00000000000000000000000000000000 P 49f56d566f8249a0b42904768be77f6c [sys.catalog]: This master's current role is: LEADER
W20260923 22:12:41.842715 7074 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 49f56d566f8249a0b42904768be77f6c: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260923 22:12:41.842768 7074 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260923 22:12:41.842819 7076 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260923 22:12:41.842993 7076 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260923 22:12:41.844336 7076 catalog_manager.cc:1347] Generated new cluster ID: 08f0ca8ead7f4cefba4496a38a1eca1b
I20260923 22:12:41.844388 7076 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260923 22:12:41.847604 7073 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 22:12:41.841871 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41284 (local address 127.1.135.254:41941)
0923 22:12:41.842047 (+ 176us) server_negotiation.cc:206] Beginning negotiation
0923 22:12:41.842056 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0923 22:12:41.842315 (+ 259us) server_negotiation.cc:407] Connection header received
0923 22:12:41.842540 (+ 225us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 22:12:41.842548 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 22:12:41.842617 (+ 69us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 22:12:41.842740 (+ 123us) server_negotiation.cc:226] Negotiated authn=SASL
0923 22:12:41.843081 (+ 341us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 22:12:41.844470 (+ 1389us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 22:12:41.845540 (+ 1070us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 22:12:41.846149 (+ 609us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 22:12:41.846290 (+ 141us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 22:12:41.846922 (+ 632us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0923 22:12:41.846926 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0923 22:12:41.846926 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0923 22:12:41.847129 (+ 203us) server_negotiation.cc:922] calling sasl_server_start()
0923 22:12:41.847173 (+ 44us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0923 22:12:41.847189 (+ 16us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0923 22:12:41.847259 (+ 70us) server_negotiation.cc:1035] Waiting for connection context
0923 22:12:41.847327 (+ 68us) server_negotiation.cc:299] Negotiation successful
0923 22:12:41.847401 (+ 74us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":104,"thread_start_us":54,"threads_started":1}
I20260923 22:12:41.848519 1567 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260923 22:12:41.864648 7076 catalog_manager.cc:1370] Generated new certificate authority record
I20260923 22:12:41.865306 7076 catalog_manager.cc:1504] Loading token signing keys...
I20260923 22:12:41.873489 7076 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 49f56d566f8249a0b42904768be77f6c: Generated new TSK 0
I20260923 22:12:41.873672 7076 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260923 22:12:41.959229 7073 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 22:12:41.886726 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41296 (local address 127.1.135.254:41941)
0923 22:12:41.886780 (+ 54us) server_negotiation.cc:206] Beginning negotiation
0923 22:12:41.886782 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0923 22:12:41.886887 (+ 105us) server_negotiation.cc:407] Connection header received
0923 22:12:41.886962 (+ 75us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 22:12:41.886963 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 22:12:41.886972 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 22:12:41.887024 (+ 52us) server_negotiation.cc:226] Negotiated authn=JWT
0923 22:12:41.887141 (+ 117us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 22:12:41.887599 (+ 458us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 22:12:41.888782 (+ 1183us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 22:12:41.888960 (+ 178us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 22:12:41.889035 (+ 75us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 22:12:41.889172 (+ 137us) server_negotiation.cc:365] Received JWT_EXCHANGE NegotiatePB request
0923 22:12:41.959017 (+ 69845us) server_negotiation.cc:394] Sending RPC error: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'default.url': curl error: Couldn't resolve host name: Could not resolve host: default.url
0923 22:12:41.959169 (+ 152us) negotiation.cc:327] Negotiation complete: Not authorized: Server connection negotiation failed: server connection from 127.0.0.1:41296: Failed to load JWKS: Error downloading JWKS from 'default.url': curl error: Couldn't resolve host name: Could not resolve host: default.url
Metrics: {"server-negotiator.queue_time_us":32}
W20260923 22:12:41.959313 7073 negotiation.cc:344] Unauthorized connection attempt: Server connection negotiation failed: server connection from 127.0.0.1:41296: Failed to load JWKS: Error downloading JWKS from 'default.url': curl error: Couldn't resolve host name: Could not resolve host: default.url
W20260923 22:12:41.959507 7085 negotiation.cc:337] Failed RPC negotiation. Trace:
0923 22:12:41.886670 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.135.254:41941 (local address 127.0.0.1:41296)
0923 22:12:41.886819 (+ 149us) negotiation.cc:107] Waiting for socket to connect
0923 22:12:41.886830 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0923 22:12:41.886897 (+ 67us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0923 22:12:41.887033 (+ 136us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0923 22:12:41.887033 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0923 22:12:41.887035 (+ 2us) client_negotiation.cc:189] Negotiated authn=JWT
0923 22:12:41.887090 (+ 55us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 22:12:41.887092 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 22:12:41.887690 (+ 598us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 22:12:41.887691 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 22:12:41.888668 (+ 977us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 22:12:41.888671 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 22:12:41.889027 (+ 356us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 22:12:41.889028 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 22:12:41.889087 (+ 59us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 22:12:41.889093 (+ 6us) client_negotiation.cc:252] Sending JWT_EXCHANGE NegotiatePB request
0923 22:12:41.959157 (+ 70064us) client_negotiation.cc:283] Received error response from server: Runtime error: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'default.url': curl error: Couldn't resolve host name: Could not resolve host: default.url
0923 22:12:41.959303 (+ 146us) negotiation.cc:327] Negotiation complete: Runtime error: Client connection negotiation failed: client connection to 127.1.135.254:41941: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'default.url': curl error: Couldn't resolve host name: Could not resolve host: default.url
Metrics: {"client-negotiator.queue_time_us":126,"thread_start_us":100,"threads_started":1}
I20260923 22:12:41.959805 1567 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskRiQe_a/build/release/bin/kudu with pid 6988
2026-09-23T22:12:41Z chronyd exiting
[ OK ] SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS (267 ms)
[ RUN ] SecurityITest.TestWorldReadableKeytab
[ OK ] SecurityITest.TestWorldReadableKeytab (83 ms)
[ RUN ] SecurityITest.TestWorldReadablePrivateKey
[ OK ] SecurityITest.TestWorldReadablePrivateKey (84 ms)
[ RUN ] SecurityITest.TestCorruptKerberosCC
Loading random data
Initializing database '/tmp/dist-test-taskRiQe_a/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790201539513534-1567-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Sep 23 22:12:43 dist-test-slave-s2fj krb5kdc[7097](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Sep 23 22:12:43 dist-test-slave-s2fj krb5kdc[7097](info): set up 2 sockets
Sep 23 22:12:43 dist-test-slave-s2fj krb5kdc[7097](info): commencing operation
krb5kdc: starting...
W20260923 22:12:44.260360 1567 mini_kdc.cc:121] Time spent starting KDC: real 2.115s user 0.001s sys 0.005s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
W20260923 22:12:44.498558 1567 mini_kdc.cc:253] Time spent creating user principal test-admin: real 0.238s user 0.001s sys 0.001s
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
W20260923 22:12:44.613864 1567 mini_kdc.cc:253] Time spent creating user principal test-user: real 0.115s user 0.000s sys 0.001s
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
W20260923 22:12:44.732160 1567 mini_kdc.cc:253] Time spent creating user principal joe-interloper: real 0.118s user 0.001s sys 0.001s
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskRiQe_a/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790201539513534-1567-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskRiQe_a/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790201539513534-1567-0/krb5kdc/test-user.keytab.
W20260923 22:12:45.656502 1567 mini_kdc.cc:302] Time spent creating keytab for test-user: real 0.924s user 0.001s sys 0.001s
Sep 23 22:12:45 dist-test-slave-s2fj krb5kdc[7097](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790201565, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-09-23T22:12:45Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-09-23T22:12:45Z Disabled control of system clock
WARNING: no policy specified for kudu/127.1.135.254@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.1.135.254@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.1.135.254 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskRiQe_a/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790201539513534-1567-0/krb5kdc/kudu_127.1.135.254.keytab.
Entry for principal kudu/127.1.135.254 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskRiQe_a/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790201539513534-1567-0/krb5kdc/kudu_127.1.135.254.keytab.
W20260923 22:12:46.607771 1567 mini_kdc.cc:262] Time spent creating service keytab for kudu/127.1.135.254: real 0.931s user 0.001s sys 0.002s
I20260923 22:12:46.608189 1567 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskRiQe_a/build/release/bin/kudu
/tmp/dist-test-taskRiQe_a/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskRiQe_a/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790201539513534-1567-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskRiQe_a/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790201539513534-1567-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskRiQe_a/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790201539513534-1567-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskRiQe_a/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790201539513534-1567-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.1.135.254:45605
--webserver_interface=127.1.135.254
--webserver_port=0
--keytab_file=/tmp/dist-test-taskRiQe_a/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790201539513534-1567-0/krb5kdc/kudu_127.1.135.254.keytab
--principal=kudu/127.1.135.254
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.1.135.212:46511
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.1.135.254:45605
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskRiQe_a/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790201539513534-1567-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskRiQe_a/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790201539513534-1567-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskRiQe_a/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790201539513534-1567-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260923 22:12:46.689688 7110 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260923 22:12:46.689873 7110 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260923 22:12:46.689894 7110 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260923 22:12:46.691021 7110 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260923 22:12:46.691067 7110 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260923 22:12:46.691082 7110 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260923 22:12:46.691095 7110 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260923 22:12:46.691149 7110 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260923 22:12:46.692682 7110 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.1.135.212:46511
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskRiQe_a/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790201539513534-1567-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskRiQe_a/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790201539513534-1567-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.1.135.254:45605
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskRiQe_a/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790201539513534-1567-0/krb5kdc/kudu_127.1.135.254.keytab
--principal=kudu/127.1.135.254
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.1.135.254:45605
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskRiQe_a/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790201539513534-1567-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.1.135.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.7110
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskRiQe_a/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790201539513534-1567-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 23 Sep 2026 20:53:37 UTC on ccycloud.quasar-rkviyx.root.comops.site
build id 4668083
I20260923 22:12:46.692912 7110 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260923 22:12:46.693125 7110 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260923 22:12:46.697657 7118 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260923 22:12:46.697710 7115 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260923 22:12:46.697700 7116 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260923 22:12:46.697810 7110 server_base.cc:1034] running on GCE node
I20260923 22:12:46.698068 7110 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260923 22:12:46.698344 7110 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260923 22:12:46.699499 7110 hybrid_clock.cc:648] HybridClock initialized: now 1790201566699490 us; error 37 us; skew 500 ppm
Sep 23 22:12:46 dist-test-slave-s2fj krb5kdc[7097](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790201566, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.135.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260923 22:12:46.701860 7110 init.cc:377] Logged in from keytab as kudu/127.1.135.254@KRBTEST.COM (short username kudu)
I20260923 22:12:46.702268 7110 webserver.cc:460] Webserver started at http://127.1.135.254:40557/ using document root <none> and password file <none>
I20260923 22:12:46.702491 7110 fs_manager.cc:359] Metadata directory not provided
I20260923 22:12:46.702554 7110 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260923 22:12:46.702696 7110 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260923 22:12:46.703717 7110 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskRiQe_a/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790201539513534-1567-0/minicluster-data/master-0/data/instance:
uuid: "f9ef37d0677b4225947dc96ee13399ed"
format_stamp: "Formatted at 2026-09-23 22:12:46 on dist-test-slave-s2fj"
server_key: "ca62fa6eef36b4e17e464edfd65e7e16"
server_key_iv: "b54017a9a874cc6ff5c48b96937ef024"
server_key_version: "encryptionkey@0"
I20260923 22:12:46.704056 7110 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskRiQe_a/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790201539513534-1567-0/minicluster-data/master-0/wal/instance:
uuid: "f9ef37d0677b4225947dc96ee13399ed"
format_stamp: "Formatted at 2026-09-23 22:12:46 on dist-test-slave-s2fj"
server_key: "ca62fa6eef36b4e17e464edfd65e7e16"
server_key_iv: "b54017a9a874cc6ff5c48b96937ef024"
server_key_version: "encryptionkey@0"
I20260923 22:12:46.705360 7110 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.001s sys 0.000s
I20260923 22:12:46.706049 7125 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260923 22:12:46.706202 7110 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260923 22:12:46.706293 7110 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskRiQe_a/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790201539513534-1567-0/minicluster-data/master-0/data,/tmp/dist-test-taskRiQe_a/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790201539513534-1567-0/minicluster-data/master-0/wal
uuid: "f9ef37d0677b4225947dc96ee13399ed"
format_stamp: "Formatted at 2026-09-23 22:12:46 on dist-test-slave-s2fj"
server_key: "ca62fa6eef36b4e17e464edfd65e7e16"
server_key_iv: "b54017a9a874cc6ff5c48b96937ef024"
server_key_version: "encryptionkey@0"
I20260923 22:12:46.706367 7110 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskRiQe_a/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790201539513534-1567-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskRiQe_a/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790201539513534-1567-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskRiQe_a/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790201539513534-1567-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260923 22:12:46.725602 7110 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260923 22:12:46.727447 7110 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260923 22:12:46.727600 7110 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260923 22:12:46.731400 7110 rpc_server.cc:307] RPC server started. Bound to: 127.1.135.254:45605
I20260923 22:12:46.731434 7177 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.1.135.254:45605 every 8 connection(s)
I20260923 22:12:46.731777 7110 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskRiQe_a/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790201539513534-1567-0/minicluster-data/master-0/data/info.pb
I20260923 22:12:46.732694 7178 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260923 22:12:46.732820 1567 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskRiQe_a/build/release/bin/kudu as pid 7110
I20260923 22:12:46.732921 1567 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskRiQe_a/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790201539513534-1567-0/minicluster-data/master-0/wal/instance
I20260923 22:12:46.733124 1567 external_mini_cluster.cc:1428] Setting key e048d044c51c9ecb546c64f5fc74543c
I20260923 22:12:46.735986 7178 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P f9ef37d0677b4225947dc96ee13399ed: Bootstrap starting.
I20260923 22:12:46.736760 7178 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P f9ef37d0677b4225947dc96ee13399ed: Neither blocks nor log segments found. Creating new log.
I20260923 22:12:46.737048 7178 log.cc:824] T 00000000000000000000000000000000 P f9ef37d0677b4225947dc96ee13399ed: Log is configured to *not* fsync() on all Append() calls
I20260923 22:12:46.737756 7178 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P f9ef37d0677b4225947dc96ee13399ed: No bootstrap required, opened a new log
Sep 23 22:12:46 dist-test-slave-s2fj krb5kdc[7097](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790201565, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.1.135.254@KRBTEST.COM
I20260923 22:12:46.740383 7178 raft_consensus.cc:348] T 00000000000000000000000000000000 P f9ef37d0677b4225947dc96ee13399ed [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f9ef37d0677b4225947dc96ee13399ed" member_type: VOTER last_known_addr { host: "127.1.135.254" port: 45605 } }
I20260923 22:12:46.740581 7178 raft_consensus.cc:374] T 00000000000000000000000000000000 P f9ef37d0677b4225947dc96ee13399ed [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260923 22:12:46.740628 7178 raft_consensus.cc:729] T 00000000000000000000000000000000 P f9ef37d0677b4225947dc96ee13399ed [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: f9ef37d0677b4225947dc96ee13399ed, State: Initialized, Role: FOLLOWER
I20260923 22:12:46.740794 7178 consensus_queue.cc:260] T 00000000000000000000000000000000 P f9ef37d0677b4225947dc96ee13399ed [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f9ef37d0677b4225947dc96ee13399ed" member_type: VOTER last_known_addr { host: "127.1.135.254" port: 45605 } }
I20260923 22:12:46.740882 7178 raft_consensus.cc:388] T 00000000000000000000000000000000 P f9ef37d0677b4225947dc96ee13399ed [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260923 22:12:46.740918 7178 raft_consensus.cc:482] T 00000000000000000000000000000000 P f9ef37d0677b4225947dc96ee13399ed [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260923 22:12:46.741008 7178 raft_consensus.cc:3037] T 00000000000000000000000000000000 P f9ef37d0677b4225947dc96ee13399ed [term 0 FOLLOWER]: Advancing to term 1
I20260923 22:12:46.741673 7178 raft_consensus.cc:504] T 00000000000000000000000000000000 P f9ef37d0677b4225947dc96ee13399ed [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f9ef37d0677b4225947dc96ee13399ed" member_type: VOTER last_known_addr { host: "127.1.135.254" port: 45605 } }
I20260923 22:12:46.741780 7178 leader_election.cc:302] T 00000000000000000000000000000000 P f9ef37d0677b4225947dc96ee13399ed [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: f9ef37d0677b4225947dc96ee13399ed; no voters:
I20260923 22:12:46.741950 7178 leader_election.cc:288] T 00000000000000000000000000000000 P f9ef37d0677b4225947dc96ee13399ed [CANDIDATE]: Term 1 election: Requested vote from peers
I20260923 22:12:46.741986 7183 raft_consensus.cc:2781] T 00000000000000000000000000000000 P f9ef37d0677b4225947dc96ee13399ed [term 1 FOLLOWER]: Leader election won for term 1
I20260923 22:12:46.742101 7183 raft_consensus.cc:686] T 00000000000000000000000000000000 P f9ef37d0677b4225947dc96ee13399ed [term 1 LEADER]: Becoming Leader. State: Replica: f9ef37d0677b4225947dc96ee13399ed, State: Running, Role: LEADER
I20260923 22:12:46.742223 7178 sys_catalog.cc:564] T 00000000000000000000000000000000 P f9ef37d0677b4225947dc96ee13399ed [sys.catalog]: configured and running, proceeding with master startup.
I20260923 22:12:46.742245 7183 consensus_queue.cc:237] T 00000000000000000000000000000000 P f9ef37d0677b4225947dc96ee13399ed [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f9ef37d0677b4225947dc96ee13399ed" member_type: VOTER last_known_addr { host: "127.1.135.254" port: 45605 } }
I20260923 22:12:46.742923 7184 sys_catalog.cc:455] T 00000000000000000000000000000000 P f9ef37d0677b4225947dc96ee13399ed [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "f9ef37d0677b4225947dc96ee13399ed" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f9ef37d0677b4225947dc96ee13399ed" member_type: VOTER last_known_addr { host: "127.1.135.254" port: 45605 } } }
I20260923 22:12:46.743002 7184 sys_catalog.cc:458] T 00000000000000000000000000000000 P f9ef37d0677b4225947dc96ee13399ed [sys.catalog]: This master's current role is: LEADER
I20260923 22:12:46.743028 7185 sys_catalog.cc:455] T 00000000000000000000000000000000 P f9ef37d0677b4225947dc96ee13399ed [sys.catalog]: SysCatalogTable state changed. Reason: New leader f9ef37d0677b4225947dc96ee13399ed. Latest consensus state: current_term: 1 leader_uuid: "f9ef37d0677b4225947dc96ee13399ed" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f9ef37d0677b4225947dc96ee13399ed" member_type: VOTER last_known_addr { host: "127.1.135.254" port: 45605 } } }
I20260923 22:12:46.743090 7185 sys_catalog.cc:458] T 00000000000000000000000000000000 P f9ef37d0677b4225947dc96ee13399ed [sys.catalog]: This master's current role is: LEADER
I20260923 22:12:46.743477 7193 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260923 22:12:46.744042 7193 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260923 22:12:46.744338 7181 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 22:12:46.733660 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55116 (local address 127.1.135.254:45605)
0923 22:12:46.733804 (+ 144us) server_negotiation.cc:206] Beginning negotiation
0923 22:12:46.733813 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0923 22:12:46.733845 (+ 32us) server_negotiation.cc:407] Connection header received
0923 22:12:46.734285 (+ 440us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 22:12:46.734290 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 22:12:46.734335 (+ 45us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 22:12:46.734442 (+ 107us) server_negotiation.cc:226] Negotiated authn=SASL
0923 22:12:46.735222 (+ 780us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 22:12:46.736203 (+ 981us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 22:12:46.737298 (+ 1095us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 22:12:46.737780 (+ 482us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 22:12:46.737903 (+ 123us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 22:12:46.740542 (+ 2639us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0923 22:12:46.740545 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0923 22:12:46.740546 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0923 22:12:46.740555 (+ 9us) server_negotiation.cc:922] calling sasl_server_start()
0923 22:12:46.742467 (+ 1912us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0923 22:12:46.742832 (+ 365us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0923 22:12:46.742835 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0923 22:12:46.742836 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0923 22:12:46.742947 (+ 111us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0923 22:12:46.743363 (+ 416us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0923 22:12:46.743365 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0923 22:12:46.743365 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0923 22:12:46.743712 (+ 347us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0923 22:12:46.743878 (+ 166us) server_negotiation.cc:1035] Waiting for connection context
0923 22:12:46.743971 (+ 93us) server_negotiation.cc:299] Negotiation successful
0923 22:12:46.744143 (+ 172us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":95,"thread_start_us":66,"threads_started":1}
I20260923 22:12:46.745664 7193 catalog_manager.cc:1347] Generated new cluster ID: 74d32619cfba4e0d8af5ebe2adf1c717
I20260923 22:12:46.745716 7193 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260923 22:12:46.766253 7193 catalog_manager.cc:1370] Generated new certificate authority record
I20260923 22:12:46.766799 7193 catalog_manager.cc:1504] Loading token signing keys...
I20260923 22:12:46.772480 7193 catalog_manager.cc:5868] T 00000000000000000000000000000000 P f9ef37d0677b4225947dc96ee13399ed: Generated new TSK 0
I20260923 22:12:46.772646 7193 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.1.135.193@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.1.135.193@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.1.135.193 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskRiQe_a/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790201539513534-1567-0/krb5kdc/kudu_127.1.135.193.keytab.
Entry for principal kudu/127.1.135.193 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskRiQe_a/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790201539513534-1567-0/krb5kdc/kudu_127.1.135.193.keytab.
I20260923 22:12:46.805245 1567 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskRiQe_a/build/release/bin/kudu
/tmp/dist-test-taskRiQe_a/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskRiQe_a/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790201539513534-1567-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskRiQe_a/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790201539513534-1567-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskRiQe_a/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790201539513534-1567-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskRiQe_a/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790201539513534-1567-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.1.135.193:0
--local_ip_for_outbound_sockets=127.1.135.193
--webserver_interface=127.1.135.193
--webserver_port=0
--tserver_master_addrs=127.1.135.254:45605
--keytab_file=/tmp/dist-test-taskRiQe_a/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790201539513534-1567-0/krb5kdc/kudu_127.1.135.193.keytab
--principal=kudu/127.1.135.193
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.1.135.212:46511
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskRiQe_a/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790201539513534-1567-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskRiQe_a/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790201539513534-1567-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskRiQe_a/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790201539513534-1567-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260923 22:12:46.884527 7205 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260923 22:12:46.884735 7205 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260923 22:12:46.884773 7205 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260923 22:12:46.885893 7205 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260923 22:12:46.885953 7205 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260923 22:12:46.885995 7205 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.1.135.193
I20260923 22:12:46.887421 7205 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.1.135.212:46511
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskRiQe_a/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790201539513534-1567-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskRiQe_a/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790201539513534-1567-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskRiQe_a/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790201539513534-1567-0/krb5kdc/kudu_127.1.135.193.keytab
--principal=kudu/127.1.135.193
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.1.135.193:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskRiQe_a/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790201539513534-1567-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.1.135.193
--webserver_port=0
--tserver_master_addrs=127.1.135.254:45605
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.7205
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.1.135.193
--log_dir=/tmp/dist-test-taskRiQe_a/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790201539513534-1567-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 23 Sep 2026 20:53:37 UTC on ccycloud.quasar-rkviyx.root.comops.site
build id 4668083
I20260923 22:12:46.887674 7205 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260923 22:12:46.887883 7205 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260923 22:12:46.892699 7211 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260923 22:12:46.892720 7210 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260923 22:12:46.892699 7213 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260923 22:12:46.892947 7205 server_base.cc:1034] running on GCE node
I20260923 22:12:46.893169 7205 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260923 22:12:46.893412 7205 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260923 22:12:46.894565 7205 hybrid_clock.cc:648] HybridClock initialized: now 1790201566894558 us; error 32 us; skew 500 ppm
Sep 23 22:12:46 dist-test-slave-s2fj krb5kdc[7097](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790201566, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.135.193@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260923 22:12:46.896889 7205 init.cc:377] Logged in from keytab as kudu/127.1.135.193@KRBTEST.COM (short username kudu)
I20260923 22:12:46.897347 7205 webserver.cc:460] Webserver started at http://127.1.135.193:37287/ using document root <none> and password file <none>
I20260923 22:12:46.897576 7205 fs_manager.cc:359] Metadata directory not provided
I20260923 22:12:46.897640 7205 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260923 22:12:46.897768 7205 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260923 22:12:46.898741 7205 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskRiQe_a/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790201539513534-1567-0/minicluster-data/ts-0/data/instance:
uuid: "36e16b4826664bd099d86731bad430e0"
format_stamp: "Formatted at 2026-09-23 22:12:46 on dist-test-slave-s2fj"
server_key: "fc20ee2d1d9cb559986af02ae5e8a267"
server_key_iv: "765349b2907ce4d9c1f6922d11928962"
server_key_version: "encryptionkey@0"
I20260923 22:12:46.899117 7205 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskRiQe_a/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790201539513534-1567-0/minicluster-data/ts-0/wal/instance:
uuid: "36e16b4826664bd099d86731bad430e0"
format_stamp: "Formatted at 2026-09-23 22:12:46 on dist-test-slave-s2fj"
server_key: "fc20ee2d1d9cb559986af02ae5e8a267"
server_key_iv: "765349b2907ce4d9c1f6922d11928962"
server_key_version: "encryptionkey@0"
I20260923 22:12:46.900435 7205 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.003s
I20260923 22:12:46.901186 7220 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260923 22:12:46.901362 7205 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.000s
I20260923 22:12:46.901453 7205 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskRiQe_a/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790201539513534-1567-0/minicluster-data/ts-0/data,/tmp/dist-test-taskRiQe_a/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790201539513534-1567-0/minicluster-data/ts-0/wal
uuid: "36e16b4826664bd099d86731bad430e0"
format_stamp: "Formatted at 2026-09-23 22:12:46 on dist-test-slave-s2fj"
server_key: "fc20ee2d1d9cb559986af02ae5e8a267"
server_key_iv: "765349b2907ce4d9c1f6922d11928962"
server_key_version: "encryptionkey@0"
I20260923 22:12:46.901528 7205 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskRiQe_a/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790201539513534-1567-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskRiQe_a/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790201539513534-1567-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskRiQe_a/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790201539513534-1567-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260923 22:12:46.916745 7205 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260923 22:12:46.918550 7205 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260923 22:12:46.918705 7205 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260923 22:12:46.918926 7205 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260923 22:12:46.919211 7205 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260923 22:12:46.919258 7205 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260923 22:12:46.919298 7205 ts_tablet_manager.cc:610] Registered 0 tablets
I20260923 22:12:46.919325 7205 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260923 22:12:46.925058 7205 rpc_server.cc:307] RPC server started. Bound to: 127.1.135.193:41945
I20260923 22:12:46.925102 7333 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.1.135.193:41945 every 8 connection(s)
I20260923 22:12:46.925421 7205 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskRiQe_a/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790201539513534-1567-0/minicluster-data/ts-0/data/info.pb
I20260923 22:12:46.929742 1567 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskRiQe_a/build/release/bin/kudu as pid 7205
I20260923 22:12:46.929869 1567 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskRiQe_a/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790201539513534-1567-0/minicluster-data/ts-0/wal/instance
I20260923 22:12:46.930084 1567 external_mini_cluster.cc:1428] Setting key d60ac40737b69f73b240da00cfc2884d
Sep 23 22:12:46 dist-test-slave-s2fj krb5kdc[7097](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790201566, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.135.193@KRBTEST.COM for kudu/127.1.135.254@KRBTEST.COM
I20260923 22:12:46.935096 7181 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 22:12:46.926120 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.135.193:46213 (local address 127.1.135.254:45605)
0923 22:12:46.926162 (+ 42us) server_negotiation.cc:206] Beginning negotiation
0923 22:12:46.926165 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0923 22:12:46.926360 (+ 195us) server_negotiation.cc:407] Connection header received
0923 22:12:46.927281 (+ 921us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 22:12:46.927282 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 22:12:46.927292 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 22:12:46.927348 (+ 56us) server_negotiation.cc:226] Negotiated authn=SASL
0923 22:12:46.927886 (+ 538us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 22:12:46.928477 (+ 591us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 22:12:46.929418 (+ 941us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 22:12:46.929624 (+ 206us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 22:12:46.929702 (+ 78us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 22:12:46.932357 (+ 2655us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0923 22:12:46.932359 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0923 22:12:46.932359 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0923 22:12:46.932366 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0923 22:12:46.933670 (+ 1304us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0923 22:12:46.934151 (+ 481us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0923 22:12:46.934152 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0923 22:12:46.934153 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0923 22:12:46.934224 (+ 71us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0923 22:12:46.934508 (+ 284us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0923 22:12:46.934509 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0923 22:12:46.934509 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0923 22:12:46.934607 (+ 98us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0923 22:12:46.934656 (+ 49us) server_negotiation.cc:1035] Waiting for connection context
0923 22:12:46.934965 (+ 309us) server_negotiation.cc:299] Negotiation successful
0923 22:12:46.935037 (+ 72us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":18}
I20260923 22:12:46.935423 7336 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 22:12:46.926092 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.135.254:45605 (local address 127.1.135.193:46213)
0923 22:12:46.926246 (+ 154us) negotiation.cc:107] Waiting for socket to connect
0923 22:12:46.926286 (+ 40us) client_negotiation.cc:174] Beginning negotiation
0923 22:12:46.927193 (+ 907us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0923 22:12:46.927400 (+ 207us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0923 22:12:46.927403 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0923 22:12:46.927637 (+ 234us) client_negotiation.cc:189] Negotiated authn=SASL
0923 22:12:46.927817 (+ 180us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 22:12:46.927821 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 22:12:46.928558 (+ 737us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 22:12:46.928559 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 22:12:46.929349 (+ 790us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 22:12:46.929351 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 22:12:46.929715 (+ 364us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 22:12:46.929716 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 22:12:46.929856 (+ 140us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 22:12:46.930395 (+ 539us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0923 22:12:46.930404 (+ 9us) client_negotiation.cc:656] calling sasl_client_start()
0923 22:12:46.932257 (+ 1853us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0923 22:12:46.933750 (+ 1493us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0923 22:12:46.933756 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0923 22:12:46.933758 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0923 22:12:46.934073 (+ 315us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0923 22:12:46.934298 (+ 225us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0923 22:12:46.934299 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0923 22:12:46.934300 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0923 22:12:46.934443 (+ 143us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0923 22:12:46.934668 (+ 225us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0923 22:12:46.934674 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0923 22:12:46.934831 (+ 157us) client_negotiation.cc:769] Sending connection context
0923 22:12:46.934947 (+ 116us) client_negotiation.cc:240] Negotiation successful
0923 22:12:46.935121 (+ 174us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":114,"thread_start_us":88,"threads_started":1}
I20260923 22:12:46.935969 7334 heartbeater.cc:347] Connected to a master server at 127.1.135.254:45605
I20260923 22:12:46.936094 7334 heartbeater.cc:464] Registering TS with master...
I20260923 22:12:46.936275 7334 heartbeater.cc:511] Master 127.1.135.254:45605 requested a full tablet report, sending...
I20260923 22:12:46.936750 7142 ts_manager.cc:194] Registered new tserver with Master: 36e16b4826664bd099d86731bad430e0 (127.1.135.193:41945)
I20260923 22:12:46.937327 7142 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.1.135.193@KRBTEST.COM'} at 127.1.135.193:46213
WARNING: no policy specified for kudu/127.1.135.194@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.1.135.194@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.1.135.194 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskRiQe_a/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790201539513534-1567-0/krb5kdc/kudu_127.1.135.194.keytab.
Entry for principal kudu/127.1.135.194 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskRiQe_a/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790201539513534-1567-0/krb5kdc/kudu_127.1.135.194.keytab.
I20260923 22:12:46.954727 1567 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskRiQe_a/build/release/bin/kudu
/tmp/dist-test-taskRiQe_a/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskRiQe_a/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790201539513534-1567-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskRiQe_a/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790201539513534-1567-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskRiQe_a/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790201539513534-1567-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskRiQe_a/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790201539513534-1567-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.1.135.194:0
--local_ip_for_outbound_sockets=127.1.135.194
--webserver_interface=127.1.135.194
--webserver_port=0
--tserver_master_addrs=127.1.135.254:45605
--keytab_file=/tmp/dist-test-taskRiQe_a/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790201539513534-1567-0/krb5kdc/kudu_127.1.135.194.keytab
--principal=kudu/127.1.135.194
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.1.135.212:46511
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskRiQe_a/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790201539513534-1567-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskRiQe_a/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790201539513534-1567-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskRiQe_a/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790201539513534-1567-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260923 22:12:47.033187 7339 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260923 22:12:47.033407 7339 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260923 22:12:47.033457 7339 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260923 22:12:47.034581 7339 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260923 22:12:47.034648 7339 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260923 22:12:47.034689 7339 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.1.135.194
I20260923 22:12:47.036129 7339 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.1.135.212:46511
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskRiQe_a/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790201539513534-1567-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskRiQe_a/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790201539513534-1567-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskRiQe_a/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790201539513534-1567-0/krb5kdc/kudu_127.1.135.194.keytab
--principal=kudu/127.1.135.194
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.1.135.194:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskRiQe_a/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790201539513534-1567-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.1.135.194
--webserver_port=0
--tserver_master_addrs=127.1.135.254:45605
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.7339
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.1.135.194
--log_dir=/tmp/dist-test-taskRiQe_a/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790201539513534-1567-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 23 Sep 2026 20:53:37 UTC on ccycloud.quasar-rkviyx.root.comops.site
build id 4668083
I20260923 22:12:47.036372 7339 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260923 22:12:47.036586 7339 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260923 22:12:47.041251 7345 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260923 22:12:47.041349 7344 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260923 22:12:47.041370 7347 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260923 22:12:47.041425 7339 server_base.cc:1034] running on GCE node
I20260923 22:12:47.041675 7339 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260923 22:12:47.041863 7339 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260923 22:12:47.043023 7339 hybrid_clock.cc:648] HybridClock initialized: now 1790201567043024 us; error 46 us; skew 500 ppm
Sep 23 22:12:47 dist-test-slave-s2fj krb5kdc[7097](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790201567, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.135.194@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260923 22:12:47.045521 7339 init.cc:377] Logged in from keytab as kudu/127.1.135.194@KRBTEST.COM (short username kudu)
I20260923 22:12:47.045984 7339 webserver.cc:460] Webserver started at http://127.1.135.194:44969/ using document root <none> and password file <none>
I20260923 22:12:47.046224 7339 fs_manager.cc:359] Metadata directory not provided
I20260923 22:12:47.046288 7339 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260923 22:12:47.046416 7339 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260923 22:12:47.047394 7339 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskRiQe_a/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790201539513534-1567-0/minicluster-data/ts-1/data/instance:
uuid: "e049c8117ddd456e87547059e6fb175f"
format_stamp: "Formatted at 2026-09-23 22:12:47 on dist-test-slave-s2fj"
server_key: "66888b09f7ebdd36d8d6c6ef22a367d2"
server_key_iv: "9a326456a3703ffd03a14583f240a020"
server_key_version: "encryptionkey@0"
I20260923 22:12:47.047744 7339 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskRiQe_a/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790201539513534-1567-0/minicluster-data/ts-1/wal/instance:
uuid: "e049c8117ddd456e87547059e6fb175f"
format_stamp: "Formatted at 2026-09-23 22:12:47 on dist-test-slave-s2fj"
server_key: "66888b09f7ebdd36d8d6c6ef22a367d2"
server_key_iv: "9a326456a3703ffd03a14583f240a020"
server_key_version: "encryptionkey@0"
I20260923 22:12:47.049015 7339 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.003s
I20260923 22:12:47.049661 7354 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260923 22:12:47.049813 7339 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.000s
I20260923 22:12:47.049901 7339 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskRiQe_a/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790201539513534-1567-0/minicluster-data/ts-1/data,/tmp/dist-test-taskRiQe_a/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790201539513534-1567-0/minicluster-data/ts-1/wal
uuid: "e049c8117ddd456e87547059e6fb175f"
format_stamp: "Formatted at 2026-09-23 22:12:47 on dist-test-slave-s2fj"
server_key: "66888b09f7ebdd36d8d6c6ef22a367d2"
server_key_iv: "9a326456a3703ffd03a14583f240a020"
server_key_version: "encryptionkey@0"
I20260923 22:12:47.049971 7339 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskRiQe_a/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790201539513534-1567-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskRiQe_a/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790201539513534-1567-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskRiQe_a/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790201539513534-1567-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260923 22:12:47.069873 7339 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260923 22:12:47.071535 7339 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260923 22:12:47.071704 7339 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260923 22:12:47.071913 7339 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260923 22:12:47.072206 7339 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260923 22:12:47.072255 7339 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260923 22:12:47.072289 7339 ts_tablet_manager.cc:610] Registered 0 tablets
I20260923 22:12:47.072315 7339 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260923 22:12:47.077816 7339 rpc_server.cc:307] RPC server started. Bound to: 127.1.135.194:33437
I20260923 22:12:47.077862 7467 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.1.135.194:33437 every 8 connection(s)
I20260923 22:12:47.078150 7339 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskRiQe_a/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790201539513534-1567-0/minicluster-data/ts-1/data/info.pb
I20260923 22:12:47.078943 1567 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskRiQe_a/build/release/bin/kudu as pid 7339
I20260923 22:12:47.079051 1567 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskRiQe_a/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790201539513534-1567-0/minicluster-data/ts-1/wal/instance
I20260923 22:12:47.079243 1567 external_mini_cluster.cc:1428] Setting key 4ca2a123ddc1f71cf2fcecc508894df8
Sep 23 22:12:47 dist-test-slave-s2fj krb5kdc[7097](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790201567, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.135.194@KRBTEST.COM for kudu/127.1.135.254@KRBTEST.COM
WARNING: no policy specified for kudu/127.1.135.195@KRBTEST.COM; defaulting to no policy
I20260923 22:12:47.088141 7181 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 22:12:47.078819 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.135.194:41189 (local address 127.1.135.254:45605)
0923 22:12:47.078897 (+ 78us) server_negotiation.cc:206] Beginning negotiation
0923 22:12:47.078900 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0923 22:12:47.079044 (+ 144us) server_negotiation.cc:407] Connection header received
0923 22:12:47.080294 (+ 1250us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 22:12:47.080295 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 22:12:47.080307 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 22:12:47.080376 (+ 69us) server_negotiation.cc:226] Negotiated authn=SASL
0923 22:12:47.080945 (+ 569us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 22:12:47.081596 (+ 651us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 22:12:47.082556 (+ 960us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 22:12:47.082763 (+ 207us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 22:12:47.082839 (+ 76us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 22:12:47.085259 (+ 2420us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0923 22:12:47.085260 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0923 22:12:47.085261 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0923 22:12:47.085267 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0923 22:12:47.086615 (+ 1348us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0923 22:12:47.087074 (+ 459us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0923 22:12:47.087075 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0923 22:12:47.087075 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0923 22:12:47.087130 (+ 55us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0923 22:12:47.087404 (+ 274us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0923 22:12:47.087405 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0923 22:12:47.087405 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0923 22:12:47.087504 (+ 99us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0923 22:12:47.087552 (+ 48us) server_negotiation.cc:1035] Waiting for connection context
0923 22:12:47.087916 (+ 364us) server_negotiation.cc:299] Negotiation successful
0923 22:12:47.088002 (+ 86us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":42}
I20260923 22:12:47.088250 7470 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 22:12:47.078766 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.135.254:45605 (local address 127.1.135.194:41189)
0923 22:12:47.078914 (+ 148us) negotiation.cc:107] Waiting for socket to connect
0923 22:12:47.078953 (+ 39us) client_negotiation.cc:174] Beginning negotiation
0923 22:12:47.080184 (+ 1231us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0923 22:12:47.080437 (+ 253us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0923 22:12:47.080440 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0923 22:12:47.080689 (+ 249us) client_negotiation.cc:189] Negotiated authn=SASL
0923 22:12:47.080861 (+ 172us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 22:12:47.080865 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 22:12:47.081693 (+ 828us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 22:12:47.081694 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 22:12:47.082466 (+ 772us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 22:12:47.082468 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 22:12:47.082845 (+ 377us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 22:12:47.082846 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 22:12:47.082971 (+ 125us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 22:12:47.083456 (+ 485us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0923 22:12:47.083464 (+ 8us) client_negotiation.cc:656] calling sasl_client_start()
0923 22:12:47.085169 (+ 1705us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0923 22:12:47.086698 (+ 1529us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0923 22:12:47.086704 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0923 22:12:47.086706 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0923 22:12:47.087005 (+ 299us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0923 22:12:47.087191 (+ 186us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0923 22:12:47.087192 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0923 22:12:47.087193 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0923 22:12:47.087336 (+ 143us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0923 22:12:47.087566 (+ 230us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0923 22:12:47.087573 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0923 22:12:47.087772 (+ 199us) client_negotiation.cc:769] Sending connection context
0923 22:12:47.087871 (+ 99us) client_negotiation.cc:240] Negotiation successful
0923 22:12:47.087983 (+ 112us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":110,"thread_start_us":85,"threads_started":1}
I20260923 22:12:47.088671 7468 heartbeater.cc:347] Connected to a master server at 127.1.135.254:45605
I20260923 22:12:47.088748 7468 heartbeater.cc:464] Registering TS with master...
I20260923 22:12:47.088872 7468 heartbeater.cc:511] Master 127.1.135.254:45605 requested a full tablet report, sending...
I20260923 22:12:47.089160 7142 ts_manager.cc:194] Registered new tserver with Master: e049c8117ddd456e87547059e6fb175f (127.1.135.194:33437)
I20260923 22:12:47.089565 7142 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.1.135.194@KRBTEST.COM'} at 127.1.135.194:41189
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.1.135.195@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.1.135.195 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskRiQe_a/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790201539513534-1567-0/krb5kdc/kudu_127.1.135.195.keytab.
Entry for principal kudu/127.1.135.195 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskRiQe_a/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790201539513534-1567-0/krb5kdc/kudu_127.1.135.195.keytab.
I20260923 22:12:47.102843 1567 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskRiQe_a/build/release/bin/kudu
/tmp/dist-test-taskRiQe_a/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskRiQe_a/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790201539513534-1567-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskRiQe_a/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790201539513534-1567-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskRiQe_a/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790201539513534-1567-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskRiQe_a/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790201539513534-1567-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.1.135.195:0
--local_ip_for_outbound_sockets=127.1.135.195
--webserver_interface=127.1.135.195
--webserver_port=0
--tserver_master_addrs=127.1.135.254:45605
--keytab_file=/tmp/dist-test-taskRiQe_a/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790201539513534-1567-0/krb5kdc/kudu_127.1.135.195.keytab
--principal=kudu/127.1.135.195
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.1.135.212:46511
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskRiQe_a/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790201539513534-1567-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskRiQe_a/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790201539513534-1567-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskRiQe_a/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790201539513534-1567-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260923 22:12:47.183810 7473 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260923 22:12:47.184017 7473 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260923 22:12:47.184054 7473 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260923 22:12:47.185194 7473 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260923 22:12:47.185256 7473 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260923 22:12:47.185302 7473 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.1.135.195
I20260923 22:12:47.186734 7473 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.1.135.212:46511
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskRiQe_a/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790201539513534-1567-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskRiQe_a/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790201539513534-1567-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskRiQe_a/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790201539513534-1567-0/krb5kdc/kudu_127.1.135.195.keytab
--principal=kudu/127.1.135.195
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.1.135.195:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskRiQe_a/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790201539513534-1567-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.1.135.195
--webserver_port=0
--tserver_master_addrs=127.1.135.254:45605
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.7473
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.1.135.195
--log_dir=/tmp/dist-test-taskRiQe_a/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790201539513534-1567-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 23 Sep 2026 20:53:37 UTC on ccycloud.quasar-rkviyx.root.comops.site
build id 4668083
I20260923 22:12:47.186980 7473 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260923 22:12:47.187201 7473 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260923 22:12:47.191882 7479 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260923 22:12:47.191895 7481 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260923 22:12:47.191951 7478 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260923 22:12:47.192044 7473 server_base.cc:1034] running on GCE node
I20260923 22:12:47.192359 7473 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260923 22:12:47.192610 7473 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260923 22:12:47.193773 7473 hybrid_clock.cc:648] HybridClock initialized: now 1790201567193759 us; error 26 us; skew 500 ppm
Sep 23 22:12:47 dist-test-slave-s2fj krb5kdc[7097](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790201567, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.135.195@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260923 22:12:47.196054 7473 init.cc:377] Logged in from keytab as kudu/127.1.135.195@KRBTEST.COM (short username kudu)
I20260923 22:12:47.196523 7473 webserver.cc:460] Webserver started at http://127.1.135.195:33539/ using document root <none> and password file <none>
I20260923 22:12:47.196756 7473 fs_manager.cc:359] Metadata directory not provided
I20260923 22:12:47.196841 7473 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260923 22:12:47.196972 7473 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260923 22:12:47.197918 7473 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskRiQe_a/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790201539513534-1567-0/minicluster-data/ts-2/data/instance:
uuid: "09885038863a468686c51e8ba2c82e85"
format_stamp: "Formatted at 2026-09-23 22:12:47 on dist-test-slave-s2fj"
server_key: "3ca0f6fecc47c66105fc743a01538995"
server_key_iv: "8337912b2f68df6171430b628f1eaf33"
server_key_version: "encryptionkey@0"
I20260923 22:12:47.198283 7473 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskRiQe_a/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790201539513534-1567-0/minicluster-data/ts-2/wal/instance:
uuid: "09885038863a468686c51e8ba2c82e85"
format_stamp: "Formatted at 2026-09-23 22:12:47 on dist-test-slave-s2fj"
server_key: "3ca0f6fecc47c66105fc743a01538995"
server_key_iv: "8337912b2f68df6171430b628f1eaf33"
server_key_version: "encryptionkey@0"
I20260923 22:12:47.199497 7473 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260923 22:12:47.200227 7488 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260923 22:12:47.200383 7473 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260923 22:12:47.200469 7473 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskRiQe_a/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790201539513534-1567-0/minicluster-data/ts-2/data,/tmp/dist-test-taskRiQe_a/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790201539513534-1567-0/minicluster-data/ts-2/wal
uuid: "09885038863a468686c51e8ba2c82e85"
format_stamp: "Formatted at 2026-09-23 22:12:47 on dist-test-slave-s2fj"
server_key: "3ca0f6fecc47c66105fc743a01538995"
server_key_iv: "8337912b2f68df6171430b628f1eaf33"
server_key_version: "encryptionkey@0"
I20260923 22:12:47.200563 7473 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskRiQe_a/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790201539513534-1567-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskRiQe_a/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790201539513534-1567-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskRiQe_a/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790201539513534-1567-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260923 22:12:47.212039 7473 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260923 22:12:47.213724 7473 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260923 22:12:47.213867 7473 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260923 22:12:47.214057 7473 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260923 22:12:47.214344 7473 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260923 22:12:47.214392 7473 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260923 22:12:47.214430 7473 ts_tablet_manager.cc:610] Registered 0 tablets
I20260923 22:12:47.214458 7473 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260923 22:12:47.219913 7473 rpc_server.cc:307] RPC server started. Bound to: 127.1.135.195:42507
I20260923 22:12:47.219946 7601 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.1.135.195:42507 every 8 connection(s)
I20260923 22:12:47.220306 7473 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskRiQe_a/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790201539513534-1567-0/minicluster-data/ts-2/data/info.pb
Sep 23 22:12:47 dist-test-slave-s2fj krb5kdc[7097](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790201567, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.135.195@KRBTEST.COM for kudu/127.1.135.254@KRBTEST.COM
I20260923 22:12:47.227213 1567 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskRiQe_a/build/release/bin/kudu as pid 7473
I20260923 22:12:47.227316 1567 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskRiQe_a/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790201539513534-1567-0/minicluster-data/ts-2/wal/instance
I20260923 22:12:47.227523 1567 external_mini_cluster.cc:1428] Setting key 168adcd4e66dec4b2fd65e102b79a3bf
I20260923 22:12:47.228964 7181 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 22:12:47.220998 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.135.195:60395 (local address 127.1.135.254:45605)
0923 22:12:47.221052 (+ 54us) server_negotiation.cc:206] Beginning negotiation
0923 22:12:47.221054 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0923 22:12:47.221211 (+ 157us) server_negotiation.cc:407] Connection header received
0923 22:12:47.222052 (+ 841us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 22:12:47.222053 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 22:12:47.222061 (+ 8us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 22:12:47.222120 (+ 59us) server_negotiation.cc:226] Negotiated authn=SASL
0923 22:12:47.222651 (+ 531us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 22:12:47.223167 (+ 516us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 22:12:47.224164 (+ 997us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 22:12:47.224346 (+ 182us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 22:12:47.224393 (+ 47us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 22:12:47.226202 (+ 1809us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0923 22:12:47.226203 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0923 22:12:47.226203 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0923 22:12:47.226209 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0923 22:12:47.227584 (+ 1375us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0923 22:12:47.228057 (+ 473us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0923 22:12:47.228058 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0923 22:12:47.228079 (+ 21us) server_negotiation.cc:967] calling sasl_server_step()
0923 22:12:47.228166 (+ 87us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0923 22:12:47.228445 (+ 279us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0923 22:12:47.228446 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0923 22:12:47.228446 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0923 22:12:47.228578 (+ 132us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0923 22:12:47.228647 (+ 69us) server_negotiation.cc:1035] Waiting for connection context
0923 22:12:47.228849 (+ 202us) server_negotiation.cc:299] Negotiation successful
0923 22:12:47.228905 (+ 56us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":30}
I20260923 22:12:47.229179 7604 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 22:12:47.220958 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.135.254:45605 (local address 127.1.135.195:60395)
0923 22:12:47.221098 (+ 140us) negotiation.cc:107] Waiting for socket to connect
0923 22:12:47.221135 (+ 37us) client_negotiation.cc:174] Beginning negotiation
0923 22:12:47.221976 (+ 841us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0923 22:12:47.222184 (+ 208us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0923 22:12:47.222187 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0923 22:12:47.222404 (+ 217us) client_negotiation.cc:189] Negotiated authn=SASL
0923 22:12:47.222585 (+ 181us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 22:12:47.222589 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 22:12:47.223255 (+ 666us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 22:12:47.223256 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 22:12:47.224039 (+ 783us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 22:12:47.224041 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 22:12:47.224407 (+ 366us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 22:12:47.224408 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 22:12:47.224511 (+ 103us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 22:12:47.224895 (+ 384us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0923 22:12:47.224902 (+ 7us) client_negotiation.cc:656] calling sasl_client_start()
0923 22:12:47.226119 (+ 1217us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0923 22:12:47.227674 (+ 1555us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0923 22:12:47.227680 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0923 22:12:47.227682 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0923 22:12:47.227991 (+ 309us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0923 22:12:47.228254 (+ 263us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0923 22:12:47.228255 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0923 22:12:47.228256 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0923 22:12:47.228379 (+ 123us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0923 22:12:47.228655 (+ 276us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0923 22:12:47.228659 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0923 22:12:47.228758 (+ 99us) client_negotiation.cc:769] Sending connection context
0923 22:12:47.228837 (+ 79us) client_negotiation.cc:240] Negotiation successful
0923 22:12:47.228942 (+ 105us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":105,"thread_start_us":80,"threads_started":1}
I20260923 22:12:47.229637 7602 heartbeater.cc:347] Connected to a master server at 127.1.135.254:45605
I20260923 22:12:47.229732 7602 heartbeater.cc:464] Registering TS with master...
I20260923 22:12:47.229887 7602 heartbeater.cc:511] Master 127.1.135.254:45605 requested a full tablet report, sending...
I20260923 22:12:47.230242 7142 ts_manager.cc:194] Registered new tserver with Master: 09885038863a468686c51e8ba2c82e85 (127.1.135.195:42507)
I20260923 22:12:47.230551 7142 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.1.135.195@KRBTEST.COM'} at 127.1.135.195:60395
I20260923 22:12:47.239223 1567 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal test-admin with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskRiQe_a/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790201539513534-1567-0/krb5kdc/test-admin.keytab.
Entry for principal test-admin with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskRiQe_a/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790201539513534-1567-0/krb5kdc/test-admin.keytab.
Sep 23 22:12:47 dist-test-slave-s2fj krb5kdc[7097](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790201567, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260923 22:12:47.253575 1567 init.cc:377] Logged in from keytab as test-admin@KRBTEST.COM (short username test-admin)
W20260923 22:12:47.254945 7609 negotiation.cc:337] Failed RPC negotiation. Trace:
0923 22:12:47.254260 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46698 (local address 127.1.135.193:41945)
0923 22:12:47.254431 (+ 171us) server_negotiation.cc:206] Beginning negotiation
0923 22:12:47.254435 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0923 22:12:47.254444 (+ 9us) server_negotiation.cc:407] Connection header received
0923 22:12:47.254467 (+ 23us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 22:12:47.254468 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 22:12:47.254480 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 22:12:47.254541 (+ 61us) server_negotiation.cc:226] Negotiated authn=SASL
0923 22:12:47.254855 (+ 314us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:46698: BlockingRecv error: recv got EOF from 127.0.0.1:46698 (error 108)
Metrics: {"server-negotiator.queue_time_us":139,"thread_start_us":94,"threads_started":1}
W20260923 22:12:47.254987 1567 init.cc:300] Could not find kerberos principal in credential cache '/tmp/dist-test-taskRiQe_a/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790201539513534-1567-0/krb5kdc/krb5cc' of type FILE
Sep 23 22:12:47 dist-test-slave-s2fj krb5kdc[7097](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790201567, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260923 22:12:47.255884 1567 init.cc:377] Logged in from keytab as test-admin@KRBTEST.COM (short username test-admin)
Sep 23 22:12:47 dist-test-slave-s2fj krb5kdc[7097](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790201567, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260923 22:12:47.256742 1567 init.cc:295] Successfully reacquired a new kerberos TGT
Sep 23 22:12:47 dist-test-slave-s2fj krb5kdc[7097](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790201567, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.1.135.193@KRBTEST.COM
I20260923 22:12:47.264089 7609 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 22:12:47.257220 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46702 (local address 127.1.135.193:41945)
0923 22:12:47.257276 (+ 56us) server_negotiation.cc:206] Beginning negotiation
0923 22:12:47.257278 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0923 22:12:47.257304 (+ 26us) server_negotiation.cc:407] Connection header received
0923 22:12:47.257385 (+ 81us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 22:12:47.257386 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 22:12:47.257397 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 22:12:47.257461 (+ 64us) server_negotiation.cc:226] Negotiated authn=SASL
0923 22:12:47.257847 (+ 386us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 22:12:47.258518 (+ 671us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 22:12:47.259217 (+ 699us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 22:12:47.259518 (+ 301us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 22:12:47.259610 (+ 92us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 22:12:47.261526 (+ 1916us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0923 22:12:47.261533 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0923 22:12:47.261533 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0923 22:12:47.261540 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0923 22:12:47.262937 (+ 1397us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0923 22:12:47.263298 (+ 361us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0923 22:12:47.263301 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0923 22:12:47.263301 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0923 22:12:47.263355 (+ 54us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0923 22:12:47.263540 (+ 185us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0923 22:12:47.263541 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0923 22:12:47.263541 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0923 22:12:47.263669 (+ 128us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0923 22:12:47.263812 (+ 143us) server_negotiation.cc:1035] Waiting for connection context
0923 22:12:47.263882 (+ 70us) server_negotiation.cc:299] Negotiation successful
0923 22:12:47.263957 (+ 75us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":35}
W20260923 22:12:47.265563 7609 negotiation.cc:337] Failed RPC negotiation. Trace:
0923 22:12:47.265108 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46710 (local address 127.1.135.193:41945)
0923 22:12:47.265161 (+ 53us) server_negotiation.cc:206] Beginning negotiation
0923 22:12:47.265162 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0923 22:12:47.265173 (+ 11us) server_negotiation.cc:407] Connection header received
0923 22:12:47.265231 (+ 58us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 22:12:47.265231 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 22:12:47.265239 (+ 8us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 22:12:47.265282 (+ 43us) server_negotiation.cc:226] Negotiated authn=SASL
0923 22:12:47.265531 (+ 249us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:46710: BlockingRecv error: recv got EOF from 127.0.0.1:46710 (error 108)
Metrics: {"server-negotiator.queue_time_us":36}
W20260923 22:12:47.265635 1567 init.cc:300] Could not find kerberos principal in credential cache '/tmp/dist-test-taskRiQe_a/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790201539513534-1567-0/krb5kdc/krb5cc' of type FILE
Sep 23 22:12:47 dist-test-slave-s2fj krb5kdc[7097](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790201567, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260923 22:12:47.266498 1567 init.cc:377] Logged in from keytab as test-admin@KRBTEST.COM (short username test-admin)
Sep 23 22:12:47 dist-test-slave-s2fj krb5kdc[7097](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790201567, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260923 22:12:47.267253 1567 init.cc:295] Successfully reacquired a new kerberos TGT
Sep 23 22:12:47 dist-test-slave-s2fj krb5kdc[7097](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790201567, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.1.135.193@KRBTEST.COM
I20260923 22:12:47.273409 7609 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 22:12:47.267699 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46718 (local address 127.1.135.193:41945)
0923 22:12:47.267745 (+ 46us) server_negotiation.cc:206] Beginning negotiation
0923 22:12:47.267747 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0923 22:12:47.267785 (+ 38us) server_negotiation.cc:407] Connection header received
0923 22:12:47.267830 (+ 45us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 22:12:47.267831 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 22:12:47.267837 (+ 6us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 22:12:47.267879 (+ 42us) server_negotiation.cc:226] Negotiated authn=SASL
0923 22:12:47.268261 (+ 382us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 22:12:47.268716 (+ 455us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 22:12:47.269314 (+ 598us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 22:12:47.269483 (+ 169us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 22:12:47.269529 (+ 46us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 22:12:47.271203 (+ 1674us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0923 22:12:47.271205 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0923 22:12:47.271205 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0923 22:12:47.271210 (+ 5us) server_negotiation.cc:922] calling sasl_server_start()
0923 22:12:47.272267 (+ 1057us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0923 22:12:47.272681 (+ 414us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0923 22:12:47.272682 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0923 22:12:47.272682 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0923 22:12:47.272739 (+ 57us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0923 22:12:47.272952 (+ 213us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0923 22:12:47.272953 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0923 22:12:47.272953 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0923 22:12:47.273039 (+ 86us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0923 22:12:47.273109 (+ 70us) server_negotiation.cc:1035] Waiting for connection context
0923 22:12:47.273222 (+ 113us) server_negotiation.cc:299] Negotiation successful
0923 22:12:47.273272 (+ 50us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":29}
W20260923 22:12:47.274631 7609 negotiation.cc:337] Failed RPC negotiation. Trace:
0923 22:12:47.274234 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46720 (local address 127.1.135.193:41945)
0923 22:12:47.274267 (+ 33us) server_negotiation.cc:206] Beginning negotiation
0923 22:12:47.274268 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0923 22:12:47.274277 (+ 9us) server_negotiation.cc:407] Connection header received
0923 22:12:47.274334 (+ 57us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 22:12:47.274334 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 22:12:47.274340 (+ 6us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 22:12:47.274375 (+ 35us) server_negotiation.cc:226] Negotiated authn=SASL
0923 22:12:47.274574 (+ 199us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:46720: BlockingRecv error: recv got EOF from 127.0.0.1:46720 (error 108)
Metrics: {"server-negotiator.queue_time_us":17}
W20260923 22:12:47.274740 1567 init.cc:300] Could not find kerberos principal in credential cache '/tmp/dist-test-taskRiQe_a/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790201539513534-1567-0/krb5kdc/krb5cc' of type FILE
Sep 23 22:12:47 dist-test-slave-s2fj krb5kdc[7097](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790201567, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260923 22:12:47.275580 1567 init.cc:377] Logged in from keytab as test-admin@KRBTEST.COM (short username test-admin)
Sep 23 22:12:47 dist-test-slave-s2fj krb5kdc[7097](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790201567, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260923 22:12:47.276398 1567 init.cc:295] Successfully reacquired a new kerberos TGT
Sep 23 22:12:47 dist-test-slave-s2fj krb5kdc[7097](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790201567, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.1.135.193@KRBTEST.COM
I20260923 22:12:47.282618 7609 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 22:12:47.276854 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46726 (local address 127.1.135.193:41945)
0923 22:12:47.276902 (+ 48us) server_negotiation.cc:206] Beginning negotiation
0923 22:12:47.276904 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0923 22:12:47.276955 (+ 51us) server_negotiation.cc:407] Connection header received
0923 22:12:47.277025 (+ 70us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 22:12:47.277026 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 22:12:47.277035 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 22:12:47.277109 (+ 74us) server_negotiation.cc:226] Negotiated authn=SASL
0923 22:12:47.277447 (+ 338us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 22:12:47.277873 (+ 426us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 22:12:47.278524 (+ 651us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 22:12:47.278700 (+ 176us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 22:12:47.278775 (+ 75us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 22:12:47.280518 (+ 1743us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0923 22:12:47.280519 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0923 22:12:47.280520 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0923 22:12:47.280526 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0923 22:12:47.281617 (+ 1091us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0923 22:12:47.281948 (+ 331us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0923 22:12:47.281949 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0923 22:12:47.281950 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0923 22:12:47.282001 (+ 51us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0923 22:12:47.282196 (+ 195us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0923 22:12:47.282196 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0923 22:12:47.282197 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0923 22:12:47.282285 (+ 88us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0923 22:12:47.282374 (+ 89us) server_negotiation.cc:1035] Waiting for connection context
0923 22:12:47.282470 (+ 96us) server_negotiation.cc:299] Negotiation successful
0923 22:12:47.282536 (+ 66us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":30}
I20260923 22:12:47.283228 1567 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskRiQe_a/build/release/bin/kudu with pid 7205
I20260923 22:12:47.287832 1567 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskRiQe_a/build/release/bin/kudu with pid 7339
I20260923 22:12:47.292001 1567 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskRiQe_a/build/release/bin/kudu with pid 7473
I20260923 22:12:47.296595 1567 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskRiQe_a/build/release/bin/kudu with pid 7110
2026-09-23T22:12:47Z chronyd exiting
[ OK ] SecurityITest.TestCorruptKerberosCC (5170 ms)
[ RUN ] SecurityITest.TestNonDefaultPrincipalMultipleMaster
Loading random data
Initializing database '/tmp/dist-test-taskRiQe_a/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790201539513534-1567-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Sep 23 22:12:47 dist-test-slave-s2fj krb5kdc[7622](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Sep 23 22:12:47 dist-test-slave-s2fj krb5kdc[7622](info): set up 2 sockets
Sep 23 22:12:47 dist-test-slave-s2fj krb5kdc[7622](info): commencing operation
krb5kdc: starting...
W20260923 22:12:48.332319 1567 mini_kdc.cc:121] Time spent starting KDC: real 1.016s user 0.000s sys 0.005s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskRiQe_a/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790201539513534-1567-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskRiQe_a/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790201539513534-1567-0/krb5kdc/test-user.keytab.
Sep 23 22:12:48 dist-test-slave-s2fj krb5kdc[7622](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790201568, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-09-23T22:12:48Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-09-23T22:12:48Z Disabled control of system clock
WARNING: no policy specified for oryx/127.1.135.254@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.1.135.254@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.1.135.254 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskRiQe_a/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790201539513534-1567-0/krb5kdc/oryx_127.1.135.254.keytab.
Entry for principal oryx/127.1.135.254 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskRiQe_a/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790201539513534-1567-0/krb5kdc/oryx_127.1.135.254.keytab.
I20260923 22:12:48.442211 1567 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskRiQe_a/build/release/bin/kudu
/tmp/dist-test-taskRiQe_a/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskRiQe_a/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790201539513534-1567-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskRiQe_a/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790201539513534-1567-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskRiQe_a/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790201539513534-1567-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskRiQe_a/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790201539513534-1567-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.1.135.254:35949
--webserver_interface=127.1.135.254
--webserver_port=0
--keytab_file=/tmp/dist-test-taskRiQe_a/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790201539513534-1567-0/krb5kdc/oryx_127.1.135.254.keytab
--principal=oryx/127.1.135.254
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.1.135.212:35751
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.1.135.254:35949
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskRiQe_a/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790201539513534-1567-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskRiQe_a/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790201539513534-1567-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskRiQe_a/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790201539513534-1567-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260923 22:12:48.520042 7635 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260923 22:12:48.520274 7635 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260923 22:12:48.520309 7635 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260923 22:12:48.521431 7635 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260923 22:12:48.521490 7635 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260923 22:12:48.521515 7635 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260923 22:12:48.521543 7635 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260923 22:12:48.521572 7635 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260923 22:12:48.522974 7635 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.1.135.212:35751
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskRiQe_a/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790201539513534-1567-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskRiQe_a/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790201539513534-1567-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.1.135.254:35949
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskRiQe_a/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790201539513534-1567-0/krb5kdc/oryx_127.1.135.254.keytab
--principal=oryx/127.1.135.254
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.1.135.254:35949
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskRiQe_a/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790201539513534-1567-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.1.135.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.7635
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskRiQe_a/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790201539513534-1567-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 23 Sep 2026 20:53:37 UTC on ccycloud.quasar-rkviyx.root.comops.site
build id 4668083
I20260923 22:12:48.523221 7635 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260923 22:12:48.523430 7635 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260923 22:12:48.527949 7640 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260923 22:12:48.527954 7641 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260923 22:12:48.528164 7643 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260923 22:12:48.528236 7635 server_base.cc:1034] running on GCE node
I20260923 22:12:48.528503 7635 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260923 22:12:48.528774 7635 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260923 22:12:48.529935 7635 hybrid_clock.cc:648] HybridClock initialized: now 1790201568529929 us; error 40 us; skew 500 ppm
Sep 23 22:12:48 dist-test-slave-s2fj krb5kdc[7622](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790201568, etypes {rep=17 tkt=17 ses=17}, oryx/127.1.135.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260923 22:12:48.532253 7635 init.cc:377] Logged in from keytab as oryx/127.1.135.254@KRBTEST.COM (short username oryx)
I20260923 22:12:48.532696 7635 webserver.cc:460] Webserver started at http://127.1.135.254:35823/ using document root <none> and password file <none>
I20260923 22:12:48.532908 7635 fs_manager.cc:359] Metadata directory not provided
I20260923 22:12:48.532976 7635 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260923 22:12:48.533111 7635 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260923 22:12:48.534070 7635 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskRiQe_a/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790201539513534-1567-0/minicluster-data/master-0/data/instance:
uuid: "6378d422392b4a74831ae1c0f0692998"
format_stamp: "Formatted at 2026-09-23 22:12:48 on dist-test-slave-s2fj"
server_key: "de5cf64b166ac68866c74b7fe18be9b6"
server_key_iv: "850466a815b8392fe38c0b4c0d3a42f6"
server_key_version: "encryptionkey@0"
I20260923 22:12:48.534425 7635 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskRiQe_a/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790201539513534-1567-0/minicluster-data/master-0/wal/instance:
uuid: "6378d422392b4a74831ae1c0f0692998"
format_stamp: "Formatted at 2026-09-23 22:12:48 on dist-test-slave-s2fj"
server_key: "de5cf64b166ac68866c74b7fe18be9b6"
server_key_iv: "850466a815b8392fe38c0b4c0d3a42f6"
server_key_version: "encryptionkey@0"
I20260923 22:12:48.535681 7635 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260923 22:12:48.536409 7650 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260923 22:12:48.536568 7635 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.000s
I20260923 22:12:48.536659 7635 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskRiQe_a/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790201539513534-1567-0/minicluster-data/master-0/data,/tmp/dist-test-taskRiQe_a/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790201539513534-1567-0/minicluster-data/master-0/wal
uuid: "6378d422392b4a74831ae1c0f0692998"
format_stamp: "Formatted at 2026-09-23 22:12:48 on dist-test-slave-s2fj"
server_key: "de5cf64b166ac68866c74b7fe18be9b6"
server_key_iv: "850466a815b8392fe38c0b4c0d3a42f6"
server_key_version: "encryptionkey@0"
I20260923 22:12:48.536736 7635 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskRiQe_a/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790201539513534-1567-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskRiQe_a/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790201539513534-1567-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskRiQe_a/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790201539513534-1567-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260923 22:12:48.562652 7635 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260923 22:12:48.564389 7635 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260923 22:12:48.564539 7635 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260923 22:12:48.568236 7635 rpc_server.cc:307] RPC server started. Bound to: 127.1.135.254:35949
I20260923 22:12:48.568270 7702 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.1.135.254:35949 every 8 connection(s)
I20260923 22:12:48.568662 7635 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskRiQe_a/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790201539513534-1567-0/minicluster-data/master-0/data/info.pb
I20260923 22:12:48.569481 7703 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260923 22:12:48.571565 7703 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 6378d422392b4a74831ae1c0f0692998: Bootstrap starting.
I20260923 22:12:48.572144 7703 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 6378d422392b4a74831ae1c0f0692998: Neither blocks nor log segments found. Creating new log.
I20260923 22:12:48.572410 7703 log.cc:824] T 00000000000000000000000000000000 P 6378d422392b4a74831ae1c0f0692998: Log is configured to *not* fsync() on all Append() calls
I20260923 22:12:48.573050 7703 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 6378d422392b4a74831ae1c0f0692998: No bootstrap required, opened a new log
I20260923 22:12:48.574556 7703 raft_consensus.cc:348] T 00000000000000000000000000000000 P 6378d422392b4a74831ae1c0f0692998 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6378d422392b4a74831ae1c0f0692998" member_type: VOTER last_known_addr { host: "127.1.135.254" port: 35949 } }
I20260923 22:12:48.574685 7703 raft_consensus.cc:374] T 00000000000000000000000000000000 P 6378d422392b4a74831ae1c0f0692998 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260923 22:12:48.574752 7703 raft_consensus.cc:729] T 00000000000000000000000000000000 P 6378d422392b4a74831ae1c0f0692998 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6378d422392b4a74831ae1c0f0692998, State: Initialized, Role: FOLLOWER
I20260923 22:12:48.574882 7703 consensus_queue.cc:260] T 00000000000000000000000000000000 P 6378d422392b4a74831ae1c0f0692998 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6378d422392b4a74831ae1c0f0692998" member_type: VOTER last_known_addr { host: "127.1.135.254" port: 35949 } }
I20260923 22:12:48.574936 7703 raft_consensus.cc:388] T 00000000000000000000000000000000 P 6378d422392b4a74831ae1c0f0692998 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260923 22:12:48.574988 7703 raft_consensus.cc:482] T 00000000000000000000000000000000 P 6378d422392b4a74831ae1c0f0692998 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260923 22:12:48.575033 7703 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 6378d422392b4a74831ae1c0f0692998 [term 0 FOLLOWER]: Advancing to term 1
I20260923 22:12:48.575768 7703 raft_consensus.cc:504] T 00000000000000000000000000000000 P 6378d422392b4a74831ae1c0f0692998 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6378d422392b4a74831ae1c0f0692998" member_type: VOTER last_known_addr { host: "127.1.135.254" port: 35949 } }
I20260923 22:12:48.575871 7703 leader_election.cc:302] T 00000000000000000000000000000000 P 6378d422392b4a74831ae1c0f0692998 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 6378d422392b4a74831ae1c0f0692998; no voters:
I20260923 22:12:48.576040 7703 leader_election.cc:288] T 00000000000000000000000000000000 P 6378d422392b4a74831ae1c0f0692998 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260923 22:12:48.576092 7706 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 6378d422392b4a74831ae1c0f0692998 [term 1 FOLLOWER]: Leader election won for term 1
I20260923 22:12:48.576249 7706 raft_consensus.cc:686] T 00000000000000000000000000000000 P 6378d422392b4a74831ae1c0f0692998 [term 1 LEADER]: Becoming Leader. State: Replica: 6378d422392b4a74831ae1c0f0692998, State: Running, Role: LEADER
I20260923 22:12:48.576505 1567 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskRiQe_a/build/release/bin/kudu as pid 7635
I20260923 22:12:48.576619 1567 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskRiQe_a/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790201539513534-1567-0/minicluster-data/master-0/wal/instance
I20260923 22:12:48.576798 1567 external_mini_cluster.cc:1428] Setting key f476dc613c40eca24ced6155cba1c39c
I20260923 22:12:48.576323 7703 sys_catalog.cc:564] T 00000000000000000000000000000000 P 6378d422392b4a74831ae1c0f0692998 [sys.catalog]: configured and running, proceeding with master startup.
I20260923 22:12:48.577035 7706 consensus_queue.cc:237] T 00000000000000000000000000000000 P 6378d422392b4a74831ae1c0f0692998 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6378d422392b4a74831ae1c0f0692998" member_type: VOTER last_known_addr { host: "127.1.135.254" port: 35949 } }
I20260923 22:12:48.577528 7708 sys_catalog.cc:455] T 00000000000000000000000000000000 P 6378d422392b4a74831ae1c0f0692998 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 6378d422392b4a74831ae1c0f0692998. Latest consensus state: current_term: 1 leader_uuid: "6378d422392b4a74831ae1c0f0692998" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6378d422392b4a74831ae1c0f0692998" member_type: VOTER last_known_addr { host: "127.1.135.254" port: 35949 } } }
I20260923 22:12:48.577793 7708 sys_catalog.cc:458] T 00000000000000000000000000000000 P 6378d422392b4a74831ae1c0f0692998 [sys.catalog]: This master's current role is: LEADER
I20260923 22:12:48.577733 7707 sys_catalog.cc:455] T 00000000000000000000000000000000 P 6378d422392b4a74831ae1c0f0692998 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "6378d422392b4a74831ae1c0f0692998" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6378d422392b4a74831ae1c0f0692998" member_type: VOTER last_known_addr { host: "127.1.135.254" port: 35949 } } }
I20260923 22:12:48.578040 7707 sys_catalog.cc:458] T 00000000000000000000000000000000 P 6378d422392b4a74831ae1c0f0692998 [sys.catalog]: This master's current role is: LEADER
I20260923 22:12:48.578195 7717 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260923 22:12:48.578864 7717 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260923 22:12:48.580201 7717 catalog_manager.cc:1347] Generated new cluster ID: e03b1c9cbcbb49658f49f0be6e6c5546
I20260923 22:12:48.580251 7717 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
Sep 23 22:12:48 dist-test-slave-s2fj krb5kdc[7622](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790201568, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for oryx/127.1.135.254@KRBTEST.COM
I20260923 22:12:48.588261 7713 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 22:12:48.577404 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52028 (local address 127.1.135.254:35949)
0923 22:12:48.577576 (+ 172us) server_negotiation.cc:206] Beginning negotiation
0923 22:12:48.577584 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0923 22:12:48.577615 (+ 31us) server_negotiation.cc:407] Connection header received
0923 22:12:48.577732 (+ 117us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 22:12:48.577740 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 22:12:48.577795 (+ 55us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 22:12:48.577941 (+ 146us) server_negotiation.cc:226] Negotiated authn=SASL
0923 22:12:48.578905 (+ 964us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 22:12:48.580309 (+ 1404us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 22:12:48.581327 (+ 1018us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 22:12:48.581974 (+ 647us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 22:12:48.582076 (+ 102us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 22:12:48.584786 (+ 2710us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0923 22:12:48.584789 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0923 22:12:48.584789 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0923 22:12:48.584799 (+ 10us) server_negotiation.cc:922] calling sasl_server_start()
0923 22:12:48.586646 (+ 1847us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0923 22:12:48.587058 (+ 412us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0923 22:12:48.587061 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0923 22:12:48.587061 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0923 22:12:48.587169 (+ 108us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0923 22:12:48.587372 (+ 203us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0923 22:12:48.587373 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0923 22:12:48.587373 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0923 22:12:48.587659 (+ 286us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0923 22:12:48.587807 (+ 148us) server_negotiation.cc:1035] Waiting for connection context
0923 22:12:48.587912 (+ 105us) server_negotiation.cc:299] Negotiation successful
0923 22:12:48.588036 (+ 124us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":131,"thread_start_us":82,"threads_started":1}
I20260923 22:12:48.589061 7717 catalog_manager.cc:1370] Generated new certificate authority record
I20260923 22:12:48.589511 7717 catalog_manager.cc:1504] Loading token signing keys...
I20260923 22:12:48.591287 7717 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 6378d422392b4a74831ae1c0f0692998: Generated new TSK 0
I20260923 22:12:48.591442 7717 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for oryx/127.1.135.193@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.1.135.193@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.1.135.193 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskRiQe_a/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790201539513534-1567-0/krb5kdc/oryx_127.1.135.193.keytab.
Entry for principal oryx/127.1.135.193 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskRiQe_a/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790201539513534-1567-0/krb5kdc/oryx_127.1.135.193.keytab.
I20260923 22:12:48.616904 1567 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskRiQe_a/build/release/bin/kudu
/tmp/dist-test-taskRiQe_a/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskRiQe_a/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790201539513534-1567-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskRiQe_a/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790201539513534-1567-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskRiQe_a/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790201539513534-1567-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskRiQe_a/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790201539513534-1567-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.1.135.193:0
--local_ip_for_outbound_sockets=127.1.135.193
--webserver_interface=127.1.135.193
--webserver_port=0
--tserver_master_addrs=127.1.135.254:35949
--keytab_file=/tmp/dist-test-taskRiQe_a/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790201539513534-1567-0/krb5kdc/oryx_127.1.135.193.keytab
--principal=oryx/127.1.135.193
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.1.135.212:35751
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskRiQe_a/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790201539513534-1567-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskRiQe_a/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790201539513534-1567-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskRiQe_a/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790201539513534-1567-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260923 22:12:48.695343 7729 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260923 22:12:48.695549 7729 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260923 22:12:48.695587 7729 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260923 22:12:48.696744 7729 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260923 22:12:48.696802 7729 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260923 22:12:48.696842 7729 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.1.135.193
I20260923 22:12:48.698254 7729 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.1.135.212:35751
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskRiQe_a/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790201539513534-1567-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskRiQe_a/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790201539513534-1567-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskRiQe_a/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790201539513534-1567-0/krb5kdc/oryx_127.1.135.193.keytab
--principal=oryx/127.1.135.193
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.1.135.193:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskRiQe_a/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790201539513534-1567-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.1.135.193
--webserver_port=0
--tserver_master_addrs=127.1.135.254:35949
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.7729
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.1.135.193
--log_dir=/tmp/dist-test-taskRiQe_a/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790201539513534-1567-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 23 Sep 2026 20:53:37 UTC on ccycloud.quasar-rkviyx.root.comops.site
build id 4668083
I20260923 22:12:48.698489 7729 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260923 22:12:48.698700 7729 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260923 22:12:48.704294 7737 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260923 22:12:48.704299 7734 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260923 22:12:48.704281 7735 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260923 22:12:48.704514 7729 server_base.cc:1034] running on GCE node
I20260923 22:12:48.704779 7729 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260923 22:12:48.705011 7729 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260923 22:12:48.706166 7729 hybrid_clock.cc:648] HybridClock initialized: now 1790201568706161 us; error 34 us; skew 500 ppm
Sep 23 22:12:48 dist-test-slave-s2fj krb5kdc[7622](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790201568, etypes {rep=17 tkt=17 ses=17}, oryx/127.1.135.193@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260923 22:12:48.708577 7729 init.cc:377] Logged in from keytab as oryx/127.1.135.193@KRBTEST.COM (short username oryx)
I20260923 22:12:48.709085 7729 webserver.cc:460] Webserver started at http://127.1.135.193:44031/ using document root <none> and password file <none>
I20260923 22:12:48.709311 7729 fs_manager.cc:359] Metadata directory not provided
I20260923 22:12:48.709370 7729 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260923 22:12:48.709494 7729 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260923 22:12:48.710441 7729 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskRiQe_a/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790201539513534-1567-0/minicluster-data/ts-0/data/instance:
uuid: "e01531ae61ec4025942988b1a6dd62f0"
format_stamp: "Formatted at 2026-09-23 22:12:48 on dist-test-slave-s2fj"
server_key: "6daedd3333b0ce6e7fa0ae7ca7e6e6ea"
server_key_iv: "fbf19d41bd7d3e3c84024fec410a7ad5"
server_key_version: "encryptionkey@0"
I20260923 22:12:48.710788 7729 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskRiQe_a/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790201539513534-1567-0/minicluster-data/ts-0/wal/instance:
uuid: "e01531ae61ec4025942988b1a6dd62f0"
format_stamp: "Formatted at 2026-09-23 22:12:48 on dist-test-slave-s2fj"
server_key: "6daedd3333b0ce6e7fa0ae7ca7e6e6ea"
server_key_iv: "fbf19d41bd7d3e3c84024fec410a7ad5"
server_key_version: "encryptionkey@0"
I20260923 22:12:48.712049 7729 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.001s
I20260923 22:12:48.712764 7744 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260923 22:12:48.712920 7729 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.001s
I20260923 22:12:48.713016 7729 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskRiQe_a/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790201539513534-1567-0/minicluster-data/ts-0/data,/tmp/dist-test-taskRiQe_a/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790201539513534-1567-0/minicluster-data/ts-0/wal
uuid: "e01531ae61ec4025942988b1a6dd62f0"
format_stamp: "Formatted at 2026-09-23 22:12:48 on dist-test-slave-s2fj"
server_key: "6daedd3333b0ce6e7fa0ae7ca7e6e6ea"
server_key_iv: "fbf19d41bd7d3e3c84024fec410a7ad5"
server_key_version: "encryptionkey@0"
I20260923 22:12:48.713091 7729 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskRiQe_a/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790201539513534-1567-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskRiQe_a/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790201539513534-1567-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskRiQe_a/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790201539513534-1567-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260923 22:12:48.735209 7729 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260923 22:12:48.736935 7729 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260923 22:12:48.737082 7729 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260923 22:12:48.737295 7729 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260923 22:12:48.737563 7729 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260923 22:12:48.737615 7729 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260923 22:12:48.737654 7729 ts_tablet_manager.cc:610] Registered 0 tablets
I20260923 22:12:48.737684 7729 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260923 22:12:48.743245 7729 rpc_server.cc:307] RPC server started. Bound to: 127.1.135.193:44597
I20260923 22:12:48.743304 7857 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.1.135.193:44597 every 8 connection(s)
I20260923 22:12:48.743618 7729 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskRiQe_a/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790201539513534-1567-0/minicluster-data/ts-0/data/info.pb
Sep 23 22:12:48 dist-test-slave-s2fj krb5kdc[7622](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790201568, etypes {rep=17 tkt=17 ses=17}, oryx/127.1.135.193@KRBTEST.COM for oryx/127.1.135.254@KRBTEST.COM
I20260923 22:12:48.751359 1567 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskRiQe_a/build/release/bin/kudu as pid 7729
I20260923 22:12:48.751449 1567 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskRiQe_a/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790201539513534-1567-0/minicluster-data/ts-0/wal/instance
I20260923 22:12:48.751655 1567 external_mini_cluster.cc:1428] Setting key 4784f719199ae444558a84568dccccc0
I20260923 22:12:48.752398 7860 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 22:12:48.744281 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.135.254:35949 (local address 127.1.135.193:51439)
0923 22:12:48.744489 (+ 208us) negotiation.cc:107] Waiting for socket to connect
0923 22:12:48.744532 (+ 43us) client_negotiation.cc:174] Beginning negotiation
0923 22:12:48.744971 (+ 439us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0923 22:12:48.745176 (+ 205us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0923 22:12:48.745179 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0923 22:12:48.745767 (+ 588us) client_negotiation.cc:189] Negotiated authn=SASL
0923 22:12:48.745955 (+ 188us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 22:12:48.745959 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 22:12:48.746616 (+ 657us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 22:12:48.746617 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 22:12:48.747373 (+ 756us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 22:12:48.747375 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 22:12:48.747691 (+ 316us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 22:12:48.747691 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 22:12:48.747780 (+ 89us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 22:12:48.748242 (+ 462us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0923 22:12:48.748250 (+ 8us) client_negotiation.cc:656] calling sasl_client_start()
0923 22:12:48.749472 (+ 1222us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0923 22:12:48.750947 (+ 1475us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0923 22:12:48.750951 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0923 22:12:48.750953 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0923 22:12:48.751146 (+ 193us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0923 22:12:48.751341 (+ 195us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0923 22:12:48.751341 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0923 22:12:48.751341 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0923 22:12:48.751451 (+ 110us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0923 22:12:48.751719 (+ 268us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0923 22:12:48.751725 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0923 22:12:48.751881 (+ 156us) client_negotiation.cc:769] Sending connection context
0923 22:12:48.752001 (+ 120us) client_negotiation.cc:240] Negotiation successful
0923 22:12:48.752174 (+ 173us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":151,"thread_start_us":87,"threads_started":1}
I20260923 22:12:48.752418 7713 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 22:12:48.744290 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.135.193:51439 (local address 127.1.135.254:35949)
0923 22:12:48.744338 (+ 48us) server_negotiation.cc:206] Beginning negotiation
0923 22:12:48.744340 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0923 22:12:48.744606 (+ 266us) server_negotiation.cc:407] Connection header received
0923 22:12:48.745057 (+ 451us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 22:12:48.745058 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 22:12:48.745078 (+ 20us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 22:12:48.745129 (+ 51us) server_negotiation.cc:226] Negotiated authn=SASL
0923 22:12:48.746041 (+ 912us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 22:12:48.746541 (+ 500us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 22:12:48.747441 (+ 900us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 22:12:48.747626 (+ 185us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 22:12:48.747674 (+ 48us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 22:12:48.749551 (+ 1877us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0923 22:12:48.749552 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0923 22:12:48.749552 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0923 22:12:48.749559 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0923 22:12:48.750884 (+ 1325us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0923 22:12:48.751215 (+ 331us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0923 22:12:48.751216 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0923 22:12:48.751216 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0923 22:12:48.751277 (+ 61us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0923 22:12:48.751533 (+ 256us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0923 22:12:48.751534 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0923 22:12:48.751535 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0923 22:12:48.751652 (+ 117us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0923 22:12:48.751728 (+ 76us) server_negotiation.cc:1035] Waiting for connection context
0923 22:12:48.752261 (+ 533us) server_negotiation.cc:299] Negotiation successful
0923 22:12:48.752335 (+ 74us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":20}
I20260923 22:12:48.753029 7858 heartbeater.cc:347] Connected to a master server at 127.1.135.254:35949
I20260923 22:12:48.753135 7858 heartbeater.cc:464] Registering TS with master...
I20260923 22:12:48.753310 7858 heartbeater.cc:511] Master 127.1.135.254:35949 requested a full tablet report, sending...
I20260923 22:12:48.753747 7667 ts_manager.cc:194] Registered new tserver with Master: e01531ae61ec4025942988b1a6dd62f0 (127.1.135.193:44597)
I20260923 22:12:48.754329 7667 master_service.cc:496] Signed X509 certificate for tserver {username='oryx', principal='oryx/127.1.135.193@KRBTEST.COM'} at 127.1.135.193:51439
WARNING: no policy specified for oryx/127.1.135.194@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.1.135.194@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.1.135.194 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskRiQe_a/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790201539513534-1567-0/krb5kdc/oryx_127.1.135.194.keytab.
Entry for principal oryx/127.1.135.194 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskRiQe_a/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790201539513534-1567-0/krb5kdc/oryx_127.1.135.194.keytab.
I20260923 22:12:48.774408 1567 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskRiQe_a/build/release/bin/kudu
/tmp/dist-test-taskRiQe_a/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskRiQe_a/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790201539513534-1567-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskRiQe_a/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790201539513534-1567-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskRiQe_a/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790201539513534-1567-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskRiQe_a/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790201539513534-1567-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.1.135.194:0
--local_ip_for_outbound_sockets=127.1.135.194
--webserver_interface=127.1.135.194
--webserver_port=0
--tserver_master_addrs=127.1.135.254:35949
--keytab_file=/tmp/dist-test-taskRiQe_a/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790201539513534-1567-0/krb5kdc/oryx_127.1.135.194.keytab
--principal=oryx/127.1.135.194
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.1.135.212:35751
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskRiQe_a/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790201539513534-1567-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskRiQe_a/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790201539513534-1567-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskRiQe_a/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790201539513534-1567-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260923 22:12:48.853681 7863 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260923 22:12:48.853878 7863 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260923 22:12:48.853900 7863 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260923 22:12:48.855059 7863 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260923 22:12:48.855103 7863 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260923 22:12:48.855129 7863 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.1.135.194
I20260923 22:12:48.856570 7863 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.1.135.212:35751
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskRiQe_a/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790201539513534-1567-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskRiQe_a/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790201539513534-1567-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskRiQe_a/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790201539513534-1567-0/krb5kdc/oryx_127.1.135.194.keytab
--principal=oryx/127.1.135.194
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.1.135.194:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskRiQe_a/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790201539513534-1567-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.1.135.194
--webserver_port=0
--tserver_master_addrs=127.1.135.254:35949
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.7863
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.1.135.194
--log_dir=/tmp/dist-test-taskRiQe_a/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790201539513534-1567-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 23 Sep 2026 20:53:37 UTC on ccycloud.quasar-rkviyx.root.comops.site
build id 4668083
I20260923 22:12:48.856786 7863 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260923 22:12:48.857013 7863 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260923 22:12:48.861730 7871 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260923 22:12:48.861744 7868 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260923 22:12:48.861774 7869 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260923 22:12:48.862026 7863 server_base.cc:1034] running on GCE node
I20260923 22:12:48.862214 7863 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260923 22:12:48.862437 7863 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260923 22:12:48.863579 7863 hybrid_clock.cc:648] HybridClock initialized: now 1790201568863562 us; error 31 us; skew 500 ppm
Sep 23 22:12:48 dist-test-slave-s2fj krb5kdc[7622](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790201568, etypes {rep=17 tkt=17 ses=17}, oryx/127.1.135.194@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260923 22:12:48.866030 7863 init.cc:377] Logged in from keytab as oryx/127.1.135.194@KRBTEST.COM (short username oryx)
I20260923 22:12:48.866423 7863 webserver.cc:460] Webserver started at http://127.1.135.194:42793/ using document root <none> and password file <none>
I20260923 22:12:48.866664 7863 fs_manager.cc:359] Metadata directory not provided
I20260923 22:12:48.866729 7863 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260923 22:12:48.866863 7863 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260923 22:12:48.867835 7863 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskRiQe_a/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790201539513534-1567-0/minicluster-data/ts-1/data/instance:
uuid: "8a9da5f92d7c4186877987e0ddd62caf"
format_stamp: "Formatted at 2026-09-23 22:12:48 on dist-test-slave-s2fj"
server_key: "93df3b32f1b85eb54bcee7bdb0a9b043"
server_key_iv: "3860080038c180c731262ff3d508819a"
server_key_version: "encryptionkey@0"
I20260923 22:12:48.868217 7863 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskRiQe_a/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790201539513534-1567-0/minicluster-data/ts-1/wal/instance:
uuid: "8a9da5f92d7c4186877987e0ddd62caf"
format_stamp: "Formatted at 2026-09-23 22:12:48 on dist-test-slave-s2fj"
server_key: "93df3b32f1b85eb54bcee7bdb0a9b043"
server_key_iv: "3860080038c180c731262ff3d508819a"
server_key_version: "encryptionkey@0"
I20260923 22:12:48.869501 7863 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.001s
I20260923 22:12:48.870179 7878 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260923 22:12:48.870328 7863 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.001s
I20260923 22:12:48.870416 7863 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskRiQe_a/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790201539513534-1567-0/minicluster-data/ts-1/data,/tmp/dist-test-taskRiQe_a/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790201539513534-1567-0/minicluster-data/ts-1/wal
uuid: "8a9da5f92d7c4186877987e0ddd62caf"
format_stamp: "Formatted at 2026-09-23 22:12:48 on dist-test-slave-s2fj"
server_key: "93df3b32f1b85eb54bcee7bdb0a9b043"
server_key_iv: "3860080038c180c731262ff3d508819a"
server_key_version: "encryptionkey@0"
I20260923 22:12:48.870491 7863 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskRiQe_a/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790201539513534-1567-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskRiQe_a/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790201539513534-1567-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskRiQe_a/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790201539513534-1567-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260923 22:12:48.883708 7863 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260923 22:12:48.885447 7863 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260923 22:12:48.885596 7863 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260923 22:12:48.885813 7863 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260923 22:12:48.886081 7863 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260923 22:12:48.886129 7863 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260923 22:12:48.886162 7863 ts_tablet_manager.cc:610] Registered 0 tablets
I20260923 22:12:48.886188 7863 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260923 22:12:48.891700 7863 rpc_server.cc:307] RPC server started. Bound to: 127.1.135.194:41925
I20260923 22:12:48.891760 7991 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.1.135.194:41925 every 8 connection(s)
I20260923 22:12:48.892072 7863 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskRiQe_a/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790201539513534-1567-0/minicluster-data/ts-1/data/info.pb
Sep 23 22:12:48 dist-test-slave-s2fj krb5kdc[7622](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790201568, etypes {rep=17 tkt=17 ses=17}, oryx/127.1.135.194@KRBTEST.COM for oryx/127.1.135.254@KRBTEST.COM
I20260923 22:12:48.898799 1567 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskRiQe_a/build/release/bin/kudu as pid 7863
I20260923 22:12:48.898900 1567 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskRiQe_a/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790201539513534-1567-0/minicluster-data/ts-1/wal/instance
I20260923 22:12:48.899096 1567 external_mini_cluster.cc:1428] Setting key b9f51118db92749f61e4cd979a839a69
I20260923 22:12:48.900700 7713 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 22:12:48.892767 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.135.194:42533 (local address 127.1.135.254:35949)
0923 22:12:48.892822 (+ 55us) server_negotiation.cc:206] Beginning negotiation
0923 22:12:48.892824 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0923 22:12:48.893011 (+ 187us) server_negotiation.cc:407] Connection header received
0923 22:12:48.893455 (+ 444us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 22:12:48.893456 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 22:12:48.893465 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 22:12:48.893515 (+ 50us) server_negotiation.cc:226] Negotiated authn=SASL
0923 22:12:48.894362 (+ 847us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 22:12:48.894830 (+ 468us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 22:12:48.895728 (+ 898us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 22:12:48.895901 (+ 173us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 22:12:48.895949 (+ 48us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 22:12:48.897829 (+ 1880us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0923 22:12:48.897830 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0923 22:12:48.897830 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0923 22:12:48.897836 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0923 22:12:48.899357 (+ 1521us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0923 22:12:48.899755 (+ 398us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0923 22:12:48.899756 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0923 22:12:48.899757 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0923 22:12:48.899818 (+ 61us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0923 22:12:48.900051 (+ 233us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0923 22:12:48.900052 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0923 22:12:48.900052 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0923 22:12:48.900184 (+ 132us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0923 22:12:48.900257 (+ 73us) server_negotiation.cc:1035] Waiting for connection context
0923 22:12:48.900526 (+ 269us) server_negotiation.cc:299] Negotiation successful
0923 22:12:48.900617 (+ 91us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":26}
I20260923 22:12:48.900848 7994 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 22:12:48.892744 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.135.254:35949 (local address 127.1.135.194:42533)
0923 22:12:48.892902 (+ 158us) negotiation.cc:107] Waiting for socket to connect
0923 22:12:48.892933 (+ 31us) client_negotiation.cc:174] Beginning negotiation
0923 22:12:48.893375 (+ 442us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0923 22:12:48.893574 (+ 199us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0923 22:12:48.893577 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0923 22:12:48.894121 (+ 544us) client_negotiation.cc:189] Negotiated authn=SASL
0923 22:12:48.894292 (+ 171us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 22:12:48.894296 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 22:12:48.894897 (+ 601us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 22:12:48.894898 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 22:12:48.895658 (+ 760us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 22:12:48.895659 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 22:12:48.895964 (+ 305us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 22:12:48.895965 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 22:12:48.896054 (+ 89us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 22:12:48.896513 (+ 459us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0923 22:12:48.896520 (+ 7us) client_negotiation.cc:656] calling sasl_client_start()
0923 22:12:48.897751 (+ 1231us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0923 22:12:48.899428 (+ 1677us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0923 22:12:48.899434 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0923 22:12:48.899436 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0923 22:12:48.899687 (+ 251us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0923 22:12:48.899874 (+ 187us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0923 22:12:48.899875 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0923 22:12:48.899876 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0923 22:12:48.899985 (+ 109us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0923 22:12:48.900249 (+ 264us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0923 22:12:48.900255 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0923 22:12:48.900386 (+ 131us) client_negotiation.cc:769] Sending connection context
0923 22:12:48.900492 (+ 106us) client_negotiation.cc:240] Negotiation successful
0923 22:12:48.900638 (+ 146us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":124,"thread_start_us":86,"threads_started":1}
I20260923 22:12:48.901237 7992 heartbeater.cc:347] Connected to a master server at 127.1.135.254:35949
I20260923 22:12:48.901338 7992 heartbeater.cc:464] Registering TS with master...
I20260923 22:12:48.901511 7992 heartbeater.cc:511] Master 127.1.135.254:35949 requested a full tablet report, sending...
I20260923 22:12:48.901827 7667 ts_manager.cc:194] Registered new tserver with Master: 8a9da5f92d7c4186877987e0ddd62caf (127.1.135.194:41925)
I20260923 22:12:48.902271 7667 master_service.cc:496] Signed X509 certificate for tserver {username='oryx', principal='oryx/127.1.135.194@KRBTEST.COM'} at 127.1.135.194:42533
WARNING: no policy specified for oryx/127.1.135.195@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.1.135.195@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.1.135.195 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskRiQe_a/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790201539513534-1567-0/krb5kdc/oryx_127.1.135.195.keytab.
Entry for principal oryx/127.1.135.195 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskRiQe_a/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790201539513534-1567-0/krb5kdc/oryx_127.1.135.195.keytab.
I20260923 22:12:48.922266 1567 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskRiQe_a/build/release/bin/kudu
/tmp/dist-test-taskRiQe_a/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskRiQe_a/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790201539513534-1567-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskRiQe_a/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790201539513534-1567-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskRiQe_a/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790201539513534-1567-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskRiQe_a/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790201539513534-1567-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.1.135.195:0
--local_ip_for_outbound_sockets=127.1.135.195
--webserver_interface=127.1.135.195
--webserver_port=0
--tserver_master_addrs=127.1.135.254:35949
--keytab_file=/tmp/dist-test-taskRiQe_a/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790201539513534-1567-0/krb5kdc/oryx_127.1.135.195.keytab
--principal=oryx/127.1.135.195
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.1.135.212:35751
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskRiQe_a/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790201539513534-1567-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskRiQe_a/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790201539513534-1567-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskRiQe_a/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790201539513534-1567-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260923 22:12:49.001446 7997 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260923 22:12:49.001645 7997 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260923 22:12:49.001668 7997 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260923 22:12:49.002784 7997 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260923 22:12:49.002825 7997 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260923 22:12:49.002872 7997 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.1.135.195
I20260923 22:12:49.004330 7997 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.1.135.212:35751
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskRiQe_a/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790201539513534-1567-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskRiQe_a/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790201539513534-1567-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskRiQe_a/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790201539513534-1567-0/krb5kdc/oryx_127.1.135.195.keytab
--principal=oryx/127.1.135.195
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.1.135.195:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskRiQe_a/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790201539513534-1567-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.1.135.195
--webserver_port=0
--tserver_master_addrs=127.1.135.254:35949
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.7997
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.1.135.195
--log_dir=/tmp/dist-test-taskRiQe_a/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790201539513534-1567-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 23 Sep 2026 20:53:37 UTC on ccycloud.quasar-rkviyx.root.comops.site
build id 4668083
I20260923 22:12:49.004560 7997 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260923 22:12:49.004781 7997 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260923 22:12:49.009465 8005 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260923 22:12:49.009472 8003 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260923 22:12:49.009677 7997 server_base.cc:1034] running on GCE node
W20260923 22:12:49.009472 8002 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260923 22:12:49.009941 7997 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260923 22:12:49.010167 7997 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260923 22:12:49.011334 7997 hybrid_clock.cc:648] HybridClock initialized: now 1790201569011324 us; error 28 us; skew 500 ppm
Sep 23 22:12:49 dist-test-slave-s2fj krb5kdc[7622](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790201569, etypes {rep=17 tkt=17 ses=17}, oryx/127.1.135.195@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260923 22:12:49.013739 7997 init.cc:377] Logged in from keytab as oryx/127.1.135.195@KRBTEST.COM (short username oryx)
I20260923 22:12:49.014139 7997 webserver.cc:460] Webserver started at http://127.1.135.195:39665/ using document root <none> and password file <none>
I20260923 22:12:49.014351 7997 fs_manager.cc:359] Metadata directory not provided
I20260923 22:12:49.014413 7997 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260923 22:12:49.014550 7997 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260923 22:12:49.015509 7997 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskRiQe_a/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790201539513534-1567-0/minicluster-data/ts-2/data/instance:
uuid: "4d0b914e7ab746e7b17789e7b32aca0a"
format_stamp: "Formatted at 2026-09-23 22:12:49 on dist-test-slave-s2fj"
server_key: "7b659ba92ef47ef8579fcaedc6163353"
server_key_iv: "d45dcb8288a91c35a1745c34b2a22713"
server_key_version: "encryptionkey@0"
I20260923 22:12:49.015911 7997 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskRiQe_a/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790201539513534-1567-0/minicluster-data/ts-2/wal/instance:
uuid: "4d0b914e7ab746e7b17789e7b32aca0a"
format_stamp: "Formatted at 2026-09-23 22:12:49 on dist-test-slave-s2fj"
server_key: "7b659ba92ef47ef8579fcaedc6163353"
server_key_iv: "d45dcb8288a91c35a1745c34b2a22713"
server_key_version: "encryptionkey@0"
I20260923 22:12:49.017432 7997 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.001s
I20260923 22:12:49.018131 8012 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260923 22:12:49.018277 7997 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.001s
I20260923 22:12:49.018366 7997 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskRiQe_a/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790201539513534-1567-0/minicluster-data/ts-2/data,/tmp/dist-test-taskRiQe_a/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790201539513534-1567-0/minicluster-data/ts-2/wal
uuid: "4d0b914e7ab746e7b17789e7b32aca0a"
format_stamp: "Formatted at 2026-09-23 22:12:49 on dist-test-slave-s2fj"
server_key: "7b659ba92ef47ef8579fcaedc6163353"
server_key_iv: "d45dcb8288a91c35a1745c34b2a22713"
server_key_version: "encryptionkey@0"
I20260923 22:12:49.018438 7997 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskRiQe_a/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790201539513534-1567-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskRiQe_a/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790201539513534-1567-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskRiQe_a/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790201539513534-1567-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260923 22:12:49.034538 7997 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260923 22:12:49.036396 7997 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260923 22:12:49.036541 7997 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260923 22:12:49.036742 7997 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260923 22:12:49.037021 7997 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260923 22:12:49.037071 7997 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260923 22:12:49.037112 7997 ts_tablet_manager.cc:610] Registered 0 tablets
I20260923 22:12:49.037140 7997 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260923 22:12:49.042889 7997 rpc_server.cc:307] RPC server started. Bound to: 127.1.135.195:33563
I20260923 22:12:49.042953 8125 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.1.135.195:33563 every 8 connection(s)
I20260923 22:12:49.043267 7997 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskRiQe_a/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790201539513534-1567-0/minicluster-data/ts-2/data/info.pb
I20260923 22:12:49.046460 1567 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskRiQe_a/build/release/bin/kudu as pid 7997
I20260923 22:12:49.046579 1567 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskRiQe_a/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790201539513534-1567-0/minicluster-data/ts-2/wal/instance
I20260923 22:12:49.046772 1567 external_mini_cluster.cc:1428] Setting key 514fb18304de54d27db5e0c7ec3c1979
Sep 23 22:12:49 dist-test-slave-s2fj krb5kdc[7622](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790201569, etypes {rep=17 tkt=17 ses=17}, oryx/127.1.135.195@KRBTEST.COM for oryx/127.1.135.254@KRBTEST.COM
I20260923 22:12:49.052388 7713 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 22:12:49.043929 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.135.195:38389 (local address 127.1.135.254:35949)
0923 22:12:49.043971 (+ 42us) server_negotiation.cc:206] Beginning negotiation
0923 22:12:49.043973 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0923 22:12:49.044207 (+ 234us) server_negotiation.cc:407] Connection header received
0923 22:12:49.044712 (+ 505us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 22:12:49.044713 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 22:12:49.044722 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 22:12:49.044771 (+ 49us) server_negotiation.cc:226] Negotiated authn=SASL
0923 22:12:49.045719 (+ 948us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 22:12:49.046347 (+ 628us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 22:12:49.047766 (+ 1419us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 22:12:49.047954 (+ 188us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 22:12:49.048007 (+ 53us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 22:12:49.050012 (+ 2005us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0923 22:12:49.050013 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0923 22:12:49.050014 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0923 22:12:49.050019 (+ 5us) server_negotiation.cc:922] calling sasl_server_start()
0923 22:12:49.051301 (+ 1282us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0923 22:12:49.051640 (+ 339us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0923 22:12:49.051641 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0923 22:12:49.051641 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0923 22:12:49.051686 (+ 45us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0923 22:12:49.051877 (+ 191us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0923 22:12:49.051878 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0923 22:12:49.051878 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0923 22:12:49.051965 (+ 87us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0923 22:12:49.052013 (+ 48us) server_negotiation.cc:1035] Waiting for connection context
0923 22:12:49.052267 (+ 254us) server_negotiation.cc:299] Negotiation successful
0923 22:12:49.052327 (+ 60us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":19}
I20260923 22:12:49.052573 8128 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 22:12:49.043904 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.135.254:35949 (local address 127.1.135.195:38389)
0923 22:12:49.044040 (+ 136us) negotiation.cc:107] Waiting for socket to connect
0923 22:12:49.044126 (+ 86us) client_negotiation.cc:174] Beginning negotiation
0923 22:12:49.044627 (+ 501us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0923 22:12:49.044824 (+ 197us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0923 22:12:49.044827 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0923 22:12:49.045439 (+ 612us) client_negotiation.cc:189] Negotiated authn=SASL
0923 22:12:49.045621 (+ 182us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 22:12:49.045626 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 22:12:49.046448 (+ 822us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 22:12:49.046449 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 22:12:49.047694 (+ 1245us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 22:12:49.047696 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 22:12:49.048019 (+ 323us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 22:12:49.048020 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 22:12:49.048161 (+ 141us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 22:12:49.048627 (+ 466us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0923 22:12:49.048635 (+ 8us) client_negotiation.cc:656] calling sasl_client_start()
0923 22:12:49.049938 (+ 1303us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0923 22:12:49.051366 (+ 1428us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0923 22:12:49.051371 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0923 22:12:49.051372 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0923 22:12:49.051568 (+ 196us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0923 22:12:49.051740 (+ 172us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0923 22:12:49.051741 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0923 22:12:49.051741 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0923 22:12:49.051828 (+ 87us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0923 22:12:49.052014 (+ 186us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0923 22:12:49.052017 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0923 22:12:49.052159 (+ 142us) client_negotiation.cc:769] Sending connection context
0923 22:12:49.052241 (+ 82us) client_negotiation.cc:240] Negotiation successful
0923 22:12:49.052349 (+ 108us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":105,"thread_start_us":80,"threads_started":1}
I20260923 22:12:49.053076 8126 heartbeater.cc:347] Connected to a master server at 127.1.135.254:35949
I20260923 22:12:49.053164 8126 heartbeater.cc:464] Registering TS with master...
I20260923 22:12:49.053315 8126 heartbeater.cc:511] Master 127.1.135.254:35949 requested a full tablet report, sending...
I20260923 22:12:49.053629 7667 ts_manager.cc:194] Registered new tserver with Master: 4d0b914e7ab746e7b17789e7b32aca0a (127.1.135.195:33563)
I20260923 22:12:49.053966 7667 master_service.cc:496] Signed X509 certificate for tserver {username='oryx', principal='oryx/127.1.135.195@KRBTEST.COM'} at 127.1.135.195:38389
I20260923 22:12:49.058193 1567 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
WARNING: no policy specified for oryx/127.1.135.253@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.1.135.253@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.1.135.253 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskRiQe_a/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790201539513534-1567-0/krb5kdc/oryx_127.1.135.253.keytab.
Entry for principal oryx/127.1.135.253 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskRiQe_a/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790201539513534-1567-0/krb5kdc/oryx_127.1.135.253.keytab.
I20260923 22:12:49.080854 1567 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskRiQe_a/build/release/bin/kudu
/tmp/dist-test-taskRiQe_a/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskRiQe_a/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790201539513534-1567-0/minicluster-data/master-1/wal
--fs_data_dirs=/tmp/dist-test-taskRiQe_a/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790201539513534-1567-0/minicluster-data/master-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskRiQe_a/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790201539513534-1567-0/minicluster-data/master-1/logs
--server_dump_info_path=/tmp/dist-test-taskRiQe_a/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790201539513534-1567-0/minicluster-data/master-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.1.135.253:39445
--webserver_interface=127.1.135.253
--webserver_port=0
--keytab_file=/tmp/dist-test-taskRiQe_a/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790201539513534-1567-0/krb5kdc/oryx_127.1.135.253.keytab
--principal=oryx/127.1.135.253
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.1.135.212:35751
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.1.135.254:35949,127.1.135.253:39445
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskRiQe_a/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790201539513534-1567-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskRiQe_a/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790201539513534-1567-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskRiQe_a/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790201539513534-1567-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260923 22:12:49.161317 8131 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260923 22:12:49.161515 8131 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260923 22:12:49.161535 8131 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260923 22:12:49.162554 8131 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260923 22:12:49.162585 8131 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260923 22:12:49.162598 8131 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260923 22:12:49.162611 8131 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260923 22:12:49.162621 8131 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260923 22:12:49.163862 8131 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.1.135.212:35751
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskRiQe_a/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790201539513534-1567-0/minicluster-data/master-1/data
--fs_wal_dir=/tmp/dist-test-taskRiQe_a/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790201539513534-1567-0/minicluster-data/master-1/wal
--ipki_ca_key_size=768
--master_addresses=127.1.135.254:35949,127.1.135.253:39445
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskRiQe_a/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790201539513534-1567-0/krb5kdc/oryx_127.1.135.253.keytab
--principal=oryx/127.1.135.253
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.1.135.253:39445
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskRiQe_a/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790201539513534-1567-0/minicluster-data/master-1/data/info.pb
--webserver_interface=127.1.135.253
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.8131
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskRiQe_a/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790201539513534-1567-0/minicluster-data/master-1/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 23 Sep 2026 20:53:37 UTC on ccycloud.quasar-rkviyx.root.comops.site
build id 4668083
W20260923 22:12:49.163949 8131 master_options.cc:55] Only 2 masters are specified by master_addresses_flag ('127.1.135.254:35949,127.1.135.253:39445'), but minimum of 3 are required to tolerate failures of any one master. It is recommended to use at least 3 masters.
I20260923 22:12:49.164044 8131 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260923 22:12:49.164211 8131 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260923 22:12:49.168793 8136 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260923 22:12:49.168758 8139 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260923 22:12:49.168759 8137 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260923 22:12:49.169039 8131 server_base.cc:1034] running on GCE node
I20260923 22:12:49.169308 8131 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260923 22:12:49.169616 8131 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260923 22:12:49.170845 8131 hybrid_clock.cc:648] HybridClock initialized: now 1790201569170830 us; error 32 us; skew 500 ppm
Sep 23 22:12:49 dist-test-slave-s2fj krb5kdc[7622](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790201569, etypes {rep=17 tkt=17 ses=17}, oryx/127.1.135.253@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260923 22:12:49.173368 8131 init.cc:377] Logged in from keytab as oryx/127.1.135.253@KRBTEST.COM (short username oryx)
I20260923 22:12:49.173785 8131 webserver.cc:460] Webserver started at http://127.1.135.253:41423/ using document root <none> and password file <none>
I20260923 22:12:49.173967 8131 fs_manager.cc:359] Metadata directory not provided
I20260923 22:12:49.174006 8131 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260923 22:12:49.174093 8131 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260923 22:12:49.175096 8131 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskRiQe_a/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790201539513534-1567-0/minicluster-data/master-1/data/instance:
uuid: "b32070aba06045cabad174bee345066a"
format_stamp: "Formatted at 2026-09-23 22:12:49 on dist-test-slave-s2fj"
server_key: "31fccec455e25eb0421b16b43800671f"
server_key_iv: "a6c45e16a98ad9d79fb71bda83d54e1f"
server_key_version: "encryptionkey@0"
I20260923 22:12:49.175379 8131 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskRiQe_a/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790201539513534-1567-0/minicluster-data/master-1/wal/instance:
uuid: "b32070aba06045cabad174bee345066a"
format_stamp: "Formatted at 2026-09-23 22:12:49 on dist-test-slave-s2fj"
server_key: "31fccec455e25eb0421b16b43800671f"
server_key_iv: "a6c45e16a98ad9d79fb71bda83d54e1f"
server_key_version: "encryptionkey@0"
I20260923 22:12:49.176793 8131 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.003s
I20260923 22:12:49.177618 8146 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260923 22:12:49.177811 8131 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.000s
I20260923 22:12:49.177910 8131 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskRiQe_a/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790201539513534-1567-0/minicluster-data/master-1/data,/tmp/dist-test-taskRiQe_a/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790201539513534-1567-0/minicluster-data/master-1/wal
uuid: "b32070aba06045cabad174bee345066a"
format_stamp: "Formatted at 2026-09-23 22:12:49 on dist-test-slave-s2fj"
server_key: "31fccec455e25eb0421b16b43800671f"
server_key_iv: "a6c45e16a98ad9d79fb71bda83d54e1f"
server_key_version: "encryptionkey@0"
I20260923 22:12:49.177989 8131 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskRiQe_a/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790201539513534-1567-0/minicluster-data/master-1/wal
metadata directory: /tmp/dist-test-taskRiQe_a/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790201539513534-1567-0/minicluster-data/master-1/wal
1 data directories: /tmp/dist-test-taskRiQe_a/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790201539513534-1567-0/minicluster-data/master-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260923 22:12:49.191202 8131 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260923 22:12:49.193413 8131 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260923 22:12:49.193595 8131 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260923 22:12:49.197435 8131 rpc_server.cc:307] RPC server started. Bound to: 127.1.135.253:39445
I20260923 22:12:49.197461 8198 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.1.135.253:39445 every 8 connection(s)
I20260923 22:12:49.197782 8131 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskRiQe_a/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790201539513534-1567-0/minicluster-data/master-1/data/info.pb
I20260923 22:12:49.198652 8199 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260923 22:12:49.199994 8199 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.1.135.254" port: 35949 } has no permanent_uuid. Determining permanent_uuid...
I20260923 22:12:49.205031 1567 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskRiQe_a/build/release/bin/kudu as pid 8131
Sep 23 22:12:49 dist-test-slave-s2fj krb5kdc[7622](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790201569, etypes {rep=17 tkt=17 ses=17}, oryx/127.1.135.253@KRBTEST.COM for oryx/127.1.135.254@KRBTEST.COM
I20260923 22:12:49.208544 7713 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 22:12:49.200689 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52032 (local address 127.1.135.254:35949)
0923 22:12:49.200853 (+ 164us) server_negotiation.cc:206] Beginning negotiation
0923 22:12:49.200855 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0923 22:12:49.200909 (+ 54us) server_negotiation.cc:407] Connection header received
0923 22:12:49.201345 (+ 436us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 22:12:49.201346 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 22:12:49.201355 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 22:12:49.201416 (+ 61us) server_negotiation.cc:226] Negotiated authn=SASL
0923 22:12:49.201943 (+ 527us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 22:12:49.202418 (+ 475us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 22:12:49.203340 (+ 922us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 22:12:49.203515 (+ 175us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 22:12:49.203562 (+ 47us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 22:12:49.205716 (+ 2154us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0923 22:12:49.205718 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0923 22:12:49.205719 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0923 22:12:49.205726 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0923 22:12:49.207166 (+ 1440us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0923 22:12:49.207573 (+ 407us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0923 22:12:49.207575 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0923 22:12:49.207575 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0923 22:12:49.207631 (+ 56us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0923 22:12:49.207878 (+ 247us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0923 22:12:49.207878 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0923 22:12:49.207879 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0923 22:12:49.208018 (+ 139us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0923 22:12:49.208117 (+ 99us) server_negotiation.cc:1035] Waiting for connection context
0923 22:12:49.208376 (+ 259us) server_negotiation.cc:299] Negotiation successful
0923 22:12:49.208458 (+ 82us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":139}
I20260923 22:12:49.208690 8201 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 22:12:49.200643 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.135.254:35949 (local address 127.0.0.1:52032)
0923 22:12:49.200793 (+ 150us) negotiation.cc:107] Waiting for socket to connect
0923 22:12:49.200832 (+ 39us) client_negotiation.cc:174] Beginning negotiation
0923 22:12:49.201266 (+ 434us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0923 22:12:49.201466 (+ 200us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0923 22:12:49.201470 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0923 22:12:49.201703 (+ 233us) client_negotiation.cc:189] Negotiated authn=SASL
0923 22:12:49.201877 (+ 174us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 22:12:49.201881 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 22:12:49.202489 (+ 608us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 22:12:49.202489 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 22:12:49.203270 (+ 781us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 22:12:49.203272 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 22:12:49.203586 (+ 314us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 22:12:49.203587 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 22:12:49.203680 (+ 93us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 22:12:49.204148 (+ 468us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0923 22:12:49.204162 (+ 14us) client_negotiation.cc:656] calling sasl_client_start()
0923 22:12:49.205617 (+ 1455us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0923 22:12:49.207267 (+ 1650us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0923 22:12:49.207274 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0923 22:12:49.207275 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0923 22:12:49.207482 (+ 207us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0923 22:12:49.207694 (+ 212us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0923 22:12:49.207694 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0923 22:12:49.207695 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0923 22:12:49.207805 (+ 110us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0923 22:12:49.208105 (+ 300us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0923 22:12:49.208110 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0923 22:12:49.208243 (+ 133us) client_negotiation.cc:769] Sending connection context
0923 22:12:49.208339 (+ 96us) client_negotiation.cc:240] Negotiation successful
0923 22:12:49.208479 (+ 140us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":112,"thread_start_us":83,"threads_started":1}
I20260923 22:12:49.209267 8199 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.1.135.253" port: 39445 } has no permanent_uuid. Determining permanent_uuid...
Sep 23 22:12:49 dist-test-slave-s2fj krb5kdc[7622](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790201568, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for oryx/127.1.135.253@KRBTEST.COM
Sep 23 22:12:49 dist-test-slave-s2fj krb5kdc[7622](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790201569, etypes {rep=17 tkt=17 ses=17}, oryx/127.1.135.253@KRBTEST.COM for oryx/127.1.135.253@KRBTEST.COM
I20260923 22:12:49.213934 8203 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 22:12:49.205476 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53466 (local address 127.1.135.253:39445)
0923 22:12:49.205625 (+ 149us) server_negotiation.cc:206] Beginning negotiation
0923 22:12:49.205629 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0923 22:12:49.205652 (+ 23us) server_negotiation.cc:407] Connection header received
0923 22:12:49.205677 (+ 25us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 22:12:49.205679 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 22:12:49.205691 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 22:12:49.205763 (+ 72us) server_negotiation.cc:226] Negotiated authn=SASL
0923 22:12:49.206423 (+ 660us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 22:12:49.207075 (+ 652us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 22:12:49.207798 (+ 723us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 22:12:49.208147 (+ 349us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 22:12:49.208218 (+ 71us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 22:12:49.210785 (+ 2567us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0923 22:12:49.210793 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0923 22:12:49.210793 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0923 22:12:49.210801 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0923 22:12:49.212600 (+ 1799us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0923 22:12:49.212942 (+ 342us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0923 22:12:49.212945 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0923 22:12:49.212945 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0923 22:12:49.213001 (+ 56us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0923 22:12:49.213192 (+ 191us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0923 22:12:49.213193 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0923 22:12:49.213193 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0923 22:12:49.213348 (+ 155us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0923 22:12:49.213485 (+ 137us) server_negotiation.cc:1035] Waiting for connection context
0923 22:12:49.213563 (+ 78us) server_negotiation.cc:299] Negotiation successful
0923 22:12:49.213645 (+ 82us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":118,"thread_start_us":77,"threads_started":1}
I20260923 22:12:49.216166 8201 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 22:12:49.209506 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.135.253:39445 (local address 127.0.0.1:53470)
0923 22:12:49.209676 (+ 170us) negotiation.cc:107] Waiting for socket to connect
0923 22:12:49.209684 (+ 8us) client_negotiation.cc:174] Beginning negotiation
0923 22:12:49.209717 (+ 33us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0923 22:12:49.209835 (+ 118us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0923 22:12:49.209836 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0923 22:12:49.209934 (+ 98us) client_negotiation.cc:189] Negotiated authn=SASL
0923 22:12:49.209975 (+ 41us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 22:12:49.209976 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 22:12:49.210642 (+ 666us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 22:12:49.210643 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 22:12:49.211400 (+ 757us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 22:12:49.211403 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 22:12:49.211869 (+ 466us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 22:12:49.211871 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 22:12:49.211945 (+ 74us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 22:12:49.212283 (+ 338us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0923 22:12:49.212287 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0923 22:12:49.213490 (+ 1203us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0923 22:12:49.215180 (+ 1690us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0923 22:12:49.215181 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0923 22:12:49.215182 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0923 22:12:49.215354 (+ 172us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0923 22:12:49.215580 (+ 226us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0923 22:12:49.215581 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0923 22:12:49.215581 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0923 22:12:49.215634 (+ 53us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0923 22:12:49.215852 (+ 218us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0923 22:12:49.215853 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0923 22:12:49.215887 (+ 34us) client_negotiation.cc:769] Sending connection context
0923 22:12:49.215949 (+ 62us) client_negotiation.cc:240] Negotiation successful
0923 22:12:49.216032 (+ 83us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":159}
I20260923 22:12:49.216248 8204 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 22:12:49.209573 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53470 (local address 127.1.135.253:39445)
0923 22:12:49.209714 (+ 141us) server_negotiation.cc:206] Beginning negotiation
0923 22:12:49.209716 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0923 22:12:49.209724 (+ 8us) server_negotiation.cc:407] Connection header received
0923 22:12:49.209748 (+ 24us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 22:12:49.209748 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 22:12:49.209758 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 22:12:49.209834 (+ 76us) server_negotiation.cc:226] Negotiated authn=SASL
0923 22:12:49.210043 (+ 209us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 22:12:49.210554 (+ 511us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 22:12:49.211510 (+ 956us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 22:12:49.211802 (+ 292us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 22:12:49.211852 (+ 50us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 22:12:49.213589 (+ 1737us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0923 22:12:49.213590 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0923 22:12:49.213590 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0923 22:12:49.213596 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0923 22:12:49.215087 (+ 1491us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0923 22:12:49.215430 (+ 343us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0923 22:12:49.215431 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0923 22:12:49.215431 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0923 22:12:49.215479 (+ 48us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0923 22:12:49.215691 (+ 212us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0923 22:12:49.215692 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0923 22:12:49.215693 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0923 22:12:49.215792 (+ 99us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0923 22:12:49.215860 (+ 68us) server_negotiation.cc:1035] Waiting for connection context
0923 22:12:49.215966 (+ 106us) server_negotiation.cc:299] Negotiation successful
0923 22:12:49.216106 (+ 140us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":57,"server-negotiator.queue_time_us":121,"thread_start_us":80,"threads_started":1}
I20260923 22:12:49.217430 8199 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P b32070aba06045cabad174bee345066a: Bootstrap starting.
I20260923 22:12:49.217989 8199 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P b32070aba06045cabad174bee345066a: Neither blocks nor log segments found. Creating new log.
I20260923 22:12:49.218250 8199 log.cc:824] T 00000000000000000000000000000000 P b32070aba06045cabad174bee345066a: Log is configured to *not* fsync() on all Append() calls
I20260923 22:12:49.218876 8199 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P b32070aba06045cabad174bee345066a: No bootstrap required, opened a new log
I20260923 22:12:49.220212 8199 raft_consensus.cc:348] T 00000000000000000000000000000000 P b32070aba06045cabad174bee345066a [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6378d422392b4a74831ae1c0f0692998" member_type: VOTER last_known_addr { host: "127.1.135.254" port: 35949 } } peers { permanent_uuid: "b32070aba06045cabad174bee345066a" member_type: VOTER last_known_addr { host: "127.1.135.253" port: 39445 } }
I20260923 22:12:49.220321 8199 raft_consensus.cc:374] T 00000000000000000000000000000000 P b32070aba06045cabad174bee345066a [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260923 22:12:49.220359 8199 raft_consensus.cc:729] T 00000000000000000000000000000000 P b32070aba06045cabad174bee345066a [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b32070aba06045cabad174bee345066a, State: Initialized, Role: FOLLOWER
I20260923 22:12:49.220472 8199 consensus_queue.cc:260] T 00000000000000000000000000000000 P b32070aba06045cabad174bee345066a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6378d422392b4a74831ae1c0f0692998" member_type: VOTER last_known_addr { host: "127.1.135.254" port: 35949 } } peers { permanent_uuid: "b32070aba06045cabad174bee345066a" member_type: VOTER last_known_addr { host: "127.1.135.253" port: 39445 } }
I20260923 22:12:49.220688 8206 sys_catalog.cc:455] T 00000000000000000000000000000000 P b32070aba06045cabad174bee345066a [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6378d422392b4a74831ae1c0f0692998" member_type: VOTER last_known_addr { host: "127.1.135.254" port: 35949 } } peers { permanent_uuid: "b32070aba06045cabad174bee345066a" member_type: VOTER last_known_addr { host: "127.1.135.253" port: 39445 } } }
I20260923 22:12:49.220724 8199 sys_catalog.cc:564] T 00000000000000000000000000000000 P b32070aba06045cabad174bee345066a [sys.catalog]: configured and running, proceeding with master startup.
I20260923 22:12:49.220768 8206 sys_catalog.cc:458] T 00000000000000000000000000000000 P b32070aba06045cabad174bee345066a [sys.catalog]: This master's current role is: FOLLOWER
W20260923 22:12:49.222281 8217 catalog_manager.cc:1558] T 00000000000000000000000000000000 P b32070aba06045cabad174bee345066a: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260923 22:12:49.222334 8217 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260923 22:12:49.223727 8131 master_runner.cc:428] Detected that this master b32070aba06045cabad174bee345066a is joining an existing cluster
I20260923 22:12:49.223781 8131 master_runner.cc:432] Initiating AddMaster RPC to add 127.1.135.253:39445
I20260923 22:12:49.230875 8224 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 22:12:49.224791 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.135.253:39445 (local address 127.0.0.1:53474)
0923 22:12:49.224905 (+ 114us) negotiation.cc:107] Waiting for socket to connect
0923 22:12:49.224915 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0923 22:12:49.224996 (+ 81us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0923 22:12:49.225164 (+ 168us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0923 22:12:49.225165 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0923 22:12:49.225340 (+ 175us) client_negotiation.cc:189] Negotiated authn=SASL
0923 22:12:49.225391 (+ 51us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 22:12:49.225392 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 22:12:49.225991 (+ 599us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 22:12:49.225991 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 22:12:49.226778 (+ 787us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 22:12:49.226780 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 22:12:49.227253 (+ 473us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 22:12:49.227254 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 22:12:49.227288 (+ 34us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 22:12:49.227717 (+ 429us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0923 22:12:49.227721 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0923 22:12:49.228204 (+ 483us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0923 22:12:49.229870 (+ 1666us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0923 22:12:49.229870 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0923 22:12:49.229871 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0923 22:12:49.230032 (+ 161us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0923 22:12:49.230229 (+ 197us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0923 22:12:49.230230 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0923 22:12:49.230231 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0923 22:12:49.230282 (+ 51us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0923 22:12:49.230551 (+ 269us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0923 22:12:49.230552 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0923 22:12:49.230594 (+ 42us) client_negotiation.cc:769] Sending connection context
0923 22:12:49.230664 (+ 70us) client_negotiation.cc:240] Negotiation successful
0923 22:12:49.230760 (+ 96us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":99,"thread_start_us":60,"threads_started":1}
I20260923 22:12:49.230923 8204 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 22:12:49.224884 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53474 (local address 127.1.135.253:39445)
0923 22:12:49.224914 (+ 30us) server_negotiation.cc:206] Beginning negotiation
0923 22:12:49.224915 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0923 22:12:49.224959 (+ 44us) server_negotiation.cc:407] Connection header received
0923 22:12:49.225077 (+ 118us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 22:12:49.225078 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 22:12:49.225087 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 22:12:49.225114 (+ 27us) server_negotiation.cc:226] Negotiated authn=SASL
0923 22:12:49.225442 (+ 328us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 22:12:49.225915 (+ 473us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 22:12:49.226861 (+ 946us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 22:12:49.227168 (+ 307us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 22:12:49.227196 (+ 28us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 22:12:49.228285 (+ 1089us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0923 22:12:49.228286 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0923 22:12:49.228286 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0923 22:12:49.228292 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0923 22:12:49.229773 (+ 1481us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0923 22:12:49.230115 (+ 342us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0923 22:12:49.230116 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0923 22:12:49.230117 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0923 22:12:49.230163 (+ 46us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0923 22:12:49.230347 (+ 184us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0923 22:12:49.230348 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0923 22:12:49.230348 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0923 22:12:49.230473 (+ 125us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0923 22:12:49.230531 (+ 58us) server_negotiation.cc:1035] Waiting for connection context
0923 22:12:49.230684 (+ 153us) server_negotiation.cc:299] Negotiation successful
0923 22:12:49.230809 (+ 125us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":65,"server-negotiator.queue_time_us":16}
I20260923 22:12:49.231240 7713 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 22:12:49.224660 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52044 (local address 127.1.135.254:35949)
0923 22:12:49.224712 (+ 52us) server_negotiation.cc:206] Beginning negotiation
0923 22:12:49.224714 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0923 22:12:49.224857 (+ 143us) server_negotiation.cc:407] Connection header received
0923 22:12:49.224959 (+ 102us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 22:12:49.224961 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 22:12:49.224972 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 22:12:49.225030 (+ 58us) server_negotiation.cc:226] Negotiated authn=SASL
0923 22:12:49.225405 (+ 375us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 22:12:49.226158 (+ 753us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 22:12:49.227103 (+ 945us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 22:12:49.227397 (+ 294us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 22:12:49.227474 (+ 77us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 22:12:49.228355 (+ 881us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0923 22:12:49.228356 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0923 22:12:49.228357 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0923 22:12:49.228364 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0923 22:12:49.230045 (+ 1681us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0923 22:12:49.230399 (+ 354us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0923 22:12:49.230401 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0923 22:12:49.230401 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0923 22:12:49.230455 (+ 54us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0923 22:12:49.230690 (+ 235us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0923 22:12:49.230691 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0923 22:12:49.230691 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0923 22:12:49.230814 (+ 123us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0923 22:12:49.230857 (+ 43us) server_negotiation.cc:1035] Waiting for connection context
0923 22:12:49.231056 (+ 199us) server_negotiation.cc:299] Negotiation successful
0923 22:12:49.231132 (+ 76us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":33}
I20260923 22:12:49.231281 8223 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 22:12:49.224608 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.135.254:35949 (local address 127.0.0.1:52044)
0923 22:12:49.224796 (+ 188us) negotiation.cc:107] Waiting for socket to connect
0923 22:12:49.224806 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0923 22:12:49.224867 (+ 61us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0923 22:12:49.225043 (+ 176us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0923 22:12:49.225044 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0923 22:12:49.225247 (+ 203us) client_negotiation.cc:189] Negotiated authn=SASL
0923 22:12:49.225326 (+ 79us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 22:12:49.225328 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 22:12:49.226239 (+ 911us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 22:12:49.226240 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 22:12:49.227030 (+ 790us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 22:12:49.227031 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 22:12:49.227475 (+ 444us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 22:12:49.227476 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 22:12:49.227536 (+ 60us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 22:12:49.227883 (+ 347us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0923 22:12:49.227887 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0923 22:12:49.228285 (+ 398us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0923 22:12:49.230137 (+ 1852us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0923 22:12:49.230138 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0923 22:12:49.230138 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0923 22:12:49.230338 (+ 200us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0923 22:12:49.230522 (+ 184us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0923 22:12:49.230523 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0923 22:12:49.230523 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0923 22:12:49.230603 (+ 80us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0923 22:12:49.230904 (+ 301us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0923 22:12:49.230905 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0923 22:12:49.230952 (+ 47us) client_negotiation.cc:769] Sending connection context
0923 22:12:49.231018 (+ 66us) client_negotiation.cc:240] Negotiation successful
0923 22:12:49.231108 (+ 90us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":156,"thread_start_us":75,"threads_started":1}
Sep 23 22:12:49 dist-test-slave-s2fj krb5kdc[7622](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790201568, etypes {rep=17 tkt=17 ses=17}, oryx/127.1.135.254@KRBTEST.COM for oryx/127.1.135.253@KRBTEST.COM
I20260923 22:12:49.240443 8204 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 22:12:49.233470 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53476 (local address 127.1.135.253:39445)
0923 22:12:49.233508 (+ 38us) server_negotiation.cc:206] Beginning negotiation
0923 22:12:49.233509 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0923 22:12:49.233669 (+ 160us) server_negotiation.cc:407] Connection header received
0923 22:12:49.233722 (+ 53us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 22:12:49.233723 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 22:12:49.233732 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 22:12:49.233797 (+ 65us) server_negotiation.cc:226] Negotiated authn=SASL
0923 22:12:49.234132 (+ 335us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 22:12:49.234838 (+ 706us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 22:12:49.235518 (+ 680us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 22:12:49.235813 (+ 295us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 22:12:49.235879 (+ 66us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 22:12:49.237943 (+ 2064us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0923 22:12:49.237944 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0923 22:12:49.237944 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0923 22:12:49.237949 (+ 5us) server_negotiation.cc:922] calling sasl_server_start()
0923 22:12:49.239215 (+ 1266us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0923 22:12:49.239661 (+ 446us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0923 22:12:49.239661 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0923 22:12:49.239662 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0923 22:12:49.239750 (+ 88us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0923 22:12:49.239934 (+ 184us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0923 22:12:49.239935 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0923 22:12:49.239935 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0923 22:12:49.240020 (+ 85us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0923 22:12:49.240139 (+ 119us) server_negotiation.cc:1035] Waiting for connection context
0923 22:12:49.240288 (+ 149us) server_negotiation.cc:299] Negotiation successful
0923 22:12:49.240360 (+ 72us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":20}
I20260923 22:12:49.240749 8225 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 22:12:49.233431 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.135.253:39445 (local address 127.0.0.1:53476)
0923 22:12:49.233568 (+ 137us) negotiation.cc:107] Waiting for socket to connect
0923 22:12:49.233605 (+ 37us) client_negotiation.cc:174] Beginning negotiation
0923 22:12:49.233669 (+ 64us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0923 22:12:49.233800 (+ 131us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0923 22:12:49.233801 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0923 22:12:49.234027 (+ 226us) client_negotiation.cc:189] Negotiated authn=SASL
0923 22:12:49.234069 (+ 42us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 22:12:49.234071 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 22:12:49.234910 (+ 839us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 22:12:49.234910 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 22:12:49.235448 (+ 538us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 22:12:49.235449 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 22:12:49.235886 (+ 437us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 22:12:49.235887 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 22:12:49.235939 (+ 52us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 22:12:49.236381 (+ 442us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0923 22:12:49.236386 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0923 22:12:49.237867 (+ 1481us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0923 22:12:49.239286 (+ 1419us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0923 22:12:49.239292 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0923 22:12:49.239294 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0923 22:12:49.239585 (+ 291us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0923 22:12:49.239810 (+ 225us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0923 22:12:49.239811 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0923 22:12:49.239812 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0923 22:12:49.239875 (+ 63us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0923 22:12:49.240094 (+ 219us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0923 22:12:49.240101 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0923 22:12:49.240151 (+ 50us) client_negotiation.cc:769] Sending connection context
0923 22:12:49.240516 (+ 365us) client_negotiation.cc:240] Negotiation successful
0923 22:12:49.240640 (+ 124us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":107,"thread_start_us":58,"threads_started":1}
I20260923 22:12:49.241279 7667 catalog_manager.cc:6979] Initiating ChangeConfig request to add master 127.1.135.253:39445: tablet_id: "00000000000000000000000000000000"
type: ADD_PEER
server {
permanent_uuid: "b32070aba06045cabad174bee345066a"
member_type: NON_VOTER
last_known_addr {
host: "127.1.135.253"
port: 39445
}
attrs {
promote: true
}
}
dest_uuid: "6378d422392b4a74831ae1c0f0692998"
cas_config_opid_index: -1
I20260923 22:12:49.241587 7667 consensus_queue.cc:237] T 00000000000000000000000000000000 P 6378d422392b4a74831ae1c0f0692998 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: 5 OBSOLETE_local: true peers { permanent_uuid: "6378d422392b4a74831ae1c0f0692998" member_type: VOTER last_known_addr { host: "127.1.135.254" port: 35949 } } peers { permanent_uuid: "b32070aba06045cabad174bee345066a" member_type: NON_VOTER last_known_addr { host: "127.1.135.253" port: 39445 } attrs { promote: true } }
I20260923 22:12:49.242033 8227 raft_consensus.cc:2932] T 00000000000000000000000000000000 P 6378d422392b4a74831ae1c0f0692998 [term 1 LEADER]: Committing config change with OpId 1.5: config changed from index -1 to 5, NON_VOTER b32070aba06045cabad174bee345066a (127.1.135.253) added. New config: { opid_index: 5 OBSOLETE_local: true peers { permanent_uuid: "6378d422392b4a74831ae1c0f0692998" member_type: VOTER last_known_addr { host: "127.1.135.254" port: 35949 } } peers { permanent_uuid: "b32070aba06045cabad174bee345066a" member_type: NON_VOTER last_known_addr { host: "127.1.135.253" port: 39445 } attrs { promote: true } } }
I20260923 22:12:49.242245 8173 raft_consensus.cc:3037] T 00000000000000000000000000000000 P b32070aba06045cabad174bee345066a [term 0 FOLLOWER]: Advancing to term 1
I20260923 22:12:49.242918 8173 raft_consensus.cc:1261] T 00000000000000000000000000000000 P b32070aba06045cabad174bee345066a [term 1 FOLLOWER]: Refusing update from remote peer 6378d422392b4a74831ae1c0f0692998: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 5. (index mismatch)
I20260923 22:12:49.242959 8227 catalog_manager.cc:6983] Successfully completed master ChangeConfig request to add master 127.1.135.253:39445
I20260923 22:12:49.243031 8227 sys_catalog.cc:455] T 00000000000000000000000000000000 P 6378d422392b4a74831ae1c0f0692998 [sys.catalog]: SysCatalogTable state changed. Reason: Config change replication complete. Latest consensus state: current_term: 1 leader_uuid: "6378d422392b4a74831ae1c0f0692998" committed_config { opid_index: 5 OBSOLETE_local: true peers { permanent_uuid: "6378d422392b4a74831ae1c0f0692998" member_type: VOTER last_known_addr { host: "127.1.135.254" port: 35949 } } peers { permanent_uuid: "b32070aba06045cabad174bee345066a" member_type: NON_VOTER last_known_addr { host: "127.1.135.253" port: 39445 } attrs { promote: true } } }
I20260923 22:12:49.243088 8227 sys_catalog.cc:458] T 00000000000000000000000000000000 P 6378d422392b4a74831ae1c0f0692998 [sys.catalog]: This master's current role is: LEADER
I20260923 22:12:49.243206 8131 master.cc:537] Master@127.1.135.253:39445 shutting down...
I20260923 22:12:49.243237 8227 consensus_queue.cc:1035] T 00000000000000000000000000000000 P 6378d422392b4a74831ae1c0f0692998 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b32070aba06045cabad174bee345066a" member_type: NON_VOTER last_known_addr { host: "127.1.135.253" port: 39445 } attrs { promote: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 0, Time since last communication: 0.000s
W20260923 22:12:49.243913 7652 proxy.cc:239] Call had error, refreshing address and retrying: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on Master
W20260923 22:12:49.244426 7652 consensus_peers.cc:487] T 00000000000000000000000000000000 P 6378d422392b4a74831ae1c0f0692998 -> Peer b32070aba06045cabad174bee345066a (127.1.135.253:39445): Couldn't send request to peer b32070aba06045cabad174bee345066a. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on Master. This is attempt 1: this message will repeat every 5th retry.
I20260923 22:12:49.246385 8131 raft_consensus.cc:2229] T 00000000000000000000000000000000 P b32070aba06045cabad174bee345066a [term 1 FOLLOWER]: Raft consensus shutting down.
I20260923 22:12:49.246464 8131 raft_consensus.cc:2258] T 00000000000000000000000000000000 P b32070aba06045cabad174bee345066a [term 1 FOLLOWER]: Raft consensus is shut down!
I20260923 22:12:49.246526 8131 tablet_replica.cc:330] T 00000000000000000000000000000000 P b32070aba06045cabad174bee345066a: stopping tablet replica
I20260923 22:12:49.258917 8131 master.cc:552] Master@127.1.135.253:39445 shutdown complete.
I20260923 22:12:49.258963 8131 master_runner.cc:305] Clearing existing system tablet
I20260923 22:12:49.259087 8131 fs_manager.cc:359] Metadata directory not provided
I20260923 22:12:49.259135 8131 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260923 22:12:49.259810 8131 fs_manager.cc:709] Time spent opening directory manager: real 0.000s user 0.000s sys 0.001s
I20260923 22:12:49.260308 8231 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260923 22:12:49.260432 8131 fs_report.cc:383] FS layout report
--------------------
wal directory:
metadata directory:
1 data directories: /tmp/dist-test-taskRiQe_a/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790201539513534-1567-0/minicluster-data/master-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260923 22:12:49.260490 8131 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.000s
I20260923 22:12:49.260514 8131 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskRiQe_a/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790201539513534-1567-0/minicluster-data/master-1/data,/tmp/dist-test-taskRiQe_a/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790201539513534-1567-0/minicluster-data/master-1/wal
uuid: "b32070aba06045cabad174bee345066a"
format_stamp: "Formatted at 2026-09-23 22:12:49 on dist-test-slave-s2fj"
server_key: "31fccec455e25eb0421b16b43800671f"
server_key_iv: "a6c45e16a98ad9d79fb71bda83d54e1f"
server_key_version: "encryptionkey@0"
I20260923 22:12:49.261221 8131 ts_tablet_manager.cc:1905] T 00000000000000000000000000000000 P b32070aba06045cabad174bee345066a: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260923 22:12:49.262656 8131 ts_tablet_manager.cc:1918] T 00000000000000000000000000000000 P b32070aba06045cabad174bee345066a: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId unknown
I20260923 22:12:49.262727 8131 log.cc:1196] T 00000000000000000000000000000000 P b32070aba06045cabad174bee345066a: Deleting WAL directory at /tmp/dist-test-taskRiQe_a/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790201539513534-1567-0/minicluster-data/master-1/wal/wals/00000000000000000000000000000000
I20260923 22:12:49.262916 8131 ts_tablet_manager.cc:1939] T 00000000000000000000000000000000 P b32070aba06045cabad174bee345066a: Deleting consensus metadata
I20260923 22:12:49.263033 8131 master_runner.cc:315] Copying system tablet from 127.1.135.254:35949
I20260923 22:12:49.263629 8131 tablet_copy_client.cc:316] T 00000000000000000000000000000000 P b32070aba06045cabad174bee345066a: tablet copy: Beginning tablet copy session from remote peer at address 127.1.135.254:35949
I20260923 22:12:49.269918 8237 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 22:12:49.263891 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.135.254:35949 (local address 127.0.0.1:52048)
0923 22:12:49.264040 (+ 149us) negotiation.cc:107] Waiting for socket to connect
0923 22:12:49.264048 (+ 8us) client_negotiation.cc:174] Beginning negotiation
0923 22:12:49.264136 (+ 88us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0923 22:12:49.264261 (+ 125us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0923 22:12:49.264262 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0923 22:12:49.264419 (+ 157us) client_negotiation.cc:189] Negotiated authn=SASL
0923 22:12:49.264736 (+ 317us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 22:12:49.264739 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 22:12:49.265654 (+ 915us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 22:12:49.265655 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 22:12:49.266190 (+ 535us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 22:12:49.266191 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 22:12:49.266648 (+ 457us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 22:12:49.266649 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 22:12:49.266693 (+ 44us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 22:12:49.267158 (+ 465us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0923 22:12:49.267162 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0923 22:12:49.267432 (+ 270us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0923 22:12:49.269060 (+ 1628us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0923 22:12:49.269061 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0923 22:12:49.269061 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0923 22:12:49.269216 (+ 155us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0923 22:12:49.269404 (+ 188us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0923 22:12:49.269404 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0923 22:12:49.269404 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0923 22:12:49.269451 (+ 47us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0923 22:12:49.269702 (+ 251us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0923 22:12:49.269703 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0923 22:12:49.269734 (+ 31us) client_negotiation.cc:769] Sending connection context
0923 22:12:49.269783 (+ 49us) client_negotiation.cc:240] Negotiation successful
0923 22:12:49.269848 (+ 65us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":128,"thread_start_us":74,"threads_started":1}
I20260923 22:12:49.269960 7713 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 22:12:49.263944 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52048 (local address 127.1.135.254:35949)
0923 22:12:49.263995 (+ 51us) server_negotiation.cc:206] Beginning negotiation
0923 22:12:49.263997 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0923 22:12:49.264122 (+ 125us) server_negotiation.cc:407] Connection header received
0923 22:12:49.264196 (+ 74us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 22:12:49.264197 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 22:12:49.264205 (+ 8us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 22:12:49.264249 (+ 44us) server_negotiation.cc:226] Negotiated authn=SASL
0923 22:12:49.264815 (+ 566us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 22:12:49.265569 (+ 754us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 22:12:49.266264 (+ 695us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 22:12:49.266570 (+ 306us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 22:12:49.266645 (+ 75us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 22:12:49.267509 (+ 864us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0923 22:12:49.267510 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0923 22:12:49.267510 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0923 22:12:49.267516 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0923 22:12:49.268989 (+ 1473us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0923 22:12:49.269283 (+ 294us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0923 22:12:49.269284 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0923 22:12:49.269285 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0923 22:12:49.269341 (+ 56us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0923 22:12:49.269505 (+ 164us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0923 22:12:49.269506 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0923 22:12:49.269506 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0923 22:12:49.269639 (+ 133us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0923 22:12:49.269708 (+ 69us) server_negotiation.cc:1035] Waiting for connection context
0923 22:12:49.269805 (+ 97us) server_negotiation.cc:299] Negotiation successful
0923 22:12:49.269874 (+ 69us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":26}
I20260923 22:12:49.270476 7687 tablet_copy_service.cc:140] P 6378d422392b4a74831ae1c0f0692998: Received BeginTabletCopySession request for tablet 00000000000000000000000000000000 from peer b32070aba06045cabad174bee345066a ({username='oryx', principal='oryx/127.1.135.253@KRBTEST.COM'} at 127.0.0.1:52048)
I20260923 22:12:49.270576 7687 tablet_copy_service.cc:161] P 6378d422392b4a74831ae1c0f0692998: Beginning new tablet copy session on tablet 00000000000000000000000000000000 from peer b32070aba06045cabad174bee345066a at {username='oryx', principal='oryx/127.1.135.253@KRBTEST.COM'} at 127.0.0.1:52048: session id = b32070aba06045cabad174bee345066a-00000000000000000000000000000000
I20260923 22:12:49.271162 7687 tablet_copy_source_session.cc:205] T 00000000000000000000000000000000 P 6378d422392b4a74831ae1c0f0692998: Tablet Copy: opened 0 blocks and 1 log segments
I20260923 22:12:49.271703 8131 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total
I20260923 22:12:49.272818 8131 tablet_copy_client.cc:791] T 00000000000000000000000000000000 P b32070aba06045cabad174bee345066a: tablet copy: Starting download of 0 data blocks...
I20260923 22:12:49.272967 8131 tablet_copy_client.cc:655] T 00000000000000000000000000000000 P b32070aba06045cabad174bee345066a: tablet copy: Starting download of 1 WAL segments...
I20260923 22:12:49.273813 8131 tablet_copy_client.cc:531] T 00000000000000000000000000000000 P b32070aba06045cabad174bee345066a: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20260923 22:12:49.274618 7687 tablet_copy_service.cc:342] P 6378d422392b4a74831ae1c0f0692998: Request end of tablet copy session b32070aba06045cabad174bee345066a-00000000000000000000000000000000 received from {username='oryx', principal='oryx/127.1.135.253@KRBTEST.COM'} at 127.0.0.1:52048
I20260923 22:12:49.274677 7687 tablet_copy_service.cc:433] P 6378d422392b4a74831ae1c0f0692998: ending tablet copy session b32070aba06045cabad174bee345066a-00000000000000000000000000000000 on tablet 00000000000000000000000000000000 with peer b32070aba06045cabad174bee345066a
I20260923 22:12:49.277242 8131 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260923 22:12:49.278318 8239 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260923 22:12:49.278400 8241 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260923 22:12:49.278651 8131 server_base.cc:1034] running on GCE node
W20260923 22:12:49.280198 8238 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260923 22:12:49.280328 8131 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260923 22:12:49.280471 8131 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260923 22:12:49.281591 8131 hybrid_clock.cc:648] HybridClock initialized: now 1790201569281580 us; error 29 us; skew 500 ppm
Sep 23 22:12:49 dist-test-slave-s2fj krb5kdc[7622](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790201569, etypes {rep=17 tkt=17 ses=17}, oryx/127.1.135.253@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260923 22:12:49.282951 8131 init.cc:377] Logged in from keytab as oryx/127.1.135.253@KRBTEST.COM (short username oryx)
I20260923 22:12:49.283226 8131 webserver.cc:460] Webserver started at http://127.1.135.253:46431/ using document root <none> and password file <none>
I20260923 22:12:49.283340 8131 fs_manager.cc:359] Metadata directory not provided
I20260923 22:12:49.283389 8131 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260923 22:12:49.283962 8131 fs_manager.cc:709] Time spent opening directory manager: real 0.000s user 0.001s sys 0.000s
I20260923 22:12:49.284475 8248 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260923 22:12:49.284618 8131 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.000s
I20260923 22:12:49.284672 8131 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskRiQe_a/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790201539513534-1567-0/minicluster-data/master-1/data,/tmp/dist-test-taskRiQe_a/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790201539513534-1567-0/minicluster-data/master-1/wal
uuid: "b32070aba06045cabad174bee345066a"
format_stamp: "Formatted at 2026-09-23 22:12:49 on dist-test-slave-s2fj"
server_key: "31fccec455e25eb0421b16b43800671f"
server_key_iv: "a6c45e16a98ad9d79fb71bda83d54e1f"
server_key_version: "encryptionkey@0"
I20260923 22:12:49.284723 8131 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskRiQe_a/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790201539513534-1567-0/minicluster-data/master-1/wal
metadata directory: /tmp/dist-test-taskRiQe_a/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790201539513534-1567-0/minicluster-data/master-1/wal
1 data directories: /tmp/dist-test-taskRiQe_a/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790201539513534-1567-0/minicluster-data/master-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260923 22:12:49.292023 8131 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260923 22:12:49.293474 8131 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260923 22:12:49.296332 8131 rpc_server.cc:307] RPC server started. Bound to: 127.1.135.253:39445
I20260923 22:12:49.296707 8300 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.1.135.253:39445 every 8 connection(s)
I20260923 22:12:49.296731 8131 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskRiQe_a/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790201539513534-1567-0/minicluster-data/master-1/data/info.pb
I20260923 22:12:49.297446 8301 sys_catalog.cc:263] Verifying existing consensus state
I20260923 22:12:49.297812 8301 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P b32070aba06045cabad174bee345066a: Bootstrap starting.
I20260923 22:12:49.299870 8301 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P b32070aba06045cabad174bee345066a: Bootstrap replayed 1/1 log segments. Stats: ops{read=5 overwritten=0 applied=5 ignored=0} inserts{seen=3 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260923 22:12:49.300156 8301 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P b32070aba06045cabad174bee345066a: Bootstrap complete.
I20260923 22:12:49.300333 8301 raft_consensus.cc:348] T 00000000000000000000000000000000 P b32070aba06045cabad174bee345066a [term 1 LEARNER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 5 OBSOLETE_local: true peers { permanent_uuid: "6378d422392b4a74831ae1c0f0692998" member_type: VOTER last_known_addr { host: "127.1.135.254" port: 35949 } } peers { permanent_uuid: "b32070aba06045cabad174bee345066a" member_type: NON_VOTER last_known_addr { host: "127.1.135.253" port: 39445 } attrs { promote: true } }
I20260923 22:12:49.300410 8301 raft_consensus.cc:729] T 00000000000000000000000000000000 P b32070aba06045cabad174bee345066a [term 1 LEARNER]: Becoming Follower/Learner. State: Replica: b32070aba06045cabad174bee345066a, State: Initialized, Role: LEARNER
I20260923 22:12:49.300455 8301 consensus_queue.cc:260] T 00000000000000000000000000000000 P b32070aba06045cabad174bee345066a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 5, Last appended: 1.5, Last appended by leader: 5, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 5 OBSOLETE_local: true peers { permanent_uuid: "6378d422392b4a74831ae1c0f0692998" member_type: VOTER last_known_addr { host: "127.1.135.254" port: 35949 } } peers { permanent_uuid: "b32070aba06045cabad174bee345066a" member_type: NON_VOTER last_known_addr { host: "127.1.135.253" port: 39445 } attrs { promote: true } }
I20260923 22:12:49.300644 8301 sys_catalog.cc:564] T 00000000000000000000000000000000 P b32070aba06045cabad174bee345066a [sys.catalog]: configured and running, proceeding with master startup.
I20260923 22:12:49.300614 8304 sys_catalog.cc:455] T 00000000000000000000000000000000 P b32070aba06045cabad174bee345066a [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 committed_config { opid_index: 5 OBSOLETE_local: true peers { permanent_uuid: "6378d422392b4a74831ae1c0f0692998" member_type: VOTER last_known_addr { host: "127.1.135.254" port: 35949 } } peers { permanent_uuid: "b32070aba06045cabad174bee345066a" member_type: NON_VOTER last_known_addr { host: "127.1.135.253" port: 39445 } attrs { promote: true } } }
I20260923 22:12:49.300700 8304 sys_catalog.cc:458] T 00000000000000000000000000000000 P b32070aba06045cabad174bee345066a [sys.catalog]: This master's current role is: LEARNER
I20260923 22:12:49.302390 8315 catalog_manager.cc:1259] Loaded cluster ID: e03b1c9cbcbb49658f49f0be6e6c5546
I20260923 22:12:49.302476 8315 catalog_manager.cc:1552] T 00000000000000000000000000000000 P b32070aba06045cabad174bee345066a: loading cluster ID for follower catalog manager: success
I20260923 22:12:49.303617 8315 catalog_manager.cc:1574] T 00000000000000000000000000000000 P b32070aba06045cabad174bee345066a: acquiring CA information for follower catalog manager: success
I20260923 22:12:49.303968 8315 catalog_manager.cc:1602] T 00000000000000000000000000000000 P b32070aba06045cabad174bee345066a: importing token verification keys for follower catalog manager: success; most recent TSK sequence number 0
I20260923 22:12:49.321925 8316 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 22:12:49.315393 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53480 (local address 127.1.135.253:39445)
0923 22:12:49.315503 (+ 110us) server_negotiation.cc:206] Beginning negotiation
0923 22:12:49.315505 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0923 22:12:49.315511 (+ 6us) server_negotiation.cc:407] Connection header received
0923 22:12:49.315531 (+ 20us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 22:12:49.315532 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 22:12:49.315541 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 22:12:49.315605 (+ 64us) server_negotiation.cc:226] Negotiated authn=SASL
0923 22:12:49.316242 (+ 637us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 22:12:49.316996 (+ 754us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 22:12:49.317706 (+ 710us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 22:12:49.317933 (+ 227us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 22:12:49.318016 (+ 83us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 22:12:49.319273 (+ 1257us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0923 22:12:49.319275 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0923 22:12:49.319275 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0923 22:12:49.319282 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0923 22:12:49.320776 (+ 1494us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0923 22:12:49.321164 (+ 388us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0923 22:12:49.321165 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0923 22:12:49.321166 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0923 22:12:49.321221 (+ 55us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0923 22:12:49.321409 (+ 188us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0923 22:12:49.321410 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0923 22:12:49.321410 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0923 22:12:49.321513 (+ 103us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0923 22:12:49.321624 (+ 111us) server_negotiation.cc:1035] Waiting for connection context
0923 22:12:49.321714 (+ 90us) server_negotiation.cc:299] Negotiation successful
0923 22:12:49.321794 (+ 80us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":87,"thread_start_us":52,"threads_started":1}
WARNING: no policy specified for oryx/127.1.135.252@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.1.135.252@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.1.135.252 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskRiQe_a/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790201539513534-1567-0/krb5kdc/oryx_127.1.135.252.keytab.
Entry for principal oryx/127.1.135.252 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskRiQe_a/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790201539513534-1567-0/krb5kdc/oryx_127.1.135.252.keytab.
I20260923 22:12:49.350270 1567 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskRiQe_a/build/release/bin/kudu
/tmp/dist-test-taskRiQe_a/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskRiQe_a/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790201539513534-1567-0/minicluster-data/master-2/wal
--fs_data_dirs=/tmp/dist-test-taskRiQe_a/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790201539513534-1567-0/minicluster-data/master-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskRiQe_a/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790201539513534-1567-0/minicluster-data/master-2/logs
--server_dump_info_path=/tmp/dist-test-taskRiQe_a/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790201539513534-1567-0/minicluster-data/master-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.1.135.252:42799
--webserver_interface=127.1.135.252
--webserver_port=0
--keytab_file=/tmp/dist-test-taskRiQe_a/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790201539513534-1567-0/krb5kdc/oryx_127.1.135.252.keytab
--principal=oryx/127.1.135.252
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.1.135.212:35751
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.1.135.254:35949,127.1.135.253:39445,127.1.135.252:42799
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskRiQe_a/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790201539513534-1567-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskRiQe_a/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790201539513534-1567-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskRiQe_a/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790201539513534-1567-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260923 22:12:49.441391 8319 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260923 22:12:49.441601 8319 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260923 22:12:49.441630 8319 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260923 22:12:49.442736 8319 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260923 22:12:49.442767 8319 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260923 22:12:49.442781 8319 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260923 22:12:49.442794 8319 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260923 22:12:49.442806 8319 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260923 22:12:49.444284 8319 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.1.135.212:35751
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskRiQe_a/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790201539513534-1567-0/minicluster-data/master-2/data
--fs_wal_dir=/tmp/dist-test-taskRiQe_a/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790201539513534-1567-0/minicluster-data/master-2/wal
--ipki_ca_key_size=768
--master_addresses=127.1.135.254:35949,127.1.135.253:39445,127.1.135.252:42799
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskRiQe_a/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790201539513534-1567-0/krb5kdc/oryx_127.1.135.252.keytab
--principal=oryx/127.1.135.252
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.1.135.252:42799
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskRiQe_a/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790201539513534-1567-0/minicluster-data/master-2/data/info.pb
--webserver_interface=127.1.135.252
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.8319
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskRiQe_a/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790201539513534-1567-0/minicluster-data/master-2/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 23 Sep 2026 20:53:37 UTC on ccycloud.quasar-rkviyx.root.comops.site
build id 4668083
I20260923 22:12:49.444475 8319 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260923 22:12:49.444685 8319 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260923 22:12:49.449038 8327 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260923 22:12:49.449074 8324 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260923 22:12:49.449157 8325 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260923 22:12:49.449358 8319 server_base.cc:1034] running on GCE node
I20260923 22:12:49.449579 8319 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260923 22:12:49.449847 8319 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260923 22:12:49.450984 8319 hybrid_clock.cc:648] HybridClock initialized: now 1790201569450972 us; error 33 us; skew 500 ppm
Sep 23 22:12:49 dist-test-slave-s2fj krb5kdc[7622](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790201569, etypes {rep=17 tkt=17 ses=17}, oryx/127.1.135.252@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260923 22:12:49.453419 8319 init.cc:377] Logged in from keytab as oryx/127.1.135.252@KRBTEST.COM (short username oryx)
I20260923 22:12:49.453815 8319 webserver.cc:460] Webserver started at http://127.1.135.252:39309/ using document root <none> and password file <none>
I20260923 22:12:49.454023 8319 fs_manager.cc:359] Metadata directory not provided
I20260923 22:12:49.454071 8319 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260923 22:12:49.454226 8319 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260923 22:12:49.455230 8319 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskRiQe_a/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790201539513534-1567-0/minicluster-data/master-2/data/instance:
uuid: "d2a5b389b5d04491a85f778e638e056d"
format_stamp: "Formatted at 2026-09-23 22:12:49 on dist-test-slave-s2fj"
server_key: "014da626af946a7f7b00cd8955be6bd8"
server_key_iv: "4bbadceaa11c86ad605ae9619c4d42ce"
server_key_version: "encryptionkey@0"
I20260923 22:12:49.455577 8319 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskRiQe_a/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790201539513534-1567-0/minicluster-data/master-2/wal/instance:
uuid: "d2a5b389b5d04491a85f778e638e056d"
format_stamp: "Formatted at 2026-09-23 22:12:49 on dist-test-slave-s2fj"
server_key: "014da626af946a7f7b00cd8955be6bd8"
server_key_iv: "4bbadceaa11c86ad605ae9619c4d42ce"
server_key_version: "encryptionkey@0"
I20260923 22:12:49.456902 8319 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.001s sys 0.001s
I20260923 22:12:49.457563 8334 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260923 22:12:49.457731 8319 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.000s
I20260923 22:12:49.457826 8319 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskRiQe_a/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790201539513534-1567-0/minicluster-data/master-2/data,/tmp/dist-test-taskRiQe_a/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790201539513534-1567-0/minicluster-data/master-2/wal
uuid: "d2a5b389b5d04491a85f778e638e056d"
format_stamp: "Formatted at 2026-09-23 22:12:49 on dist-test-slave-s2fj"
server_key: "014da626af946a7f7b00cd8955be6bd8"
server_key_iv: "4bbadceaa11c86ad605ae9619c4d42ce"
server_key_version: "encryptionkey@0"
I20260923 22:12:49.457893 8319 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskRiQe_a/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790201539513534-1567-0/minicluster-data/master-2/wal
metadata directory: /tmp/dist-test-taskRiQe_a/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790201539513534-1567-0/minicluster-data/master-2/wal
1 data directories: /tmp/dist-test-taskRiQe_a/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790201539513534-1567-0/minicluster-data/master-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260923 22:12:49.489101 8319 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260923 22:12:49.491444 8319 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260923 22:12:49.491609 8319 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260923 22:12:49.496205 8319 rpc_server.cc:307] RPC server started. Bound to: 127.1.135.252:42799
I20260923 22:12:49.496238 8386 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.1.135.252:42799 every 8 connection(s)
I20260923 22:12:49.496604 8319 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskRiQe_a/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790201539513534-1567-0/minicluster-data/master-2/data/info.pb
I20260923 22:12:49.497612 8387 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260923 22:12:49.499056 8387 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.1.135.254" port: 35949 } has no permanent_uuid. Determining permanent_uuid...
Sep 23 22:12:49 dist-test-slave-s2fj krb5kdc[7622](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790201569, etypes {rep=17 tkt=17 ses=17}, oryx/127.1.135.252@KRBTEST.COM for oryx/127.1.135.254@KRBTEST.COM
I20260923 22:12:49.505424 1567 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskRiQe_a/build/release/bin/kudu as pid 8319
I20260923 22:12:49.507807 7713 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 22:12:49.499783 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52052 (local address 127.1.135.254:35949)
0923 22:12:49.499838 (+ 55us) server_negotiation.cc:206] Beginning negotiation
0923 22:12:49.499841 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0923 22:12:49.499928 (+ 87us) server_negotiation.cc:407] Connection header received
0923 22:12:49.500446 (+ 518us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 22:12:49.500446 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 22:12:49.500456 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 22:12:49.500570 (+ 114us) server_negotiation.cc:226] Negotiated authn=SASL
0923 22:12:49.501203 (+ 633us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 22:12:49.501704 (+ 501us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 22:12:49.502696 (+ 992us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 22:12:49.502876 (+ 180us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 22:12:49.502922 (+ 46us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 22:12:49.504937 (+ 2015us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0923 22:12:49.504938 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0923 22:12:49.504939 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0923 22:12:49.504945 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0923 22:12:49.506537 (+ 1592us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0923 22:12:49.506910 (+ 373us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0923 22:12:49.506911 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0923 22:12:49.506912 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0923 22:12:49.506966 (+ 54us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0923 22:12:49.507204 (+ 238us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0923 22:12:49.507205 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0923 22:12:49.507206 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0923 22:12:49.507316 (+ 110us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0923 22:12:49.507391 (+ 75us) server_negotiation.cc:1035] Waiting for connection context
0923 22:12:49.507648 (+ 257us) server_negotiation.cc:299] Negotiation successful
0923 22:12:49.507722 (+ 74us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":32}
I20260923 22:12:49.507995 8389 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 22:12:49.499656 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.135.254:35949 (local address 127.0.0.1:52052)
0923 22:12:49.499812 (+ 156us) negotiation.cc:107] Waiting for socket to connect
0923 22:12:49.499852 (+ 40us) client_negotiation.cc:174] Beginning negotiation
0923 22:12:49.500335 (+ 483us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0923 22:12:49.500660 (+ 325us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0923 22:12:49.500668 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0923 22:12:49.500944 (+ 276us) client_negotiation.cc:189] Negotiated authn=SASL
0923 22:12:49.501128 (+ 184us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 22:12:49.501132 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 22:12:49.501771 (+ 639us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 22:12:49.501772 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 22:12:49.502620 (+ 848us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 22:12:49.502622 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 22:12:49.502951 (+ 329us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 22:12:49.502952 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 22:12:49.503050 (+ 98us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 22:12:49.503457 (+ 407us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0923 22:12:49.503465 (+ 8us) client_negotiation.cc:656] calling sasl_client_start()
0923 22:12:49.504853 (+ 1388us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0923 22:12:49.506625 (+ 1772us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0923 22:12:49.506629 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0923 22:12:49.506630 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0923 22:12:49.506828 (+ 198us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0923 22:12:49.507029 (+ 201us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0923 22:12:49.507030 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0923 22:12:49.507030 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0923 22:12:49.507139 (+ 109us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0923 22:12:49.507379 (+ 240us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0923 22:12:49.507385 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0923 22:12:49.507517 (+ 132us) client_negotiation.cc:769] Sending connection context
0923 22:12:49.507618 (+ 101us) client_negotiation.cc:240] Negotiation successful
0923 22:12:49.507759 (+ 141us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":120,"thread_start_us":91,"threads_started":1}
I20260923 22:12:49.508505 8387 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.1.135.253" port: 39445 } has no permanent_uuid. Determining permanent_uuid...
Sep 23 22:12:49 dist-test-slave-s2fj krb5kdc[7622](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790201568, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for oryx/127.1.135.252@KRBTEST.COM
Sep 23 22:12:49 dist-test-slave-s2fj krb5kdc[7622](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790201569, etypes {rep=17 tkt=17 ses=17}, oryx/127.1.135.252@KRBTEST.COM for oryx/127.1.135.253@KRBTEST.COM
I20260923 22:12:49.514664 8390 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 22:12:49.505963 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43334 (local address 127.1.135.252:42799)
0923 22:12:49.506110 (+ 147us) server_negotiation.cc:206] Beginning negotiation
0923 22:12:49.506115 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0923 22:12:49.506132 (+ 17us) server_negotiation.cc:407] Connection header received
0923 22:12:49.506155 (+ 23us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 22:12:49.506157 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 22:12:49.506166 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 22:12:49.506226 (+ 60us) server_negotiation.cc:226] Negotiated authn=SASL
0923 22:12:49.506892 (+ 666us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 22:12:49.507534 (+ 642us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 22:12:49.508483 (+ 949us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 22:12:49.508940 (+ 457us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 22:12:49.509000 (+ 60us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 22:12:49.511340 (+ 2340us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0923 22:12:49.511347 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0923 22:12:49.511347 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0923 22:12:49.511354 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0923 22:12:49.513577 (+ 2223us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0923 22:12:49.513900 (+ 323us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0923 22:12:49.513902 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0923 22:12:49.513903 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0923 22:12:49.513958 (+ 55us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0923 22:12:49.514144 (+ 186us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0923 22:12:49.514145 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0923 22:12:49.514145 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0923 22:12:49.514276 (+ 131us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0923 22:12:49.514411 (+ 135us) server_negotiation.cc:1035] Waiting for connection context
0923 22:12:49.514478 (+ 67us) server_negotiation.cc:299] Negotiation successful
0923 22:12:49.514552 (+ 74us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":112,"thread_start_us":91,"threads_started":1}
I20260923 22:12:49.515754 8389 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 22:12:49.508735 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.135.253:39445 (local address 127.0.0.1:53484)
0923 22:12:49.508778 (+ 43us) negotiation.cc:107] Waiting for socket to connect
0923 22:12:49.508790 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0923 22:12:49.508831 (+ 41us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0923 22:12:49.508985 (+ 154us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0923 22:12:49.508985 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0923 22:12:49.509132 (+ 147us) client_negotiation.cc:189] Negotiated authn=SASL
0923 22:12:49.509192 (+ 60us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 22:12:49.509194 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 22:12:49.510004 (+ 810us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 22:12:49.510006 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 22:12:49.510649 (+ 643us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 22:12:49.510651 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 22:12:49.511070 (+ 419us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 22:12:49.511071 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 22:12:49.511159 (+ 88us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 22:12:49.511545 (+ 386us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0923 22:12:49.511548 (+ 3us) client_negotiation.cc:656] calling sasl_client_start()
0923 22:12:49.513251 (+ 1703us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0923 22:12:49.514718 (+ 1467us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0923 22:12:49.514719 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0923 22:12:49.514720 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0923 22:12:49.514928 (+ 208us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0923 22:12:49.515160 (+ 232us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0923 22:12:49.515161 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0923 22:12:49.515162 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0923 22:12:49.515236 (+ 74us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0923 22:12:49.515463 (+ 227us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0923 22:12:49.515464 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0923 22:12:49.515502 (+ 38us) client_negotiation.cc:769] Sending connection context
0923 22:12:49.515584 (+ 82us) client_negotiation.cc:240] Negotiation successful
0923 22:12:49.515670 (+ 86us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":28}
I20260923 22:12:49.515776 8316 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 22:12:49.508804 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53484 (local address 127.1.135.253:39445)
0923 22:12:49.508860 (+ 56us) server_negotiation.cc:206] Beginning negotiation
0923 22:12:49.508863 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0923 22:12:49.508871 (+ 8us) server_negotiation.cc:407] Connection header received
0923 22:12:49.508898 (+ 27us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 22:12:49.508899 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 22:12:49.508911 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 22:12:49.508965 (+ 54us) server_negotiation.cc:226] Negotiated authn=SASL
0923 22:12:49.509447 (+ 482us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 22:12:49.509907 (+ 460us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 22:12:49.510725 (+ 818us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 22:12:49.511003 (+ 278us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 22:12:49.511056 (+ 53us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 22:12:49.513326 (+ 2270us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0923 22:12:49.513327 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0923 22:12:49.513327 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0923 22:12:49.513334 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0923 22:12:49.514649 (+ 1315us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0923 22:12:49.515019 (+ 370us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0923 22:12:49.515020 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0923 22:12:49.515020 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0923 22:12:49.515076 (+ 56us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0923 22:12:49.515284 (+ 208us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0923 22:12:49.515285 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0923 22:12:49.515285 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0923 22:12:49.515390 (+ 105us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0923 22:12:49.515459 (+ 69us) server_negotiation.cc:1035] Waiting for connection context
0923 22:12:49.515610 (+ 151us) server_negotiation.cc:299] Negotiation successful
0923 22:12:49.515688 (+ 78us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":32}
I20260923 22:12:49.516207 8387 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.1.135.252" port: 42799 } has no permanent_uuid. Determining permanent_uuid...
Sep 23 22:12:49 dist-test-slave-s2fj krb5kdc[7622](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790201569, etypes {rep=17 tkt=17 ses=17}, oryx/127.1.135.252@KRBTEST.COM for oryx/127.1.135.252@KRBTEST.COM
I20260923 22:12:49.521587 8389 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 22:12:49.516416 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.135.252:42799 (local address 127.0.0.1:43344)
0923 22:12:49.516458 (+ 42us) negotiation.cc:107] Waiting for socket to connect
0923 22:12:49.516468 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0923 22:12:49.516504 (+ 36us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0923 22:12:49.516614 (+ 110us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0923 22:12:49.516615 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0923 22:12:49.516722 (+ 107us) client_negotiation.cc:189] Negotiated authn=SASL
0923 22:12:49.516744 (+ 22us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 22:12:49.516745 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 22:12:49.517264 (+ 519us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 22:12:49.517265 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 22:12:49.517692 (+ 427us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 22:12:49.517693 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 22:12:49.517977 (+ 284us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 22:12:49.517977 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 22:12:49.518010 (+ 33us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 22:12:49.518381 (+ 371us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0923 22:12:49.518390 (+ 9us) client_negotiation.cc:656] calling sasl_client_start()
0923 22:12:49.519394 (+ 1004us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0923 22:12:49.520726 (+ 1332us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0923 22:12:49.520727 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0923 22:12:49.520727 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0923 22:12:49.520884 (+ 157us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0923 22:12:49.521052 (+ 168us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0923 22:12:49.521052 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0923 22:12:49.521052 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0923 22:12:49.521101 (+ 49us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0923 22:12:49.521259 (+ 158us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0923 22:12:49.521259 (+ 0us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0923 22:12:49.521288 (+ 29us) client_negotiation.cc:769] Sending connection context
0923 22:12:49.521357 (+ 69us) client_negotiation.cc:240] Negotiation successful
0923 22:12:49.521451 (+ 94us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":29}
I20260923 22:12:49.521641 8390 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 22:12:49.516470 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43344 (local address 127.1.135.252:42799)
0923 22:12:49.516517 (+ 47us) server_negotiation.cc:206] Beginning negotiation
0923 22:12:49.516518 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0923 22:12:49.516524 (+ 6us) server_negotiation.cc:407] Connection header received
0923 22:12:49.516554 (+ 30us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 22:12:49.516554 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 22:12:49.516575 (+ 21us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 22:12:49.516611 (+ 36us) server_negotiation.cc:226] Negotiated authn=SASL
0923 22:12:49.516783 (+ 172us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 22:12:49.517211 (+ 428us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 22:12:49.517753 (+ 542us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 22:12:49.517923 (+ 170us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 22:12:49.517969 (+ 46us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 22:12:49.519470 (+ 1501us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0923 22:12:49.519471 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0923 22:12:49.519471 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0923 22:12:49.519476 (+ 5us) server_negotiation.cc:922] calling sasl_server_start()
0923 22:12:49.520663 (+ 1187us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0923 22:12:49.520941 (+ 278us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0923 22:12:49.520941 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0923 22:12:49.520942 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0923 22:12:49.520983 (+ 41us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0923 22:12:49.521139 (+ 156us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0923 22:12:49.521139 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0923 22:12:49.521139 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0923 22:12:49.521215 (+ 76us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0923 22:12:49.521256 (+ 41us) server_negotiation.cc:1035] Waiting for connection context
0923 22:12:49.521386 (+ 130us) server_negotiation.cc:299] Negotiation successful
0923 22:12:49.521491 (+ 105us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":46,"server-negotiator.queue_time_us":27}
I20260923 22:12:49.522727 8387 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P d2a5b389b5d04491a85f778e638e056d: Bootstrap starting.
I20260923 22:12:49.523571 8387 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P d2a5b389b5d04491a85f778e638e056d: Neither blocks nor log segments found. Creating new log.
I20260923 22:12:49.523828 8387 log.cc:824] T 00000000000000000000000000000000 P d2a5b389b5d04491a85f778e638e056d: Log is configured to *not* fsync() on all Append() calls
I20260923 22:12:49.524590 8387 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P d2a5b389b5d04491a85f778e638e056d: No bootstrap required, opened a new log
I20260923 22:12:49.525914 8387 raft_consensus.cc:348] T 00000000000000000000000000000000 P d2a5b389b5d04491a85f778e638e056d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6378d422392b4a74831ae1c0f0692998" member_type: VOTER last_known_addr { host: "127.1.135.254" port: 35949 } } peers { permanent_uuid: "b32070aba06045cabad174bee345066a" member_type: VOTER last_known_addr { host: "127.1.135.253" port: 39445 } } peers { permanent_uuid: "d2a5b389b5d04491a85f778e638e056d" member_type: VOTER last_known_addr { host: "127.1.135.252" port: 42799 } }
I20260923 22:12:49.526046 8387 raft_consensus.cc:374] T 00000000000000000000000000000000 P d2a5b389b5d04491a85f778e638e056d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260923 22:12:49.526098 8387 raft_consensus.cc:729] T 00000000000000000000000000000000 P d2a5b389b5d04491a85f778e638e056d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d2a5b389b5d04491a85f778e638e056d, State: Initialized, Role: FOLLOWER
I20260923 22:12:49.526229 8387 consensus_queue.cc:260] T 00000000000000000000000000000000 P d2a5b389b5d04491a85f778e638e056d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6378d422392b4a74831ae1c0f0692998" member_type: VOTER last_known_addr { host: "127.1.135.254" port: 35949 } } peers { permanent_uuid: "b32070aba06045cabad174bee345066a" member_type: VOTER last_known_addr { host: "127.1.135.253" port: 39445 } } peers { permanent_uuid: "d2a5b389b5d04491a85f778e638e056d" member_type: VOTER last_known_addr { host: "127.1.135.252" port: 42799 } }
I20260923 22:12:49.526497 8387 sys_catalog.cc:564] T 00000000000000000000000000000000 P d2a5b389b5d04491a85f778e638e056d [sys.catalog]: configured and running, proceeding with master startup.
I20260923 22:12:49.526461 8392 sys_catalog.cc:455] T 00000000000000000000000000000000 P d2a5b389b5d04491a85f778e638e056d [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6378d422392b4a74831ae1c0f0692998" member_type: VOTER last_known_addr { host: "127.1.135.254" port: 35949 } } peers { permanent_uuid: "b32070aba06045cabad174bee345066a" member_type: VOTER last_known_addr { host: "127.1.135.253" port: 39445 } } peers { permanent_uuid: "d2a5b389b5d04491a85f778e638e056d" member_type: VOTER last_known_addr { host: "127.1.135.252" port: 42799 } } }
I20260923 22:12:49.526579 8392 sys_catalog.cc:458] T 00000000000000000000000000000000 P d2a5b389b5d04491a85f778e638e056d [sys.catalog]: This master's current role is: FOLLOWER
W20260923 22:12:49.527962 8403 catalog_manager.cc:1558] T 00000000000000000000000000000000 P d2a5b389b5d04491a85f778e638e056d: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260923 22:12:49.528016 8403 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260923 22:12:49.530093 8319 master_runner.cc:428] Detected that this master d2a5b389b5d04491a85f778e638e056d is joining an existing cluster
I20260923 22:12:49.530144 8319 master_runner.cc:432] Initiating AddMaster RPC to add 127.1.135.252:42799
I20260923 22:12:49.537766 8411 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 22:12:49.531037 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.135.252:42799 (local address 127.0.0.1:43348)
0923 22:12:49.531322 (+ 285us) negotiation.cc:107] Waiting for socket to connect
0923 22:12:49.531331 (+ 9us) client_negotiation.cc:174] Beginning negotiation
0923 22:12:49.531382 (+ 51us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0923 22:12:49.531516 (+ 134us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0923 22:12:49.531516 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0923 22:12:49.531732 (+ 216us) client_negotiation.cc:189] Negotiated authn=SASL
0923 22:12:49.531808 (+ 76us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 22:12:49.531809 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 22:12:49.532683 (+ 874us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 22:12:49.532684 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 22:12:49.533611 (+ 927us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 22:12:49.533613 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 22:12:49.534053 (+ 440us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 22:12:49.534054 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 22:12:49.534105 (+ 51us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 22:12:49.534612 (+ 507us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0923 22:12:49.534616 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0923 22:12:49.534896 (+ 280us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0923 22:12:49.536757 (+ 1861us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0923 22:12:49.536757 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0923 22:12:49.536758 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0923 22:12:49.536912 (+ 154us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0923 22:12:49.537095 (+ 183us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0923 22:12:49.537096 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0923 22:12:49.537096 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0923 22:12:49.537149 (+ 53us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0923 22:12:49.537395 (+ 246us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0923 22:12:49.537396 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0923 22:12:49.537434 (+ 38us) client_negotiation.cc:769] Sending connection context
0923 22:12:49.537525 (+ 91us) client_negotiation.cc:240] Negotiation successful
0923 22:12:49.537646 (+ 121us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":270,"mutex_wait_us":49,"thread_start_us":78,"threads_started":1}
I20260923 22:12:49.537782 8390 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 22:12:49.531372 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43348 (local address 127.1.135.252:42799)
0923 22:12:49.531427 (+ 55us) server_negotiation.cc:206] Beginning negotiation
0923 22:12:49.531429 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0923 22:12:49.531435 (+ 6us) server_negotiation.cc:407] Connection header received
0923 22:12:49.531450 (+ 15us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 22:12:49.531450 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 22:12:49.531457 (+ 7us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 22:12:49.531511 (+ 54us) server_negotiation.cc:226] Negotiated authn=SASL
0923 22:12:49.531880 (+ 369us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 22:12:49.532609 (+ 729us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 22:12:49.533694 (+ 1085us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 22:12:49.533952 (+ 258us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 22:12:49.533986 (+ 34us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 22:12:49.534993 (+ 1007us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0923 22:12:49.534994 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0923 22:12:49.534994 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0923 22:12:49.535001 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0923 22:12:49.536667 (+ 1666us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0923 22:12:49.536985 (+ 318us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0923 22:12:49.536986 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0923 22:12:49.536987 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0923 22:12:49.537032 (+ 45us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0923 22:12:49.537220 (+ 188us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0923 22:12:49.537221 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0923 22:12:49.537221 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0923 22:12:49.537318 (+ 97us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0923 22:12:49.537393 (+ 75us) server_negotiation.cc:1035] Waiting for connection context
0923 22:12:49.537546 (+ 153us) server_negotiation.cc:299] Negotiation successful
0923 22:12:49.537698 (+ 152us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":93,"server-negotiator.queue_time_us":28}
I20260923 22:12:49.564414 8316 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 22:12:49.531389 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53492 (local address 127.1.135.253:39445)
0923 22:12:49.531554 (+ 165us) server_negotiation.cc:206] Beginning negotiation
0923 22:12:49.531556 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0923 22:12:49.531660 (+ 104us) server_negotiation.cc:407] Connection header received
0923 22:12:49.531747 (+ 87us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 22:12:49.531748 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 22:12:49.531758 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 22:12:49.531801 (+ 43us) server_negotiation.cc:226] Negotiated authn=SASL
0923 22:12:49.532177 (+ 376us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 22:12:49.532840 (+ 663us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 22:12:49.533798 (+ 958us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 22:12:49.534105 (+ 307us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 22:12:49.534144 (+ 39us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 22:12:49.561324 (+ 27180us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0923 22:12:49.561326 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0923 22:12:49.561326 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0923 22:12:49.561334 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0923 22:12:49.563262 (+ 1928us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0923 22:12:49.563597 (+ 335us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0923 22:12:49.563598 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0923 22:12:49.563598 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0923 22:12:49.563648 (+ 50us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0923 22:12:49.563870 (+ 222us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0923 22:12:49.563871 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0923 22:12:49.563871 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0923 22:12:49.563989 (+ 118us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0923 22:12:49.564038 (+ 49us) server_negotiation.cc:1035] Waiting for connection context
0923 22:12:49.564242 (+ 204us) server_negotiation.cc:299] Negotiation successful
0923 22:12:49.564325 (+ 83us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":143}
I20260923 22:12:49.564435 7713 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 22:12:49.531171 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52056 (local address 127.1.135.254:35949)
0923 22:12:49.531240 (+ 69us) server_negotiation.cc:206] Beginning negotiation
0923 22:12:49.531242 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0923 22:12:49.531249 (+ 7us) server_negotiation.cc:407] Connection header received
0923 22:12:49.531270 (+ 21us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 22:12:49.531271 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 22:12:49.531280 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 22:12:49.531333 (+ 53us) server_negotiation.cc:226] Negotiated authn=SASL
0923 22:12:49.531758 (+ 425us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 22:12:49.532544 (+ 786us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 22:12:49.533704 (+ 1160us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 22:12:49.533989 (+ 285us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 22:12:49.534031 (+ 42us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 22:12:49.561333 (+ 27302us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0923 22:12:49.561335 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0923 22:12:49.561335 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0923 22:12:49.561343 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0923 22:12:49.563025 (+ 1682us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0923 22:12:49.563417 (+ 392us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0923 22:12:49.563418 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0923 22:12:49.563418 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0923 22:12:49.563479 (+ 61us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0923 22:12:49.563669 (+ 190us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0923 22:12:49.563670 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0923 22:12:49.563670 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0923 22:12:49.563800 (+ 130us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0923 22:12:49.563875 (+ 75us) server_negotiation.cc:1035] Waiting for connection context
0923 22:12:49.564278 (+ 403us) server_negotiation.cc:299] Negotiation successful
0923 22:12:49.564350 (+ 72us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":48}
I20260923 22:12:49.564695 8409 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 22:12:49.530997 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.135.254:35949 (local address 127.0.0.1:52056)
0923 22:12:49.531145 (+ 148us) negotiation.cc:107] Waiting for socket to connect
0923 22:12:49.531155 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0923 22:12:49.531199 (+ 44us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0923 22:12:49.531418 (+ 219us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0923 22:12:49.531419 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0923 22:12:49.531595 (+ 176us) client_negotiation.cc:189] Negotiated authn=SASL
0923 22:12:49.531663 (+ 68us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 22:12:49.531665 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 22:12:49.532641 (+ 976us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 22:12:49.532642 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 22:12:49.533601 (+ 959us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 22:12:49.533603 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 22:12:49.534116 (+ 513us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 22:12:49.534117 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 22:12:49.534182 (+ 65us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 22:12:49.560580 (+ 26398us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0923 22:12:49.560585 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0923 22:12:49.561177 (+ 592us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0923 22:12:49.563132 (+ 1955us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0923 22:12:49.563134 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0923 22:12:49.563134 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0923 22:12:49.563300 (+ 166us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0923 22:12:49.563542 (+ 242us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0923 22:12:49.563542 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0923 22:12:49.563543 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0923 22:12:49.563614 (+ 71us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0923 22:12:49.564097 (+ 483us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0923 22:12:49.564098 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0923 22:12:49.564140 (+ 42us) client_negotiation.cc:769] Sending connection context
0923 22:12:49.564515 (+ 375us) client_negotiation.cc:240] Negotiation successful
0923 22:12:49.564629 (+ 114us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":124,"mutex_wait_us":29,"thread_start_us":100,"threads_started":1}
I20260923 22:12:49.564428 8410 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 22:12:49.531221 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.135.253:39445 (local address 127.0.0.1:53492)
0923 22:12:49.531610 (+ 389us) negotiation.cc:107] Waiting for socket to connect
0923 22:12:49.531619 (+ 9us) client_negotiation.cc:174] Beginning negotiation
0923 22:12:49.531680 (+ 61us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0923 22:12:49.531859 (+ 179us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0923 22:12:49.531859 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0923 22:12:49.532014 (+ 155us) client_negotiation.cc:189] Negotiated authn=SASL
0923 22:12:49.532068 (+ 54us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 22:12:49.532099 (+ 31us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 22:12:49.532920 (+ 821us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 22:12:49.532921 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 22:12:49.533723 (+ 802us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 22:12:49.533725 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 22:12:49.534214 (+ 489us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 22:12:49.534215 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 22:12:49.534290 (+ 75us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 22:12:49.560738 (+ 26448us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0923 22:12:49.560742 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0923 22:12:49.561184 (+ 442us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0923 22:12:49.563353 (+ 2169us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0923 22:12:49.563353 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0923 22:12:49.563354 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0923 22:12:49.563542 (+ 188us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0923 22:12:49.563739 (+ 197us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0923 22:12:49.563740 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0923 22:12:49.563741 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0923 22:12:49.563800 (+ 59us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0923 22:12:49.564078 (+ 278us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0923 22:12:49.564079 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0923 22:12:49.564128 (+ 49us) client_negotiation.cc:769] Sending connection context
0923 22:12:49.564206 (+ 78us) client_negotiation.cc:240] Negotiation successful
0923 22:12:49.564313 (+ 107us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":375,"thread_start_us":67,"threads_started":1}
I20260923 22:12:49.565799 1567 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskRiQe_a/build/release/bin/kudu with pid 7729
Sep 23 22:12:49 dist-test-slave-s2fj krb5kdc[7622](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790201568, etypes {rep=17 tkt=17 ses=17}, oryx/127.1.135.254@KRBTEST.COM for oryx/127.1.135.252@KRBTEST.COM
I20260923 22:12:49.572813 8225 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 22:12:49.566650 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.135.252:42799 (local address 127.0.0.1:43350)
0923 22:12:49.566696 (+ 46us) negotiation.cc:107] Waiting for socket to connect
0923 22:12:49.566707 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0923 22:12:49.566745 (+ 38us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0923 22:12:49.566867 (+ 122us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0923 22:12:49.566868 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0923 22:12:49.567008 (+ 140us) client_negotiation.cc:189] Negotiated authn=SASL
0923 22:12:49.567046 (+ 38us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 22:12:49.567048 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 22:12:49.567641 (+ 593us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 22:12:49.567643 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 22:12:49.568239 (+ 596us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 22:12:49.568240 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 22:12:49.568597 (+ 357us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 22:12:49.568599 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 22:12:49.568673 (+ 74us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 22:12:49.569062 (+ 389us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0923 22:12:49.569067 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0923 22:12:49.570262 (+ 1195us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0923 22:12:49.571750 (+ 1488us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0923 22:12:49.571751 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0923 22:12:49.571751 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0923 22:12:49.571985 (+ 234us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0923 22:12:49.572219 (+ 234us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0923 22:12:49.572220 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0923 22:12:49.572220 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0923 22:12:49.572276 (+ 56us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0923 22:12:49.572530 (+ 254us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0923 22:12:49.572531 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0923 22:12:49.572571 (+ 40us) client_negotiation.cc:769] Sending connection context
0923 22:12:49.572631 (+ 60us) client_negotiation.cc:240] Negotiation successful
0923 22:12:49.572722 (+ 91us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":30}
I20260923 22:12:49.573195 8390 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 22:12:49.566685 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43350 (local address 127.1.135.252:42799)
0923 22:12:49.566719 (+ 34us) server_negotiation.cc:206] Beginning negotiation
0923 22:12:49.566720 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0923 22:12:49.566730 (+ 10us) server_negotiation.cc:407] Connection header received
0923 22:12:49.566802 (+ 72us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 22:12:49.566803 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 22:12:49.566812 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 22:12:49.566854 (+ 42us) server_negotiation.cc:226] Negotiated authn=SASL
0923 22:12:49.567119 (+ 265us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 22:12:49.567566 (+ 447us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 22:12:49.568312 (+ 746us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 22:12:49.568514 (+ 202us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 22:12:49.568593 (+ 79us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 22:12:49.570360 (+ 1767us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0923 22:12:49.570361 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0923 22:12:49.570362 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0923 22:12:49.570368 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0923 22:12:49.571642 (+ 1274us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0923 22:12:49.572067 (+ 425us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0923 22:12:49.572068 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0923 22:12:49.572068 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0923 22:12:49.572151 (+ 83us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0923 22:12:49.572338 (+ 187us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0923 22:12:49.572339 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0923 22:12:49.572339 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0923 22:12:49.572447 (+ 108us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0923 22:12:49.572502 (+ 55us) server_negotiation.cc:1035] Waiting for connection context
0923 22:12:49.572980 (+ 478us) server_negotiation.cc:299] Negotiation successful
0923 22:12:49.573117 (+ 137us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":17}
I20260923 22:12:49.573622 7667 catalog_manager.cc:6979] Initiating ChangeConfig request to add master 127.1.135.252:42799: tablet_id: "00000000000000000000000000000000"
type: ADD_PEER
server {
permanent_uuid: "d2a5b389b5d04491a85f778e638e056d"
member_type: NON_VOTER
last_known_addr {
host: "127.1.135.252"
port: 42799
}
attrs {
promote: true
}
}
dest_uuid: "6378d422392b4a74831ae1c0f0692998"
cas_config_opid_index: 5
I20260923 22:12:49.573822 7667 consensus_queue.cc:237] T 00000000000000000000000000000000 P 6378d422392b4a74831ae1c0f0692998 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 1.5, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: 6 OBSOLETE_local: true peers { permanent_uuid: "6378d422392b4a74831ae1c0f0692998" member_type: VOTER last_known_addr { host: "127.1.135.254" port: 35949 } } peers { permanent_uuid: "b32070aba06045cabad174bee345066a" member_type: NON_VOTER last_known_addr { host: "127.1.135.253" port: 39445 } attrs { promote: true } } peers { permanent_uuid: "d2a5b389b5d04491a85f778e638e056d" member_type: NON_VOTER last_known_addr { host: "127.1.135.252" port: 42799 } attrs { promote: true } }
I20260923 22:12:49.573987 1567 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskRiQe_a/build/release/bin/kudu
/tmp/dist-test-taskRiQe_a/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskRiQe_a/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790201539513534-1567-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskRiQe_a/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790201539513534-1567-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskRiQe_a/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790201539513534-1567-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskRiQe_a/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790201539513534-1567-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.1.135.193:44597
--local_ip_for_outbound_sockets=127.1.135.193
--tserver_master_addrs=127.1.135.254:35949
--webserver_port=44031
--webserver_interface=127.1.135.193
--keytab_file=/tmp/dist-test-taskRiQe_a/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790201539513534-1567-0/krb5kdc/oryx_127.1.135.193.keytab
--principal=oryx/127.1.135.193
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.1.135.212:35751
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--tserver_master_addrs=127.1.135.254:35949,127.1.135.253:39445
--tserver_master_addrs=127.1.135.254:35949,127.1.135.253:39445,127.1.135.252:42799 with env {KRB5CCNAME=/tmp/dist-test-taskRiQe_a/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790201539513534-1567-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskRiQe_a/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790201539513534-1567-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskRiQe_a/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790201539513534-1567-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260923 22:12:49.575592 8226 raft_consensus.cc:2932] T 00000000000000000000000000000000 P 6378d422392b4a74831ae1c0f0692998 [term 1 LEADER]: Committing config change with OpId 1.6: config changed from index 5 to 6, NON_VOTER d2a5b389b5d04491a85f778e638e056d (127.1.135.252) added. New config: { opid_index: 6 OBSOLETE_local: true peers { permanent_uuid: "6378d422392b4a74831ae1c0f0692998" member_type: VOTER last_known_addr { host: "127.1.135.254" port: 35949 } } peers { permanent_uuid: "b32070aba06045cabad174bee345066a" member_type: NON_VOTER last_known_addr { host: "127.1.135.253" port: 39445 } attrs { promote: true } } peers { permanent_uuid: "d2a5b389b5d04491a85f778e638e056d" member_type: NON_VOTER last_known_addr { host: "127.1.135.252" port: 42799 } attrs { promote: true } } }
I20260923 22:12:49.575721 8361 raft_consensus.cc:3037] T 00000000000000000000000000000000 P d2a5b389b5d04491a85f778e638e056d [term 0 FOLLOWER]: Advancing to term 1
I20260923 22:12:49.576663 8226 catalog_manager.cc:6983] Successfully completed master ChangeConfig request to add master 127.1.135.252:42799
I20260923 22:12:49.576670 8361 raft_consensus.cc:1261] T 00000000000000000000000000000000 P d2a5b389b5d04491a85f778e638e056d [term 1 FOLLOWER]: Refusing update from remote peer 6378d422392b4a74831ae1c0f0692998: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 6. (index mismatch)
I20260923 22:12:49.576742 8226 sys_catalog.cc:455] T 00000000000000000000000000000000 P 6378d422392b4a74831ae1c0f0692998 [sys.catalog]: SysCatalogTable state changed. Reason: Config change replication complete. Latest consensus state: current_term: 1 leader_uuid: "6378d422392b4a74831ae1c0f0692998" committed_config { opid_index: 6 OBSOLETE_local: true peers { permanent_uuid: "6378d422392b4a74831ae1c0f0692998" member_type: VOTER last_known_addr { host: "127.1.135.254" port: 35949 } } peers { permanent_uuid: "b32070aba06045cabad174bee345066a" member_type: NON_VOTER last_known_addr { host: "127.1.135.253" port: 39445 } attrs { promote: true } } peers { permanent_uuid: "d2a5b389b5d04491a85f778e638e056d" member_type: NON_VOTER last_known_addr { host: "127.1.135.252" port: 42799 } attrs { promote: true } } }
I20260923 22:12:49.576815 8226 sys_catalog.cc:458] T 00000000000000000000000000000000 P 6378d422392b4a74831ae1c0f0692998 [sys.catalog]: This master's current role is: LEADER
I20260923 22:12:49.576998 8227 consensus_queue.cc:1035] T 00000000000000000000000000000000 P 6378d422392b4a74831ae1c0f0692998 [LEADER]: Connected to new peer: Peer: permanent_uuid: "d2a5b389b5d04491a85f778e638e056d" member_type: NON_VOTER last_known_addr { host: "127.1.135.252" port: 42799 } attrs { promote: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 0, Time since last communication: 0.000s
I20260923 22:12:49.577327 8319 master.cc:537] Master@127.1.135.252:42799 shutting down...
W20260923 22:12:49.579658 7653 consensus_peers.cc:487] T 00000000000000000000000000000000 P 6378d422392b4a74831ae1c0f0692998 -> Peer d2a5b389b5d04491a85f778e638e056d (127.1.135.252:42799): Couldn't send request to peer d2a5b389b5d04491a85f778e638e056d. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on Master. This is attempt 1: this message will repeat every 5th retry.
I20260923 22:12:49.580435 8319 raft_consensus.cc:2229] T 00000000000000000000000000000000 P d2a5b389b5d04491a85f778e638e056d [term 1 FOLLOWER]: Raft consensus shutting down.
I20260923 22:12:49.580523 8319 raft_consensus.cc:2258] T 00000000000000000000000000000000 P d2a5b389b5d04491a85f778e638e056d [term 1 FOLLOWER]: Raft consensus is shut down!
I20260923 22:12:49.580592 8319 tablet_replica.cc:330] T 00000000000000000000000000000000 P d2a5b389b5d04491a85f778e638e056d: stopping tablet replica
I20260923 22:12:49.581933 8225 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 22:12:49.575426 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.135.253:39445 (local address 127.0.0.1:53508)
0923 22:12:49.576837 (+ 1411us) negotiation.cc:107] Waiting for socket to connect
0923 22:12:49.576848 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0923 22:12:49.576903 (+ 55us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0923 22:12:49.577110 (+ 207us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0923 22:12:49.577111 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0923 22:12:49.577112 (+ 1us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0923 22:12:49.577146 (+ 34us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 22:12:49.577147 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 22:12:49.578378 (+ 1231us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 22:12:49.578379 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 22:12:49.580807 (+ 2428us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 22:12:49.580809 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 22:12:49.581669 (+ 860us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 22:12:49.581670 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 22:12:49.581740 (+ 70us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 22:12:49.581760 (+ 20us) client_negotiation.cc:769] Sending connection context
0923 22:12:49.581810 (+ 50us) client_negotiation.cc:240] Negotiation successful
0923 22:12:49.581851 (+ 41us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1394}
I20260923 22:12:49.582260 8316 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 22:12:49.575501 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53508 (local address 127.1.135.253:39445)
0923 22:12:49.575539 (+ 38us) server_negotiation.cc:206] Beginning negotiation
0923 22:12:49.575542 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0923 22:12:49.576895 (+ 1353us) server_negotiation.cc:407] Connection header received
0923 22:12:49.576956 (+ 61us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 22:12:49.576957 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 22:12:49.576980 (+ 23us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 22:12:49.577041 (+ 61us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0923 22:12:49.577221 (+ 180us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 22:12:49.577993 (+ 772us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 22:12:49.580900 (+ 2907us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 22:12:49.581485 (+ 585us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 22:12:49.581546 (+ 61us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 22:12:49.581635 (+ 89us) server_negotiation.cc:1035] Waiting for connection context
0923 22:12:49.582141 (+ 506us) server_negotiation.cc:299] Negotiation successful
0923 22:12:49.582181 (+ 40us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":20}
I20260923 22:12:49.582686 8275 raft_consensus.cc:1261] T 00000000000000000000000000000000 P b32070aba06045cabad174bee345066a [term 1 LEARNER]: Refusing update from remote peer 6378d422392b4a74831ae1c0f0692998: Log matching property violated. Preceding OpId in replica: term: 1 index: 5. Preceding OpId from leader: term: 1 index: 6. (index mismatch)
I20260923 22:12:49.582896 8227 consensus_queue.cc:1035] T 00000000000000000000000000000000 P 6378d422392b4a74831ae1c0f0692998 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b32070aba06045cabad174bee345066a" member_type: NON_VOTER last_known_addr { host: "127.1.135.253" port: 39445 } attrs { promote: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.000s
I20260923 22:12:49.583441 8275 raft_consensus.cc:2932] T 00000000000000000000000000000000 P b32070aba06045cabad174bee345066a [term 1 LEARNER]: Committing config change with OpId 1.6: config changed from index 5 to 6, NON_VOTER d2a5b389b5d04491a85f778e638e056d (127.1.135.252) added. New config: { opid_index: 6 OBSOLETE_local: true peers { permanent_uuid: "6378d422392b4a74831ae1c0f0692998" member_type: VOTER last_known_addr { host: "127.1.135.254" port: 35949 } } peers { permanent_uuid: "b32070aba06045cabad174bee345066a" member_type: NON_VOTER last_known_addr { host: "127.1.135.253" port: 39445 } attrs { promote: true } } peers { permanent_uuid: "d2a5b389b5d04491a85f778e638e056d" member_type: NON_VOTER last_known_addr { host: "127.1.135.252" port: 42799 } attrs { promote: true } } }
I20260923 22:12:49.584728 8227 raft_consensus.cc:1050] T 00000000000000000000000000000000 P 6378d422392b4a74831ae1c0f0692998: attempting to promote NON_VOTER b32070aba06045cabad174bee345066a to VOTER
I20260923 22:12:49.584740 8304 sys_catalog.cc:455] T 00000000000000000000000000000000 P b32070aba06045cabad174bee345066a [sys.catalog]: SysCatalogTable state changed. Reason: New leader 6378d422392b4a74831ae1c0f0692998. Latest consensus state: current_term: 1 leader_uuid: "6378d422392b4a74831ae1c0f0692998" committed_config { opid_index: 6 OBSOLETE_local: true peers { permanent_uuid: "6378d422392b4a74831ae1c0f0692998" member_type: VOTER last_known_addr { host: "127.1.135.254" port: 35949 } } peers { permanent_uuid: "b32070aba06045cabad174bee345066a" member_type: NON_VOTER last_known_addr { host: "127.1.135.253" port: 39445 } attrs { promote: true } } peers { permanent_uuid: "d2a5b389b5d04491a85f778e638e056d" member_type: NON_VOTER last_known_addr { host: "127.1.135.252" port: 42799 } attrs { promote: true } } }
I20260923 22:12:49.584816 8304 sys_catalog.cc:458] T 00000000000000000000000000000000 P b32070aba06045cabad174bee345066a [sys.catalog]: This master's current role is: LEARNER
I20260923 22:12:49.584851 8304 sys_catalog.cc:455] T 00000000000000000000000000000000 P b32070aba06045cabad174bee345066a [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "6378d422392b4a74831ae1c0f0692998" committed_config { opid_index: 6 OBSOLETE_local: true peers { permanent_uuid: "6378d422392b4a74831ae1c0f0692998" member_type: VOTER last_known_addr { host: "127.1.135.254" port: 35949 } } peers { permanent_uuid: "b32070aba06045cabad174bee345066a" member_type: NON_VOTER last_known_addr { host: "127.1.135.253" port: 39445 } attrs { promote: true } } peers { permanent_uuid: "d2a5b389b5d04491a85f778e638e056d" member_type: NON_VOTER last_known_addr { host: "127.1.135.252" port: 42799 } attrs { promote: true } } }
I20260923 22:12:49.584887 8304 sys_catalog.cc:458] T 00000000000000000000000000000000 P b32070aba06045cabad174bee345066a [sys.catalog]: This master's current role is: LEARNER
I20260923 22:12:49.584928 8227 consensus_queue.cc:237] T 00000000000000000000000000000000 P 6378d422392b4a74831ae1c0f0692998 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 1.6, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "6378d422392b4a74831ae1c0f0692998" member_type: VOTER last_known_addr { host: "127.1.135.254" port: 35949 } } peers { permanent_uuid: "b32070aba06045cabad174bee345066a" member_type: VOTER last_known_addr { host: "127.1.135.253" port: 39445 } attrs { promote: false } } peers { permanent_uuid: "d2a5b389b5d04491a85f778e638e056d" member_type: NON_VOTER last_known_addr { host: "127.1.135.252" port: 42799 } attrs { promote: true } }
I20260923 22:12:49.585403 8225 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 22:12:49.585288 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.135.252:42799 (local address 127.0.0.1:43354)
0923 22:12:49.585324 (+ 36us) negotiation.cc:107] Waiting for socket to connect
0923 22:12:49.585363 (+ 39us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.135.252:42799: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":23}
I20260923 22:12:49.585717 8275 raft_consensus.cc:1261] T 00000000000000000000000000000000 P b32070aba06045cabad174bee345066a [term 1 LEARNER]: Refusing update from remote peer 6378d422392b4a74831ae1c0f0692998: Log matching property violated. Preceding OpId in replica: term: 1 index: 6. Preceding OpId from leader: term: 1 index: 7. (index mismatch)
I20260923 22:12:49.585743 8225 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 22:12:49.585629 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.135.252:42799 (local address 127.0.0.1:43366)
0923 22:12:49.585692 (+ 63us) negotiation.cc:107] Waiting for socket to connect
0923 22:12:49.585715 (+ 23us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.135.252:42799: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":50}
I20260923 22:12:49.585870 8415 consensus_queue.cc:1035] T 00000000000000000000000000000000 P 6378d422392b4a74831ae1c0f0692998 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b32070aba06045cabad174bee345066a" member_type: VOTER last_known_addr { host: "127.1.135.253" port: 39445 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.000s
W20260923 22:12:49.585935 7653 consensus_peers.cc:487] T 00000000000000000000000000000000 P 6378d422392b4a74831ae1c0f0692998 -> Peer d2a5b389b5d04491a85f778e638e056d (127.1.135.252:42799): Couldn't send request to peer d2a5b389b5d04491a85f778e638e056d. Status: Network error: Client connection negotiation failed: client connection to 127.1.135.252:42799: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
I20260923 22:12:49.586413 8226 raft_consensus.cc:2932] T 00000000000000000000000000000000 P 6378d422392b4a74831ae1c0f0692998 [term 1 LEADER]: Committing config change with OpId 1.7: config changed from index 6 to 7, b32070aba06045cabad174bee345066a (127.1.135.253) changed from NON_VOTER to VOTER. New config: { opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "6378d422392b4a74831ae1c0f0692998" member_type: VOTER last_known_addr { host: "127.1.135.254" port: 35949 } } peers { permanent_uuid: "b32070aba06045cabad174bee345066a" member_type: VOTER last_known_addr { host: "127.1.135.253" port: 39445 } attrs { promote: false } } peers { permanent_uuid: "d2a5b389b5d04491a85f778e638e056d" member_type: NON_VOTER last_known_addr { host: "127.1.135.252" port: 42799 } attrs { promote: true } } }
I20260923 22:12:49.587098 8226 sys_catalog.cc:455] T 00000000000000000000000000000000 P 6378d422392b4a74831ae1c0f0692998 [sys.catalog]: SysCatalogTable state changed. Reason: Config change replication complete. Latest consensus state: current_term: 1 leader_uuid: "6378d422392b4a74831ae1c0f0692998" committed_config { opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "6378d422392b4a74831ae1c0f0692998" member_type: VOTER last_known_addr { host: "127.1.135.254" port: 35949 } } peers { permanent_uuid: "b32070aba06045cabad174bee345066a" member_type: VOTER last_known_addr { host: "127.1.135.253" port: 39445 } attrs { promote: false } } peers { permanent_uuid: "d2a5b389b5d04491a85f778e638e056d" member_type: NON_VOTER last_known_addr { host: "127.1.135.252" port: 42799 } attrs { promote: true } } }
I20260923 22:12:49.587173 8226 sys_catalog.cc:458] T 00000000000000000000000000000000 P 6378d422392b4a74831ae1c0f0692998 [sys.catalog]: This master's current role is: LEADER
I20260923 22:12:49.587229 8226 sys_catalog.cc:455] T 00000000000000000000000000000000 P 6378d422392b4a74831ae1c0f0692998 [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "6378d422392b4a74831ae1c0f0692998" committed_config { opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "6378d422392b4a74831ae1c0f0692998" member_type: VOTER last_known_addr { host: "127.1.135.254" port: 35949 } } peers { permanent_uuid: "b32070aba06045cabad174bee345066a" member_type: VOTER last_known_addr { host: "127.1.135.253" port: 39445 } attrs { promote: false } } peers { permanent_uuid: "d2a5b389b5d04491a85f778e638e056d" member_type: NON_VOTER last_known_addr { host: "127.1.135.252" port: 42799 } attrs { promote: true } } }
I20260923 22:12:49.587284 8226 sys_catalog.cc:458] T 00000000000000000000000000000000 P 6378d422392b4a74831ae1c0f0692998 [sys.catalog]: This master's current role is: LEADER
I20260923 22:12:49.587471 8275 raft_consensus.cc:2932] T 00000000000000000000000000000000 P b32070aba06045cabad174bee345066a [term 1 FOLLOWER]: Committing config change with OpId 1.7: config changed from index 6 to 7, b32070aba06045cabad174bee345066a (127.1.135.253) changed from NON_VOTER to VOTER. New config: { opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "6378d422392b4a74831ae1c0f0692998" member_type: VOTER last_known_addr { host: "127.1.135.254" port: 35949 } } peers { permanent_uuid: "b32070aba06045cabad174bee345066a" member_type: VOTER last_known_addr { host: "127.1.135.253" port: 39445 } attrs { promote: false } } peers { permanent_uuid: "d2a5b389b5d04491a85f778e638e056d" member_type: NON_VOTER last_known_addr { host: "127.1.135.252" port: 42799 } attrs { promote: true } } }
I20260923 22:12:49.588199 8414 sys_catalog.cc:455] T 00000000000000000000000000000000 P b32070aba06045cabad174bee345066a [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "6378d422392b4a74831ae1c0f0692998" committed_config { opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "6378d422392b4a74831ae1c0f0692998" member_type: VOTER last_known_addr { host: "127.1.135.254" port: 35949 } } peers { permanent_uuid: "b32070aba06045cabad174bee345066a" member_type: VOTER last_known_addr { host: "127.1.135.253" port: 39445 } attrs { promote: false } } peers { permanent_uuid: "d2a5b389b5d04491a85f778e638e056d" member_type: NON_VOTER last_known_addr { host: "127.1.135.252" port: 42799 } attrs { promote: true } } }
I20260923 22:12:49.588302 8414 sys_catalog.cc:458] T 00000000000000000000000000000000 P b32070aba06045cabad174bee345066a [sys.catalog]: This master's current role is: FOLLOWER
I20260923 22:12:49.593572 8319 master.cc:552] Master@127.1.135.252:42799 shutdown complete.
I20260923 22:12:49.593633 8319 master_runner.cc:305] Clearing existing system tablet
I20260923 22:12:49.593757 8319 fs_manager.cc:359] Metadata directory not provided
I20260923 22:12:49.593812 8319 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260923 22:12:49.594388 8319 fs_manager.cc:709] Time spent opening directory manager: real 0.000s user 0.001s sys 0.000s
I20260923 22:12:49.594835 8417 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260923 22:12:49.594977 8319 fs_report.cc:383] FS layout report
--------------------
wal directory:
metadata directory:
1 data directories: /tmp/dist-test-taskRiQe_a/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790201539513534-1567-0/minicluster-data/master-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260923 22:12:49.595033 8319 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.000s
I20260923 22:12:49.595062 8319 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskRiQe_a/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790201539513534-1567-0/minicluster-data/master-2/data,/tmp/dist-test-taskRiQe_a/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790201539513534-1567-0/minicluster-data/master-2/wal
uuid: "d2a5b389b5d04491a85f778e638e056d"
format_stamp: "Formatted at 2026-09-23 22:12:49 on dist-test-slave-s2fj"
server_key: "014da626af946a7f7b00cd8955be6bd8"
server_key_iv: "4bbadceaa11c86ad605ae9619c4d42ce"
server_key_version: "encryptionkey@0"
I20260923 22:12:49.595655 8319 ts_tablet_manager.cc:1905] T 00000000000000000000000000000000 P d2a5b389b5d04491a85f778e638e056d: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260923 22:12:49.596998 8319 ts_tablet_manager.cc:1918] T 00000000000000000000000000000000 P d2a5b389b5d04491a85f778e638e056d: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId unknown
I20260923 22:12:49.597079 8319 log.cc:1196] T 00000000000000000000000000000000 P d2a5b389b5d04491a85f778e638e056d: Deleting WAL directory at /tmp/dist-test-taskRiQe_a/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790201539513534-1567-0/minicluster-data/master-2/wal/wals/00000000000000000000000000000000
I20260923 22:12:49.597263 8319 ts_tablet_manager.cc:1939] T 00000000000000000000000000000000 P d2a5b389b5d04491a85f778e638e056d: Deleting consensus metadata
I20260923 22:12:49.597391 8319 master_runner.cc:315] Copying system tablet from 127.1.135.254:35949
I20260923 22:12:49.597956 8319 tablet_copy_client.cc:316] T 00000000000000000000000000000000 P d2a5b389b5d04491a85f778e638e056d: tablet copy: Beginning tablet copy session from remote peer at address 127.1.135.254:35949
I20260923 22:12:49.603734 7713 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 22:12:49.598270 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52060 (local address 127.1.135.254:35949)
0923 22:12:49.598310 (+ 40us) server_negotiation.cc:206] Beginning negotiation
0923 22:12:49.598312 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0923 22:12:49.598408 (+ 96us) server_negotiation.cc:407] Connection header received
0923 22:12:49.598488 (+ 80us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 22:12:49.598489 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 22:12:49.598498 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 22:12:49.598543 (+ 45us) server_negotiation.cc:226] Negotiated authn=SASL
0923 22:12:49.598824 (+ 281us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 22:12:49.599289 (+ 465us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 22:12:49.599978 (+ 689us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 22:12:49.600250 (+ 272us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 22:12:49.600340 (+ 90us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 22:12:49.601309 (+ 969us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0923 22:12:49.601310 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0923 22:12:49.601311 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0923 22:12:49.601317 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0923 22:12:49.602695 (+ 1378us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0923 22:12:49.603027 (+ 332us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0923 22:12:49.603028 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0923 22:12:49.603029 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0923 22:12:49.603079 (+ 50us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0923 22:12:49.603270 (+ 191us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0923 22:12:49.603271 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0923 22:12:49.603271 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0923 22:12:49.603374 (+ 103us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0923 22:12:49.603443 (+ 69us) server_negotiation.cc:1035] Waiting for connection context
0923 22:12:49.603582 (+ 139us) server_negotiation.cc:299] Negotiation successful
0923 22:12:49.603651 (+ 69us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":23}
I20260923 22:12:49.603741 8423 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 22:12:49.598237 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.135.254:35949 (local address 127.0.0.1:52060)
0923 22:12:49.598351 (+ 114us) negotiation.cc:107] Waiting for socket to connect
0923 22:12:49.598360 (+ 9us) client_negotiation.cc:174] Beginning negotiation
0923 22:12:49.598430 (+ 70us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0923 22:12:49.598545 (+ 115us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0923 22:12:49.598546 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0923 22:12:49.598706 (+ 160us) client_negotiation.cc:189] Negotiated authn=SASL
0923 22:12:49.598750 (+ 44us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 22:12:49.598751 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 22:12:49.599363 (+ 612us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 22:12:49.599364 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 22:12:49.599884 (+ 520us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 22:12:49.599886 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 22:12:49.600342 (+ 456us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 22:12:49.600343 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 22:12:49.600416 (+ 73us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 22:12:49.600858 (+ 442us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0923 22:12:49.600863 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0923 22:12:49.601231 (+ 368us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0923 22:12:49.602788 (+ 1557us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0923 22:12:49.602789 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0923 22:12:49.602789 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0923 22:12:49.602949 (+ 160us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0923 22:12:49.603156 (+ 207us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0923 22:12:49.603157 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0923 22:12:49.603158 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0923 22:12:49.603213 (+ 55us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0923 22:12:49.603434 (+ 221us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0923 22:12:49.603435 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0923 22:12:49.603477 (+ 42us) client_negotiation.cc:769] Sending connection context
0923 22:12:49.603556 (+ 79us) client_negotiation.cc:240] Negotiation successful
0923 22:12:49.603652 (+ 96us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":98,"thread_start_us":60,"threads_started":1}
I20260923 22:12:49.603983 7687 tablet_copy_service.cc:140] P 6378d422392b4a74831ae1c0f0692998: Received BeginTabletCopySession request for tablet 00000000000000000000000000000000 from peer d2a5b389b5d04491a85f778e638e056d ({username='oryx', principal='oryx/127.1.135.252@KRBTEST.COM'} at 127.0.0.1:52060)
I20260923 22:12:49.604025 7687 tablet_copy_service.cc:161] P 6378d422392b4a74831ae1c0f0692998: Beginning new tablet copy session on tablet 00000000000000000000000000000000 from peer d2a5b389b5d04491a85f778e638e056d at {username='oryx', principal='oryx/127.1.135.252@KRBTEST.COM'} at 127.0.0.1:52060: session id = d2a5b389b5d04491a85f778e638e056d-00000000000000000000000000000000
I20260923 22:12:49.604518 7687 tablet_copy_source_session.cc:205] T 00000000000000000000000000000000 P 6378d422392b4a74831ae1c0f0692998: Tablet Copy: opened 0 blocks and 1 log segments
I20260923 22:12:49.604910 8319 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total
I20260923 22:12:49.605705 8319 tablet_copy_client.cc:791] T 00000000000000000000000000000000 P d2a5b389b5d04491a85f778e638e056d: tablet copy: Starting download of 0 data blocks...
I20260923 22:12:49.605844 8319 tablet_copy_client.cc:655] T 00000000000000000000000000000000 P d2a5b389b5d04491a85f778e638e056d: tablet copy: Starting download of 1 WAL segments...
I20260923 22:12:49.606549 8319 tablet_copy_client.cc:531] T 00000000000000000000000000000000 P d2a5b389b5d04491a85f778e638e056d: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20260923 22:12:49.607231 7687 tablet_copy_service.cc:342] P 6378d422392b4a74831ae1c0f0692998: Request end of tablet copy session d2a5b389b5d04491a85f778e638e056d-00000000000000000000000000000000 received from {username='oryx', principal='oryx/127.1.135.252@KRBTEST.COM'} at 127.0.0.1:52060
I20260923 22:12:49.607280 7687 tablet_copy_service.cc:433] P 6378d422392b4a74831ae1c0f0692998: ending tablet copy session d2a5b389b5d04491a85f778e638e056d-00000000000000000000000000000000 on tablet 00000000000000000000000000000000 with peer d2a5b389b5d04491a85f778e638e056d
I20260923 22:12:49.609493 8319 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260923 22:12:49.610460 8427 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260923 22:12:49.610533 8424 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260923 22:12:49.612200 8319 server_base.cc:1034] running on GCE node
W20260923 22:12:49.615581 8425 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260923 22:12:49.615794 8319 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260923 22:12:49.615931 8319 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260923 22:12:49.617071 8319 hybrid_clock.cc:648] HybridClock initialized: now 1790201569617044 us; error 44 us; skew 500 ppm
Sep 23 22:12:49 dist-test-slave-s2fj krb5kdc[7622](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790201569, etypes {rep=17 tkt=17 ses=17}, oryx/127.1.135.252@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260923 22:12:49.618331 8319 init.cc:377] Logged in from keytab as oryx/127.1.135.252@KRBTEST.COM (short username oryx)
I20260923 22:12:49.618605 8319 webserver.cc:460] Webserver started at http://127.1.135.252:42139/ using document root <none> and password file <none>
I20260923 22:12:49.618714 8319 fs_manager.cc:359] Metadata directory not provided
I20260923 22:12:49.618760 8319 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260923 22:12:49.619309 8319 fs_manager.cc:709] Time spent opening directory manager: real 0.000s user 0.001s sys 0.000s
I20260923 22:12:49.619781 8434 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260923 22:12:49.619904 8319 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.000s
I20260923 22:12:49.619947 8319 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskRiQe_a/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790201539513534-1567-0/minicluster-data/master-2/data,/tmp/dist-test-taskRiQe_a/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790201539513534-1567-0/minicluster-data/master-2/wal
uuid: "d2a5b389b5d04491a85f778e638e056d"
format_stamp: "Formatted at 2026-09-23 22:12:49 on dist-test-slave-s2fj"
server_key: "014da626af946a7f7b00cd8955be6bd8"
server_key_iv: "4bbadceaa11c86ad605ae9619c4d42ce"
server_key_version: "encryptionkey@0"
I20260923 22:12:49.620002 8319 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskRiQe_a/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790201539513534-1567-0/minicluster-data/master-2/wal
metadata directory: /tmp/dist-test-taskRiQe_a/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790201539513534-1567-0/minicluster-data/master-2/wal
1 data directories: /tmp/dist-test-taskRiQe_a/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790201539513534-1567-0/minicluster-data/master-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260923 22:12:49.631933 8319 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260923 22:12:49.633612 8319 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260923 22:12:49.636588 8319 rpc_server.cc:307] RPC server started. Bound to: 127.1.135.252:42799
I20260923 22:12:49.636905 8319 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskRiQe_a/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790201539513534-1567-0/minicluster-data/master-2/data/info.pb
I20260923 22:12:49.637554 8487 sys_catalog.cc:263] Verifying existing consensus state
I20260923 22:12:49.637939 8487 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P d2a5b389b5d04491a85f778e638e056d: Bootstrap starting.
I20260923 22:12:49.638909 8486 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.1.135.252:42799 every 8 connection(s)
I20260923 22:12:49.640413 8487 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P d2a5b389b5d04491a85f778e638e056d: Bootstrap replayed 1/1 log segments. Stats: ops{read=7 overwritten=0 applied=7 ignored=0} inserts{seen=3 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260923 22:12:49.640620 8487 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P d2a5b389b5d04491a85f778e638e056d: Bootstrap complete.
I20260923 22:12:49.640825 8487 raft_consensus.cc:348] T 00000000000000000000000000000000 P d2a5b389b5d04491a85f778e638e056d [term 1 LEARNER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "6378d422392b4a74831ae1c0f0692998" member_type: VOTER last_known_addr { host: "127.1.135.254" port: 35949 } } peers { permanent_uuid: "b32070aba06045cabad174bee345066a" member_type: VOTER last_known_addr { host: "127.1.135.253" port: 39445 } attrs { promote: false } } peers { permanent_uuid: "d2a5b389b5d04491a85f778e638e056d" member_type: NON_VOTER last_known_addr { host: "127.1.135.252" port: 42799 } attrs { promote: true } }
I20260923 22:12:49.640908 8487 raft_consensus.cc:729] T 00000000000000000000000000000000 P d2a5b389b5d04491a85f778e638e056d [term 1 LEARNER]: Becoming Follower/Learner. State: Replica: d2a5b389b5d04491a85f778e638e056d, State: Initialized, Role: LEARNER
I20260923 22:12:49.640949 8487 consensus_queue.cc:260] T 00000000000000000000000000000000 P d2a5b389b5d04491a85f778e638e056d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 7, Last appended: 1.7, Last appended by leader: 7, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "6378d422392b4a74831ae1c0f0692998" member_type: VOTER last_known_addr { host: "127.1.135.254" port: 35949 } } peers { permanent_uuid: "b32070aba06045cabad174bee345066a" member_type: VOTER last_known_addr { host: "127.1.135.253" port: 39445 } attrs { promote: false } } peers { permanent_uuid: "d2a5b389b5d04491a85f778e638e056d" member_type: NON_VOTER last_known_addr { host: "127.1.135.252" port: 42799 } attrs { promote: true } }
I20260923 22:12:49.641137 8487 sys_catalog.cc:564] T 00000000000000000000000000000000 P d2a5b389b5d04491a85f778e638e056d [sys.catalog]: configured and running, proceeding with master startup.
I20260923 22:12:49.641109 8490 sys_catalog.cc:455] T 00000000000000000000000000000000 P d2a5b389b5d04491a85f778e638e056d [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 committed_config { opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "6378d422392b4a74831ae1c0f0692998" member_type: VOTER last_known_addr { host: "127.1.135.254" port: 35949 } } peers { permanent_uuid: "b32070aba06045cabad174bee345066a" member_type: VOTER last_known_addr { host: "127.1.135.253" port: 39445 } attrs { promote: false } } peers { permanent_uuid: "d2a5b389b5d04491a85f778e638e056d" member_type: NON_VOTER last_known_addr { host: "127.1.135.252" port: 42799 } attrs { promote: true } } }
I20260923 22:12:49.641295 8490 sys_catalog.cc:458] T 00000000000000000000000000000000 P d2a5b389b5d04491a85f778e638e056d [sys.catalog]: This master's current role is: LEARNER
I20260923 22:12:49.642982 8501 catalog_manager.cc:1259] Loaded cluster ID: e03b1c9cbcbb49658f49f0be6e6c5546
I20260923 22:12:49.643064 8501 catalog_manager.cc:1552] T 00000000000000000000000000000000 P d2a5b389b5d04491a85f778e638e056d: loading cluster ID for follower catalog manager: success
I20260923 22:12:49.643790 8501 catalog_manager.cc:1574] T 00000000000000000000000000000000 P d2a5b389b5d04491a85f778e638e056d: acquiring CA information for follower catalog manager: success
I20260923 22:12:49.644027 8501 catalog_manager.cc:1602] T 00000000000000000000000000000000 P d2a5b389b5d04491a85f778e638e056d: importing token verification keys for follower catalog manager: success; most recent TSK sequence number 0
W20260923 22:12:49.691419 8412 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260923 22:12:49.691740 8412 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260923 22:12:49.691797 8412 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260923 22:12:49.693528 8412 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260923 22:12:49.693584 8412 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260923 22:12:49.693624 8412 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.1.135.193
I20260923 22:12:49.696390 8412 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.1.135.212:35751
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskRiQe_a/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790201539513534-1567-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskRiQe_a/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790201539513534-1567-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskRiQe_a/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790201539513534-1567-0/krb5kdc/oryx_127.1.135.193.keytab
--principal=oryx/127.1.135.193
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.1.135.193:44597
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskRiQe_a/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790201539513534-1567-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.1.135.193
--webserver_port=44031
--tserver_master_addrs=127.1.135.254:35949,127.1.135.253:39445,127.1.135.252:42799
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.8412
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.1.135.193
--log_dir=/tmp/dist-test-taskRiQe_a/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790201539513534-1567-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 23 Sep 2026 20:53:37 UTC on ccycloud.quasar-rkviyx.root.comops.site
build id 4668083
I20260923 22:12:49.696638 8412 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260923 22:12:49.696847 8412 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260923 22:12:49.701592 8509 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260923 22:12:49.701613 8507 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260923 22:12:49.701716 8412 server_base.cc:1034] running on GCE node
W20260923 22:12:49.701879 8506 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260923 22:12:49.702065 8412 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260923 22:12:49.702291 8412 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260923 22:12:49.703435 8412 hybrid_clock.cc:648] HybridClock initialized: now 1790201569703423 us; error 28 us; skew 500 ppm
Sep 23 22:12:49 dist-test-slave-s2fj krb5kdc[7622](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790201569, etypes {rep=17 tkt=17 ses=17}, oryx/127.1.135.193@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260923 22:12:49.705807 8412 init.cc:377] Logged in from keytab as oryx/127.1.135.193@KRBTEST.COM (short username oryx)
I20260923 22:12:49.706204 8412 webserver.cc:460] Webserver started at http://127.1.135.193:44031/ using document root <none> and password file <none>
I20260923 22:12:49.706403 8412 fs_manager.cc:359] Metadata directory not provided
I20260923 22:12:49.706476 8412 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260923 22:12:49.707738 8412 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.001s sys 0.000s
I20260923 22:12:49.708292 8516 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260923 22:12:49.708443 8412 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.000s
I20260923 22:12:49.708534 8412 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskRiQe_a/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790201539513534-1567-0/minicluster-data/ts-0/data,/tmp/dist-test-taskRiQe_a/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790201539513534-1567-0/minicluster-data/ts-0/wal
uuid: "e01531ae61ec4025942988b1a6dd62f0"
format_stamp: "Formatted at 2026-09-23 22:12:48 on dist-test-slave-s2fj"
server_key: "6daedd3333b0ce6e7fa0ae7ca7e6e6ea"
server_key_iv: "fbf19d41bd7d3e3c84024fec410a7ad5"
server_key_version: "encryptionkey@0"
I20260923 22:12:49.708840 8412 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskRiQe_a/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790201539513534-1567-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskRiQe_a/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790201539513534-1567-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskRiQe_a/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790201539513534-1567-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260923 22:12:49.721406 8412 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260923 22:12:49.723470 8412 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260923 22:12:49.723620 8412 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260923 22:12:49.723858 8412 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260923 22:12:49.724175 8412 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260923 22:12:49.724222 8412 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260923 22:12:49.724260 8412 ts_tablet_manager.cc:610] Registered 0 tablets
I20260923 22:12:49.724287 8412 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260923 22:12:49.730469 8412 rpc_server.cc:307] RPC server started. Bound to: 127.1.135.193:44597
I20260923 22:12:49.730533 8629 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.1.135.193:44597 every 8 connection(s)
I20260923 22:12:49.730885 8412 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskRiQe_a/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790201539513534-1567-0/minicluster-data/ts-0/data/info.pb
Sep 23 22:12:49 dist-test-slave-s2fj krb5kdc[7622](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790201569, etypes {rep=17 tkt=17 ses=17}, oryx/127.1.135.193@KRBTEST.COM for oryx/127.1.135.254@KRBTEST.COM
Sep 23 22:12:49 dist-test-slave-s2fj krb5kdc[7622](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790201569, etypes {rep=17 tkt=17 ses=17}, oryx/127.1.135.193@KRBTEST.COM for oryx/127.1.135.253@KRBTEST.COM
Sep 23 22:12:49 dist-test-slave-s2fj krb5kdc[7622](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790201569, etypes {rep=17 tkt=17 ses=17}, oryx/127.1.135.193@KRBTEST.COM for oryx/127.1.135.252@KRBTEST.COM
I20260923 22:12:49.740218 1567 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskRiQe_a/build/release/bin/kudu as pid 8412
I20260923 22:12:49.740386 1567 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskRiQe_a/build/release/bin/kudu with pid 7863
I20260923 22:12:49.742022 7713 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 22:12:49.731531 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.135.193:48649 (local address 127.1.135.254:35949)
0923 22:12:49.731581 (+ 50us) server_negotiation.cc:206] Beginning negotiation
0923 22:12:49.731584 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0923 22:12:49.731864 (+ 280us) server_negotiation.cc:407] Connection header received
0923 22:12:49.732866 (+ 1002us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 22:12:49.732867 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 22:12:49.732880 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 22:12:49.732938 (+ 58us) server_negotiation.cc:226] Negotiated authn=SASL
0923 22:12:49.733904 (+ 966us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 22:12:49.734530 (+ 626us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 22:12:49.735577 (+ 1047us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 22:12:49.735791 (+ 214us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 22:12:49.735869 (+ 78us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 22:12:49.738293 (+ 2424us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0923 22:12:49.738295 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0923 22:12:49.738296 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0923 22:12:49.738303 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0923 22:12:49.739857 (+ 1554us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0923 22:12:49.740366 (+ 509us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0923 22:12:49.740368 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0923 22:12:49.740368 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0923 22:12:49.740441 (+ 73us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0923 22:12:49.741303 (+ 862us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0923 22:12:49.741303 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0923 22:12:49.741304 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0923 22:12:49.741455 (+ 151us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0923 22:12:49.741523 (+ 68us) server_negotiation.cc:1035] Waiting for connection context
0923 22:12:49.741848 (+ 325us) server_negotiation.cc:299] Negotiation successful
0923 22:12:49.741932 (+ 84us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":21}
I20260923 22:12:49.742219 8635 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 22:12:49.731540 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.135.254:35949 (local address 127.1.135.193:48649)
0923 22:12:49.731694 (+ 154us) negotiation.cc:107] Waiting for socket to connect
0923 22:12:49.731733 (+ 39us) client_negotiation.cc:174] Beginning negotiation
0923 22:12:49.732761 (+ 1028us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0923 22:12:49.733005 (+ 244us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0923 22:12:49.733009 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0923 22:12:49.733735 (+ 726us) client_negotiation.cc:189] Negotiated authn=SASL
0923 22:12:49.733803 (+ 68us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 22:12:49.733804 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 22:12:49.734627 (+ 823us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 22:12:49.734628 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 22:12:49.735467 (+ 839us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 22:12:49.735469 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 22:12:49.735879 (+ 410us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 22:12:49.735880 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 22:12:49.736001 (+ 121us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 22:12:49.736714 (+ 713us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0923 22:12:49.736719 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0923 22:12:49.738205 (+ 1486us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0923 22:12:49.739938 (+ 1733us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0923 22:12:49.739944 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0923 22:12:49.739946 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0923 22:12:49.740274 (+ 328us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0923 22:12:49.741113 (+ 839us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0923 22:12:49.741113 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0923 22:12:49.741113 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0923 22:12:49.741241 (+ 128us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0923 22:12:49.741512 (+ 271us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0923 22:12:49.741518 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0923 22:12:49.741696 (+ 178us) client_negotiation.cc:769] Sending connection context
0923 22:12:49.741804 (+ 108us) client_negotiation.cc:240] Negotiation successful
0923 22:12:49.741972 (+ 168us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":117,"mutex_wait_us":5,"thread_start_us":99,"threads_started":1}
I20260923 22:12:49.742630 8630 heartbeater.cc:347] Connected to a master server at 127.1.135.254:35949
I20260923 22:12:49.742756 8637 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 22:12:49.731814 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.135.253:39445 (local address 127.1.135.193:39721)
0923 22:12:49.732271 (+ 457us) negotiation.cc:107] Waiting for socket to connect
0923 22:12:49.732278 (+ 7us) client_negotiation.cc:174] Beginning negotiation
0923 22:12:49.732776 (+ 498us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0923 22:12:49.733023 (+ 247us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0923 22:12:49.733024 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0923 22:12:49.733297 (+ 273us) client_negotiation.cc:189] Negotiated authn=SASL
0923 22:12:49.733472 (+ 175us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 22:12:49.733476 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 22:12:49.734317 (+ 841us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 22:12:49.734318 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 22:12:49.735468 (+ 1150us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 22:12:49.735469 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 22:12:49.735937 (+ 468us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 22:12:49.735937 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 22:12:49.736004 (+ 67us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 22:12:49.736652 (+ 648us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0923 22:12:49.736656 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0923 22:12:49.739227 (+ 2571us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0923 22:12:49.741154 (+ 1927us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0923 22:12:49.741155 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0923 22:12:49.741155 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0923 22:12:49.741376 (+ 221us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0923 22:12:49.741573 (+ 197us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0923 22:12:49.741573 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0923 22:12:49.741574 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0923 22:12:49.741628 (+ 54us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0923 22:12:49.741935 (+ 307us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0923 22:12:49.741936 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0923 22:12:49.741974 (+ 38us) client_negotiation.cc:769] Sending connection context
0923 22:12:49.742530 (+ 556us) client_negotiation.cc:240] Negotiation successful
0923 22:12:49.742637 (+ 107us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":446,"mutex_wait_us":1,"thread_start_us":55,"threads_started":1}
I20260923 22:12:49.742787 8316 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 22:12:49.731892 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.135.193:39721 (local address 127.1.135.253:39445)
0923 22:12:49.731963 (+ 71us) server_negotiation.cc:206] Beginning negotiation
0923 22:12:49.731965 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0923 22:12:49.732323 (+ 358us) server_negotiation.cc:407] Connection header received
0923 22:12:49.732926 (+ 603us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 22:12:49.732927 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 22:12:49.732938 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 22:12:49.732984 (+ 46us) server_negotiation.cc:226] Negotiated authn=SASL
0923 22:12:49.733563 (+ 579us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 22:12:49.734199 (+ 636us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 22:12:49.735541 (+ 1342us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 22:12:49.735840 (+ 299us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 22:12:49.735894 (+ 54us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 22:12:49.739327 (+ 3433us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0923 22:12:49.739328 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0923 22:12:49.739328 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0923 22:12:49.739335 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0923 22:12:49.741066 (+ 1731us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0923 22:12:49.741445 (+ 379us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0923 22:12:49.741447 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0923 22:12:49.741447 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0923 22:12:49.741500 (+ 53us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0923 22:12:49.741707 (+ 207us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0923 22:12:49.741708 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0923 22:12:49.741708 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0923 22:12:49.741853 (+ 145us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0923 22:12:49.741909 (+ 56us) server_negotiation.cc:1035] Waiting for connection context
0923 22:12:49.742579 (+ 670us) server_negotiation.cc:299] Negotiation successful
0923 22:12:49.742658 (+ 79us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":45}
I20260923 22:12:49.742764 8630 heartbeater.cc:464] Registering TS with master...
I20260923 22:12:49.743103 8632 heartbeater.cc:347] Connected to a master server at 127.1.135.253:39445
I20260923 22:12:49.743258 8632 heartbeater.cc:464] Registering TS with master...
I20260923 22:12:49.743379 8630 heartbeater.cc:511] Master 127.1.135.254:35949 requested a full tablet report, sending...
I20260923 22:12:49.743379 8632 heartbeater.cc:511] Master 127.1.135.253:39445 requested a full tablet report, sending...
I20260923 22:12:49.743813 7667 ts_manager.cc:194] Re-registered known tserver with Master: e01531ae61ec4025942988b1a6dd62f0 (127.1.135.193:44597)
I20260923 22:12:49.743957 8265 ts_manager.cc:194] Registered new tserver with Master: e01531ae61ec4025942988b1a6dd62f0 (127.1.135.193:44597)
I20260923 22:12:49.744313 7667 master_service.cc:496] Signed X509 certificate for tserver {username='oryx', principal='oryx/127.1.135.193@KRBTEST.COM'} at 127.1.135.193:48649
I20260923 22:12:49.748872 1567 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskRiQe_a/build/release/bin/kudu
/tmp/dist-test-taskRiQe_a/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskRiQe_a/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790201539513534-1567-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskRiQe_a/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790201539513534-1567-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskRiQe_a/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790201539513534-1567-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskRiQe_a/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790201539513534-1567-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.1.135.194:41925
--local_ip_for_outbound_sockets=127.1.135.194
--tserver_master_addrs=127.1.135.254:35949
--webserver_port=42793
--webserver_interface=127.1.135.194
--keytab_file=/tmp/dist-test-taskRiQe_a/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790201539513534-1567-0/krb5kdc/oryx_127.1.135.194.keytab
--principal=oryx/127.1.135.194
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.1.135.212:35751
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--tserver_master_addrs=127.1.135.254:35949,127.1.135.253:39445
--tserver_master_addrs=127.1.135.254:35949,127.1.135.253:39445,127.1.135.252:42799 with env {KRB5CCNAME=/tmp/dist-test-taskRiQe_a/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790201539513534-1567-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskRiQe_a/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790201539513534-1567-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskRiQe_a/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790201539513534-1567-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260923 22:12:49.751004 8634 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 22:12:49.731532 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.135.193:34489 (local address 127.1.135.252:42799)
0923 22:12:49.731669 (+ 137us) server_negotiation.cc:206] Beginning negotiation
0923 22:12:49.731671 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0923 22:12:49.732222 (+ 551us) server_negotiation.cc:407] Connection header received
0923 22:12:49.732911 (+ 689us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 22:12:49.732913 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 22:12:49.732927 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 22:12:49.732983 (+ 56us) server_negotiation.cc:226] Negotiated authn=SASL
0923 22:12:49.733881 (+ 898us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 22:12:49.734667 (+ 786us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 22:12:49.735457 (+ 790us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 22:12:49.735777 (+ 320us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 22:12:49.735853 (+ 76us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 22:12:49.739386 (+ 3533us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0923 22:12:49.739387 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0923 22:12:49.739387 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0923 22:12:49.739393 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0923 22:12:49.749919 (+ 10526us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0923 22:12:49.750253 (+ 334us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0923 22:12:49.750254 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0923 22:12:49.750255 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0923 22:12:49.750317 (+ 62us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0923 22:12:49.750505 (+ 188us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0923 22:12:49.750506 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0923 22:12:49.750506 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0923 22:12:49.750618 (+ 112us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0923 22:12:49.750690 (+ 72us) server_negotiation.cc:1035] Waiting for connection context
0923 22:12:49.750831 (+ 141us) server_negotiation.cc:299] Negotiation successful
0923 22:12:49.750909 (+ 78us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":116,"thread_start_us":87,"threads_started":1}
I20260923 22:12:49.751024 8636 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 22:12:49.731539 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.135.252:42799 (local address 127.1.135.193:34489)
0923 22:12:49.732149 (+ 610us) negotiation.cc:107] Waiting for socket to connect
0923 22:12:49.732158 (+ 9us) client_negotiation.cc:174] Beginning negotiation
0923 22:12:49.732773 (+ 615us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0923 22:12:49.733040 (+ 267us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0923 22:12:49.733041 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0923 22:12:49.733738 (+ 697us) client_negotiation.cc:189] Negotiated authn=SASL
0923 22:12:49.733799 (+ 61us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 22:12:49.733800 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 22:12:49.734739 (+ 939us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 22:12:49.734740 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 22:12:49.735370 (+ 630us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 22:12:49.735372 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 22:12:49.735909 (+ 537us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 22:12:49.735910 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 22:12:49.736004 (+ 94us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 22:12:49.736624 (+ 620us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0923 22:12:49.736634 (+ 10us) client_negotiation.cc:656] calling sasl_client_start()
0923 22:12:49.739304 (+ 2670us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0923 22:12:49.749994 (+ 10690us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0923 22:12:49.749995 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0923 22:12:49.749995 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0923 22:12:49.750162 (+ 167us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0923 22:12:49.750380 (+ 218us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0923 22:12:49.750381 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0923 22:12:49.750381 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0923 22:12:49.750437 (+ 56us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0923 22:12:49.750683 (+ 246us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0923 22:12:49.750684 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0923 22:12:49.750724 (+ 40us) client_negotiation.cc:769] Sending connection context
0923 22:12:49.750796 (+ 72us) client_negotiation.cc:240] Negotiation successful
0923 22:12:49.750907 (+ 111us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":577,"mutex_wait_us":5,"thread_start_us":79,"threads_started":1}
I20260923 22:12:49.751344 8631 heartbeater.cc:347] Connected to a master server at 127.1.135.252:42799
I20260923 22:12:49.751389 8631 heartbeater.cc:464] Registering TS with master...
I20260923 22:12:49.751467 8631 heartbeater.cc:511] Master 127.1.135.252:42799 requested a full tablet report, sending...
I20260923 22:12:49.751787 8451 ts_manager.cc:194] Registered new tserver with Master: e01531ae61ec4025942988b1a6dd62f0 (127.1.135.193:44597)
W20260923 22:12:49.833600 8638 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260923 22:12:49.833813 8638 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260923 22:12:49.833849 8638 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260923 22:12:49.835405 8638 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260923 22:12:49.835466 8638 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260923 22:12:49.835520 8638 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.1.135.194
I20260923 22:12:49.837055 8638 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.1.135.212:35751
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskRiQe_a/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790201539513534-1567-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskRiQe_a/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790201539513534-1567-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskRiQe_a/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790201539513534-1567-0/krb5kdc/oryx_127.1.135.194.keytab
--principal=oryx/127.1.135.194
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.1.135.194:41925
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskRiQe_a/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790201539513534-1567-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.1.135.194
--webserver_port=42793
--tserver_master_addrs=127.1.135.254:35949,127.1.135.253:39445,127.1.135.252:42799
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.8638
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.1.135.194
--log_dir=/tmp/dist-test-taskRiQe_a/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790201539513534-1567-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 23 Sep 2026 20:53:37 UTC on ccycloud.quasar-rkviyx.root.comops.site
build id 4668083
I20260923 22:12:49.837230 8638 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260923 22:12:49.837397 8638 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260923 22:12:49.842312 8644 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260923 22:12:49.842325 8643 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260923 22:12:49.842325 8646 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260923 22:12:49.842512 8638 server_base.cc:1034] running on GCE node
I20260923 22:12:49.842859 8638 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260923 22:12:49.843106 8638 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260923 22:12:49.844270 8638 hybrid_clock.cc:648] HybridClock initialized: now 1790201569844263 us; error 37 us; skew 500 ppm
Sep 23 22:12:49 dist-test-slave-s2fj krb5kdc[7622](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790201569, etypes {rep=17 tkt=17 ses=17}, oryx/127.1.135.194@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260923 22:12:49.846750 8638 init.cc:377] Logged in from keytab as oryx/127.1.135.194@KRBTEST.COM (short username oryx)
I20260923 22:12:49.847211 8638 webserver.cc:460] Webserver started at http://127.1.135.194:42793/ using document root <none> and password file <none>
I20260923 22:12:49.847451 8638 fs_manager.cc:359] Metadata directory not provided
I20260923 22:12:49.847544 8638 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260923 22:12:49.848897 8638 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.001s sys 0.000s
I20260923 22:12:49.849459 8653 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260923 22:12:49.849640 8638 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260923 22:12:49.849747 8638 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskRiQe_a/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790201539513534-1567-0/minicluster-data/ts-1/data,/tmp/dist-test-taskRiQe_a/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790201539513534-1567-0/minicluster-data/ts-1/wal
uuid: "8a9da5f92d7c4186877987e0ddd62caf"
format_stamp: "Formatted at 2026-09-23 22:12:48 on dist-test-slave-s2fj"
server_key: "93df3b32f1b85eb54bcee7bdb0a9b043"
server_key_iv: "3860080038c180c731262ff3d508819a"
server_key_version: "encryptionkey@0"
I20260923 22:12:49.850056 8638 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskRiQe_a/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790201539513534-1567-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskRiQe_a/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790201539513534-1567-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskRiQe_a/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790201539513534-1567-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260923 22:12:49.868490 8638 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260923 22:12:49.870185 8638 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260923 22:12:49.870348 8638 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260923 22:12:49.870589 8638 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260923 22:12:49.870880 8638 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260923 22:12:49.870935 8638 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260923 22:12:49.870975 8638 ts_tablet_manager.cc:610] Registered 0 tablets
I20260923 22:12:49.871029 8638 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260923 22:12:49.876716 8638 rpc_server.cc:307] RPC server started. Bound to: 127.1.135.194:41925
I20260923 22:12:49.876750 8766 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.1.135.194:41925 every 8 connection(s)
I20260923 22:12:49.877188 8638 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskRiQe_a/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790201539513534-1567-0/minicluster-data/ts-1/data/info.pb
I20260923 22:12:49.884054 1567 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskRiQe_a/build/release/bin/kudu as pid 8638
I20260923 22:12:49.884274 1567 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskRiQe_a/build/release/bin/kudu with pid 7997
Sep 23 22:12:49 dist-test-slave-s2fj krb5kdc[7622](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790201569, etypes {rep=17 tkt=17 ses=17}, oryx/127.1.135.194@KRBTEST.COM for oryx/127.1.135.253@KRBTEST.COM
Sep 23 22:12:49 dist-test-slave-s2fj krb5kdc[7622](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790201569, etypes {rep=17 tkt=17 ses=17}, oryx/127.1.135.194@KRBTEST.COM for oryx/127.1.135.252@KRBTEST.COM
Sep 23 22:12:49 dist-test-slave-s2fj krb5kdc[7622](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790201569, etypes {rep=17 tkt=17 ses=17}, oryx/127.1.135.194@KRBTEST.COM for oryx/127.1.135.254@KRBTEST.COM
I20260923 22:12:49.888794 8634 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 22:12:49.878006 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.135.194:56079 (local address 127.1.135.252:42799)
0923 22:12:49.878053 (+ 47us) server_negotiation.cc:206] Beginning negotiation
0923 22:12:49.878055 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0923 22:12:49.878210 (+ 155us) server_negotiation.cc:407] Connection header received
0923 22:12:49.879255 (+ 1045us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 22:12:49.879256 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 22:12:49.879272 (+ 16us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 22:12:49.879323 (+ 51us) server_negotiation.cc:226] Negotiated authn=SASL
0923 22:12:49.879980 (+ 657us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 22:12:49.880524 (+ 544us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 22:12:49.881974 (+ 1450us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 22:12:49.882171 (+ 197us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 22:12:49.882218 (+ 47us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 22:12:49.885460 (+ 3242us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0923 22:12:49.885461 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0923 22:12:49.885462 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0923 22:12:49.885468 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0923 22:12:49.887173 (+ 1705us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0923 22:12:49.887665 (+ 492us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0923 22:12:49.887666 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0923 22:12:49.887667 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0923 22:12:49.887720 (+ 53us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0923 22:12:49.887978 (+ 258us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0923 22:12:49.887978 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0923 22:12:49.887979 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0923 22:12:49.888130 (+ 151us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0923 22:12:49.888173 (+ 43us) server_negotiation.cc:1035] Waiting for connection context
0923 22:12:49.888530 (+ 357us) server_negotiation.cc:299] Negotiation successful
0923 22:12:49.888639 (+ 109us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":21}
I20260923 22:12:49.889518 8316 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 22:12:49.878190 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.135.194:52257 (local address 127.1.135.253:39445)
0923 22:12:49.878232 (+ 42us) server_negotiation.cc:206] Beginning negotiation
0923 22:12:49.878260 (+ 28us) server_negotiation.cc:399] Waiting for connection header
0923 22:12:49.878360 (+ 100us) server_negotiation.cc:407] Connection header received
0923 22:12:49.879269 (+ 909us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 22:12:49.879270 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 22:12:49.879283 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 22:12:49.879316 (+ 33us) server_negotiation.cc:226] Negotiated authn=SASL
0923 22:12:49.879985 (+ 669us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 22:12:49.880765 (+ 780us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 22:12:49.882260 (+ 1495us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 22:12:49.882429 (+ 169us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 22:12:49.882508 (+ 79us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 22:12:49.886047 (+ 3539us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0923 22:12:49.886049 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0923 22:12:49.886049 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0923 22:12:49.886056 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0923 22:12:49.887902 (+ 1846us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0923 22:12:49.888328 (+ 426us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0923 22:12:49.888329 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0923 22:12:49.888330 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0923 22:12:49.888382 (+ 52us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0923 22:12:49.888594 (+ 212us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0923 22:12:49.888595 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0923 22:12:49.888595 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0923 22:12:49.888734 (+ 139us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0923 22:12:49.888778 (+ 44us) server_negotiation.cc:1035] Waiting for connection context
0923 22:12:49.889352 (+ 574us) server_negotiation.cc:299] Negotiation successful
0923 22:12:49.889430 (+ 78us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":22}
I20260923 22:12:49.889562 7713 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 22:12:49.877832 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.135.194:37351 (local address 127.1.135.254:35949)
0923 22:12:49.877878 (+ 46us) server_negotiation.cc:206] Beginning negotiation
0923 22:12:49.877881 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0923 22:12:49.878336 (+ 455us) server_negotiation.cc:407] Connection header received
0923 22:12:49.879270 (+ 934us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 22:12:49.879271 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 22:12:49.879283 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 22:12:49.879316 (+ 33us) server_negotiation.cc:226] Negotiated authn=SASL
0923 22:12:49.879987 (+ 671us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 22:12:49.880761 (+ 774us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 22:12:49.882779 (+ 2018us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 22:12:49.883085 (+ 306us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 22:12:49.883168 (+ 83us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 22:12:49.886426 (+ 3258us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0923 22:12:49.886427 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0923 22:12:49.886427 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0923 22:12:49.886435 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0923 22:12:49.888146 (+ 1711us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0923 22:12:49.888518 (+ 372us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0923 22:12:49.888519 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0923 22:12:49.888520 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0923 22:12:49.888585 (+ 65us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0923 22:12:49.888793 (+ 208us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0923 22:12:49.888794 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0923 22:12:49.888794 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0923 22:12:49.888930 (+ 136us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0923 22:12:49.888975 (+ 45us) server_negotiation.cc:1035] Waiting for connection context
0923 22:12:49.889398 (+ 423us) server_negotiation.cc:299] Negotiation successful
0923 22:12:49.889465 (+ 67us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":22}
I20260923 22:12:49.889640 8771 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 22:12:49.877846 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.135.252:42799 (local address 127.1.135.194:56079)
0923 22:12:49.878094 (+ 248us) negotiation.cc:107] Waiting for socket to connect
0923 22:12:49.878137 (+ 43us) client_negotiation.cc:174] Beginning negotiation
0923 22:12:49.879132 (+ 995us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0923 22:12:49.879389 (+ 257us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0923 22:12:49.879394 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0923 22:12:49.879699 (+ 305us) client_negotiation.cc:189] Negotiated authn=SASL
0923 22:12:49.879881 (+ 182us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 22:12:49.879885 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 22:12:49.880632 (+ 747us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 22:12:49.880633 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 22:12:49.881884 (+ 1251us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 22:12:49.881886 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 22:12:49.882244 (+ 358us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 22:12:49.882245 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 22:12:49.882373 (+ 128us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 22:12:49.883104 (+ 731us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0923 22:12:49.883109 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0923 22:12:49.885309 (+ 2200us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0923 22:12:49.887262 (+ 1953us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0923 22:12:49.887269 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0923 22:12:49.887271 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0923 22:12:49.887569 (+ 298us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0923 22:12:49.887786 (+ 217us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0923 22:12:49.887787 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0923 22:12:49.887788 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0923 22:12:49.887911 (+ 123us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0923 22:12:49.888219 (+ 308us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0923 22:12:49.888225 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0923 22:12:49.888381 (+ 156us) client_negotiation.cc:769] Sending connection context
0923 22:12:49.888882 (+ 501us) client_negotiation.cc:240] Negotiation successful
0923 22:12:49.889364 (+ 482us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":207,"mutex_wait_us":296,"thread_start_us":93,"threads_started":1}
I20260923 22:12:49.889962 8772 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 22:12:49.877868 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.135.254:35949 (local address 127.1.135.194:37351)
0923 22:12:49.878104 (+ 236us) negotiation.cc:107] Waiting for socket to connect
0923 22:12:49.878136 (+ 32us) client_negotiation.cc:174] Beginning negotiation
0923 22:12:49.879157 (+ 1021us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0923 22:12:49.879403 (+ 246us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0923 22:12:49.879404 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0923 22:12:49.879779 (+ 375us) client_negotiation.cc:189] Negotiated authn=SASL
0923 22:12:49.879884 (+ 105us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 22:12:49.879886 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 22:12:49.881983 (+ 2097us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 22:12:49.881984 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 22:12:49.882703 (+ 719us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 22:12:49.882705 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 22:12:49.883181 (+ 476us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 22:12:49.883182 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 22:12:49.883250 (+ 68us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 22:12:49.883671 (+ 421us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0923 22:12:49.883676 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0923 22:12:49.886327 (+ 2651us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0923 22:12:49.888233 (+ 1906us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0923 22:12:49.888234 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0923 22:12:49.888234 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0923 22:12:49.888437 (+ 203us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0923 22:12:49.888659 (+ 222us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0923 22:12:49.888659 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0923 22:12:49.888659 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0923 22:12:49.888718 (+ 59us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0923 22:12:49.889021 (+ 303us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0923 22:12:49.889022 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0923 22:12:49.889277 (+ 255us) client_negotiation.cc:769] Sending connection context
0923 22:12:49.889726 (+ 449us) client_negotiation.cc:240] Negotiation successful
0923 22:12:49.889850 (+ 124us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":222,"mutex_wait_us":240,"thread_start_us":77,"threads_started":1}
I20260923 22:12:49.890363 8767 heartbeater.cc:347] Connected to a master server at 127.1.135.254:35949
I20260923 22:12:49.890462 8767 heartbeater.cc:464] Registering TS with master...
I20260923 22:12:49.890486 8773 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 22:12:49.878138 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.135.253:39445 (local address 127.1.135.194:52257)
0923 22:12:49.878279 (+ 141us) negotiation.cc:107] Waiting for socket to connect
0923 22:12:49.878288 (+ 9us) client_negotiation.cc:174] Beginning negotiation
0923 22:12:49.879157 (+ 869us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0923 22:12:49.879419 (+ 262us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0923 22:12:49.879420 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0923 22:12:49.879776 (+ 356us) client_negotiation.cc:189] Negotiated authn=SASL
0923 22:12:49.879884 (+ 108us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 22:12:49.879886 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 22:12:49.880840 (+ 954us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 22:12:49.880840 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 22:12:49.881884 (+ 1044us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 22:12:49.881886 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 22:12:49.882509 (+ 623us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 22:12:49.882510 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 22:12:49.882581 (+ 71us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 22:12:49.882946 (+ 365us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0923 22:12:49.882955 (+ 9us) client_negotiation.cc:656] calling sasl_client_start()
0923 22:12:49.885951 (+ 2996us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0923 22:12:49.887997 (+ 2046us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0923 22:12:49.887998 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0923 22:12:49.887998 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0923 22:12:49.888248 (+ 250us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0923 22:12:49.888454 (+ 206us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0923 22:12:49.888454 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0923 22:12:49.888455 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0923 22:12:49.888512 (+ 57us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0923 22:12:49.888825 (+ 313us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0923 22:12:49.888826 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0923 22:12:49.889201 (+ 375us) client_negotiation.cc:769] Sending connection context
0923 22:12:49.890270 (+ 1069us) client_negotiation.cc:240] Negotiation successful
0923 22:12:49.890371 (+ 101us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":127,"mutex_wait_us":2,"thread_start_us":66,"threads_started":1}
I20260923 22:12:49.890728 8767 heartbeater.cc:511] Master 127.1.135.254:35949 requested a full tablet report, sending...
I20260923 22:12:49.890931 8769 heartbeater.cc:347] Connected to a master server at 127.1.135.253:39445
I20260923 22:12:49.890973 8769 heartbeater.cc:464] Registering TS with master...
I20260923 22:12:49.891033 8769 heartbeater.cc:511] Master 127.1.135.253:39445 requested a full tablet report, sending...
I20260923 22:12:49.891114 7667 ts_manager.cc:194] Re-registered known tserver with Master: 8a9da5f92d7c4186877987e0ddd62caf (127.1.135.194:41925)
I20260923 22:12:49.891372 8265 ts_manager.cc:194] Registered new tserver with Master: 8a9da5f92d7c4186877987e0ddd62caf (127.1.135.194:41925)
I20260923 22:12:49.891448 8768 heartbeater.cc:347] Connected to a master server at 127.1.135.252:42799
I20260923 22:12:49.891495 8768 heartbeater.cc:464] Registering TS with master...
I20260923 22:12:49.891547 8768 heartbeater.cc:511] Master 127.1.135.252:42799 requested a full tablet report, sending...
I20260923 22:12:49.891659 7667 master_service.cc:496] Signed X509 certificate for tserver {username='oryx', principal='oryx/127.1.135.194@KRBTEST.COM'} at 127.1.135.194:37351
I20260923 22:12:49.891805 8451 ts_manager.cc:194] Registered new tserver with Master: 8a9da5f92d7c4186877987e0ddd62caf (127.1.135.194:41925)
I20260923 22:12:49.892308 1567 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskRiQe_a/build/release/bin/kudu
/tmp/dist-test-taskRiQe_a/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskRiQe_a/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790201539513534-1567-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskRiQe_a/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790201539513534-1567-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskRiQe_a/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790201539513534-1567-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskRiQe_a/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790201539513534-1567-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.1.135.195:33563
--local_ip_for_outbound_sockets=127.1.135.195
--tserver_master_addrs=127.1.135.254:35949
--webserver_port=39665
--webserver_interface=127.1.135.195
--keytab_file=/tmp/dist-test-taskRiQe_a/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790201539513534-1567-0/krb5kdc/oryx_127.1.135.195.keytab
--principal=oryx/127.1.135.195
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.1.135.212:35751
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--tserver_master_addrs=127.1.135.254:35949,127.1.135.253:39445
--tserver_master_addrs=127.1.135.254:35949,127.1.135.253:39445,127.1.135.252:42799 with env {KRB5CCNAME=/tmp/dist-test-taskRiQe_a/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790201539513534-1567-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskRiQe_a/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790201539513534-1567-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskRiQe_a/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790201539513534-1567-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260923 22:12:49.972195 8774 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260923 22:12:49.972400 8774 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260923 22:12:49.972435 8774 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260923 22:12:49.973611 8774 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260923 22:12:49.973675 8774 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260923 22:12:49.973721 8774 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.1.135.195
I20260923 22:12:49.975203 8774 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.1.135.212:35751
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskRiQe_a/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790201539513534-1567-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskRiQe_a/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790201539513534-1567-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskRiQe_a/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790201539513534-1567-0/krb5kdc/oryx_127.1.135.195.keytab
--principal=oryx/127.1.135.195
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.1.135.195:33563
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskRiQe_a/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790201539513534-1567-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.1.135.195
--webserver_port=39665
--tserver_master_addrs=127.1.135.254:35949,127.1.135.253:39445,127.1.135.252:42799
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.8774
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.1.135.195
--log_dir=/tmp/dist-test-taskRiQe_a/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790201539513534-1567-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 23 Sep 2026 20:53:37 UTC on ccycloud.quasar-rkviyx.root.comops.site
build id 4668083
I20260923 22:12:49.975456 8774 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260923 22:12:49.975682 8774 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260923 22:12:49.980742 8782 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260923 22:12:49.980839 8780 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260923 22:12:49.980834 8779 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260923 22:12:49.981119 8774 server_base.cc:1034] running on GCE node
I20260923 22:12:49.981329 8774 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260923 22:12:49.981559 8774 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260923 22:12:49.982707 8774 hybrid_clock.cc:648] HybridClock initialized: now 1790201569982682 us; error 38 us; skew 500 ppm
Sep 23 22:12:49 dist-test-slave-s2fj krb5kdc[7622](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790201569, etypes {rep=17 tkt=17 ses=17}, oryx/127.1.135.195@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260923 22:12:49.985255 8774 init.cc:377] Logged in from keytab as oryx/127.1.135.195@KRBTEST.COM (short username oryx)
I20260923 22:12:49.985756 8774 webserver.cc:460] Webserver started at http://127.1.135.195:39665/ using document root <none> and password file <none>
I20260923 22:12:49.985999 8774 fs_manager.cc:359] Metadata directory not provided
I20260923 22:12:49.986078 8774 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260923 22:12:49.987656 8774 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.002s sys 0.000s
I20260923 22:12:49.988292 8789 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260923 22:12:49.988464 8774 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260923 22:12:49.988570 8774 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskRiQe_a/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790201539513534-1567-0/minicluster-data/ts-2/data,/tmp/dist-test-taskRiQe_a/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790201539513534-1567-0/minicluster-data/ts-2/wal
uuid: "4d0b914e7ab746e7b17789e7b32aca0a"
format_stamp: "Formatted at 2026-09-23 22:12:49 on dist-test-slave-s2fj"
server_key: "7b659ba92ef47ef8579fcaedc6163353"
server_key_iv: "d45dcb8288a91c35a1745c34b2a22713"
server_key_version: "encryptionkey@0"
I20260923 22:12:49.988895 8774 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskRiQe_a/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790201539513534-1567-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskRiQe_a/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790201539513534-1567-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskRiQe_a/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790201539513534-1567-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260923 22:12:49.989960 8634 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 22:12:49.987203 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43370 (local address 127.1.135.252:42799)
0923 22:12:49.987248 (+ 45us) server_negotiation.cc:206] Beginning negotiation
0923 22:12:49.987250 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0923 22:12:49.987258 (+ 8us) server_negotiation.cc:407] Connection header received
0923 22:12:49.987315 (+ 57us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 22:12:49.987316 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 22:12:49.987334 (+ 18us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 22:12:49.987412 (+ 78us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0923 22:12:49.987535 (+ 123us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 22:12:49.988121 (+ 586us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 22:12:49.989250 (+ 1129us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 22:12:49.989634 (+ 384us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 22:12:49.989717 (+ 83us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 22:12:49.989781 (+ 64us) server_negotiation.cc:1035] Waiting for connection context
0923 22:12:49.989860 (+ 79us) server_negotiation.cc:299] Negotiation successful
0923 22:12:49.989892 (+ 32us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":22}
I20260923 22:12:49.989964 8225 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 22:12:49.987136 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.135.252:42799 (local address 127.0.0.1:43370)
0923 22:12:49.987195 (+ 59us) negotiation.cc:107] Waiting for socket to connect
0923 22:12:49.987208 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0923 22:12:49.987261 (+ 53us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0923 22:12:49.987418 (+ 157us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0923 22:12:49.987419 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0923 22:12:49.987421 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0923 22:12:49.987472 (+ 51us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 22:12:49.987474 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 22:12:49.988200 (+ 726us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 22:12:49.988201 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 22:12:49.989159 (+ 958us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 22:12:49.989162 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 22:12:49.989728 (+ 566us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 22:12:49.989729 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 22:12:49.989795 (+ 66us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 22:12:49.989798 (+ 3us) client_negotiation.cc:769] Sending connection context
0923 22:12:49.989854 (+ 56us) client_negotiation.cc:240] Negotiation successful
0923 22:12:49.989900 (+ 46us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":31}
I20260923 22:12:49.990249 8461 raft_consensus.cc:1203] T 00000000000000000000000000000000 P d2a5b389b5d04491a85f778e638e056d [term 1 LEARNER]: Deduplicated request from leader. Original: 1.6->[1.7-1.7] Dedup: 1.7->[]
I20260923 22:12:49.990447 8490 sys_catalog.cc:455] T 00000000000000000000000000000000 P d2a5b389b5d04491a85f778e638e056d [sys.catalog]: SysCatalogTable state changed. Reason: New leader 6378d422392b4a74831ae1c0f0692998. Latest consensus state: current_term: 1 leader_uuid: "6378d422392b4a74831ae1c0f0692998" committed_config { opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "6378d422392b4a74831ae1c0f0692998" member_type: VOTER last_known_addr { host: "127.1.135.254" port: 35949 } } peers { permanent_uuid: "b32070aba06045cabad174bee345066a" member_type: VOTER last_known_addr { host: "127.1.135.253" port: 39445 } attrs { promote: false } } peers { permanent_uuid: "d2a5b389b5d04491a85f778e638e056d" member_type: NON_VOTER last_known_addr { host: "127.1.135.252" port: 42799 } attrs { promote: true } } }
I20260923 22:12:49.990543 8490 sys_catalog.cc:458] T 00000000000000000000000000000000 P d2a5b389b5d04491a85f778e638e056d [sys.catalog]: This master's current role is: LEARNER
I20260923 22:12:50.007673 8774 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260923 22:12:50.009441 8774 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260923 22:12:50.009588 8774 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260923 22:12:50.009816 8774 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260923 22:12:50.010107 8774 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260923 22:12:50.010138 8774 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260923 22:12:50.010159 8774 ts_tablet_manager.cc:610] Registered 0 tablets
I20260923 22:12:50.010205 8774 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260923 22:12:50.015470 8774 rpc_server.cc:307] RPC server started. Bound to: 127.1.135.195:33563
I20260923 22:12:50.015630 8902 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.1.135.195:33563 every 8 connection(s)
I20260923 22:12:50.015883 8774 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskRiQe_a/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790201539513534-1567-0/minicluster-data/ts-2/data/info.pb
I20260923 22:12:50.016585 1567 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskRiQe_a/build/release/bin/kudu as pid 8774
Sep 23 22:12:50 dist-test-slave-s2fj krb5kdc[7622](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790201569, etypes {rep=17 tkt=17 ses=17}, oryx/127.1.135.195@KRBTEST.COM for oryx/127.1.135.253@KRBTEST.COM
Sep 23 22:12:50 dist-test-slave-s2fj krb5kdc[7622](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790201569, etypes {rep=17 tkt=17 ses=17}, oryx/127.1.135.195@KRBTEST.COM for oryx/127.1.135.254@KRBTEST.COM
I20260923 22:12:50.026618 8915 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 22:12:50.017647 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43382 (local address 127.1.135.252:42799)
0923 22:12:50.017837 (+ 190us) server_negotiation.cc:206] Beginning negotiation
0923 22:12:50.017839 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0923 22:12:50.018201 (+ 362us) server_negotiation.cc:407] Connection header received
0923 22:12:50.018303 (+ 102us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 22:12:50.018304 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 22:12:50.018316 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 22:12:50.018356 (+ 40us) server_negotiation.cc:226] Negotiated authn=SASL
0923 22:12:50.019150 (+ 794us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 22:12:50.019942 (+ 792us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 22:12:50.021126 (+ 1184us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 22:12:50.021405 (+ 279us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 22:12:50.021444 (+ 39us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 22:12:50.023387 (+ 1943us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0923 22:12:50.023388 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0923 22:12:50.023388 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0923 22:12:50.023394 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0923 22:12:50.025283 (+ 1889us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0923 22:12:50.025700 (+ 417us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0923 22:12:50.025701 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0923 22:12:50.025702 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0923 22:12:50.025761 (+ 59us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0923 22:12:50.026041 (+ 280us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0923 22:12:50.026042 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0923 22:12:50.026042 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0923 22:12:50.026170 (+ 128us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0923 22:12:50.026240 (+ 70us) server_negotiation.cc:1035] Waiting for connection context
0923 22:12:50.026423 (+ 183us) server_negotiation.cc:299] Negotiation successful
0923 22:12:50.026495 (+ 72us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":172,"thread_start_us":61,"threads_started":1}
I20260923 22:12:50.026961 8916 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 22:12:50.017644 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52072 (local address 127.1.135.254:35949)
0923 22:12:50.017788 (+ 144us) server_negotiation.cc:206] Beginning negotiation
0923 22:12:50.017790 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0923 22:12:50.017974 (+ 184us) server_negotiation.cc:407] Connection header received
0923 22:12:50.018099 (+ 125us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 22:12:50.018100 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 22:12:50.018111 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 22:12:50.018172 (+ 61us) server_negotiation.cc:226] Negotiated authn=SASL
0923 22:12:50.019144 (+ 972us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 22:12:50.020008 (+ 864us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 22:12:50.021169 (+ 1161us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 22:12:50.021449 (+ 280us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 22:12:50.021526 (+ 77us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 22:12:50.023306 (+ 1780us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0923 22:12:50.023307 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0923 22:12:50.023308 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0923 22:12:50.023314 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0923 22:12:50.025599 (+ 2285us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0923 22:12:50.025982 (+ 383us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0923 22:12:50.025983 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0923 22:12:50.025983 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0923 22:12:50.026042 (+ 59us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0923 22:12:50.026265 (+ 223us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0923 22:12:50.026266 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0923 22:12:50.026266 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0923 22:12:50.026390 (+ 124us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0923 22:12:50.026476 (+ 86us) server_negotiation.cc:1035] Waiting for connection context
0923 22:12:50.026824 (+ 348us) server_negotiation.cc:299] Negotiation successful
0923 22:12:50.026899 (+ 75us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":46,"server-negotiator.queue_time_us":117,"thread_start_us":83,"threads_started":1}
I20260923 22:12:50.030625 8909 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 22:12:50.016909 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.135.254:35949 (local address 127.1.135.195:32953)
0923 22:12:50.017140 (+ 231us) negotiation.cc:107] Waiting for socket to connect
0923 22:12:50.017148 (+ 8us) client_negotiation.cc:174] Beginning negotiation
0923 22:12:50.017743 (+ 595us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0923 22:12:50.018528 (+ 785us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0923 22:12:50.018533 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0923 22:12:50.018907 (+ 374us) client_negotiation.cc:189] Negotiated authn=SASL
0923 22:12:50.019180 (+ 273us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 22:12:50.019185 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 22:12:50.022881 (+ 3696us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 22:12:50.022881 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 22:12:50.023707 (+ 826us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 22:12:50.023709 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 22:12:50.024221 (+ 512us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 22:12:50.024222 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 22:12:50.024282 (+ 60us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 22:12:50.024782 (+ 500us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0923 22:12:50.024787 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0923 22:12:50.026924 (+ 2137us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0923 22:12:50.029038 (+ 2114us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0923 22:12:50.029044 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0923 22:12:50.029046 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0923 22:12:50.029342 (+ 296us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0923 22:12:50.029552 (+ 210us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0923 22:12:50.029553 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0923 22:12:50.029554 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0923 22:12:50.029675 (+ 121us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0923 22:12:50.029948 (+ 273us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0923 22:12:50.029954 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0923 22:12:50.030118 (+ 164us) client_negotiation.cc:769] Sending connection context
0923 22:12:50.030224 (+ 106us) client_negotiation.cc:240] Negotiation successful
0923 22:12:50.030387 (+ 163us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":213,"thread_start_us":59,"threads_started":1}
I20260923 22:12:50.030869 8907 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 22:12:50.016628 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.135.253:39445 (local address 127.1.135.195:33021)
0923 22:12:50.016907 (+ 279us) negotiation.cc:107] Waiting for socket to connect
0923 22:12:50.016915 (+ 8us) client_negotiation.cc:174] Beginning negotiation
0923 22:12:50.017614 (+ 699us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0923 22:12:50.020109 (+ 2495us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0923 22:12:50.020111 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0923 22:12:50.020290 (+ 179us) client_negotiation.cc:189] Negotiated authn=SASL
0923 22:12:50.020346 (+ 56us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 22:12:50.020347 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 22:12:50.021215 (+ 868us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 22:12:50.021216 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 22:12:50.022303 (+ 1087us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 22:12:50.022305 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 22:12:50.023832 (+ 1527us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 22:12:50.023833 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 22:12:50.023957 (+ 124us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 22:12:50.024428 (+ 471us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0923 22:12:50.024454 (+ 26us) client_negotiation.cc:656] calling sasl_client_start()
0923 22:12:50.027464 (+ 3010us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0923 22:12:50.029841 (+ 2377us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0923 22:12:50.029842 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0923 22:12:50.029842 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0923 22:12:50.030025 (+ 183us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0923 22:12:50.030257 (+ 232us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0923 22:12:50.030258 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0923 22:12:50.030258 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0923 22:12:50.030313 (+ 55us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0923 22:12:50.030568 (+ 255us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0923 22:12:50.030569 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0923 22:12:50.030613 (+ 44us) client_negotiation.cc:769] Sending connection context
0923 22:12:50.030676 (+ 63us) client_negotiation.cc:240] Negotiation successful
0923 22:12:50.030769 (+ 93us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":255,"thread_start_us":94,"threads_started":1}
I20260923 22:12:50.031002 7713 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 22:12:50.017186 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.135.195:32953 (local address 127.1.135.254:35949)
0923 22:12:50.017247 (+ 61us) server_negotiation.cc:206] Beginning negotiation
0923 22:12:50.017250 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0923 22:12:50.017258 (+ 8us) server_negotiation.cc:407] Connection header received
0923 22:12:50.017956 (+ 698us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 22:12:50.017957 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 22:12:50.017969 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 22:12:50.018017 (+ 48us) server_negotiation.cc:226] Negotiated authn=SASL
0923 22:12:50.019268 (+ 1251us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 22:12:50.019972 (+ 704us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 22:12:50.023805 (+ 3833us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 22:12:50.024108 (+ 303us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 22:12:50.024173 (+ 65us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 22:12:50.027020 (+ 2847us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0923 22:12:50.027021 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0923 22:12:50.027022 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0923 22:12:50.027037 (+ 15us) server_negotiation.cc:922] calling sasl_server_start()
0923 22:12:50.028958 (+ 1921us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0923 22:12:50.029427 (+ 469us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0923 22:12:50.029428 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0923 22:12:50.029429 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0923 22:12:50.029482 (+ 53us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0923 22:12:50.029745 (+ 263us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0923 22:12:50.029746 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0923 22:12:50.029746 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0923 22:12:50.029871 (+ 125us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0923 22:12:50.029925 (+ 54us) server_negotiation.cc:1035] Waiting for connection context
0923 22:12:50.030849 (+ 924us) server_negotiation.cc:299] Negotiation successful
0923 22:12:50.030920 (+ 71us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":2,"server-negotiator.queue_time_us":34}
I20260923 22:12:50.031153 8316 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 22:12:50.017020 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.135.195:33021 (local address 127.1.135.253:39445)
0923 22:12:50.017128 (+ 108us) server_negotiation.cc:206] Beginning negotiation
0923 22:12:50.017131 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0923 22:12:50.017140 (+ 9us) server_negotiation.cc:407] Connection header received
0923 22:12:50.017828 (+ 688us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 22:12:50.017829 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 22:12:50.017842 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 22:12:50.017898 (+ 56us) server_negotiation.cc:226] Negotiated authn=SASL
0923 22:12:50.020438 (+ 2540us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 22:12:50.021112 (+ 674us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 22:12:50.022426 (+ 1314us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 22:12:50.022767 (+ 341us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 22:12:50.022831 (+ 64us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 22:12:50.027551 (+ 4720us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0923 22:12:50.027552 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0923 22:12:50.027553 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0923 22:12:50.027560 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0923 22:12:50.029770 (+ 2210us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0923 22:12:50.030096 (+ 326us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0923 22:12:50.030097 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0923 22:12:50.030097 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0923 22:12:50.030192 (+ 95us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0923 22:12:50.030375 (+ 183us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0923 22:12:50.030376 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0923 22:12:50.030376 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0923 22:12:50.030491 (+ 115us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0923 22:12:50.030544 (+ 53us) server_negotiation.cc:1035] Waiting for connection context
0923 22:12:50.030990 (+ 446us) server_negotiation.cc:299] Negotiation successful
0923 22:12:50.031060 (+ 70us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":83}
I20260923 22:12:50.031334 8903 heartbeater.cc:347] Connected to a master server at 127.1.135.254:35949
I20260923 22:12:50.031420 8903 heartbeater.cc:464] Registering TS with master...
I20260923 22:12:50.029309 7667 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:52072:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20260923 22:12:50.031497 8905 heartbeater.cc:347] Connected to a master server at 127.1.135.253:39445
I20260923 22:12:50.031526 8905 heartbeater.cc:464] Registering TS with master...
I20260923 22:12:50.031608 8903 heartbeater.cc:511] Master 127.1.135.254:35949 requested a full tablet report, sending...
I20260923 22:12:50.031607 8905 heartbeater.cc:511] Master 127.1.135.253:39445 requested a full tablet report, sending...
W20260923 22:12:50.031680 7667 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260923 22:12:50.031893 8919 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 22:12:50.018059 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53520 (local address 127.1.135.253:39445)
0923 22:12:50.018151 (+ 92us) server_negotiation.cc:206] Beginning negotiation
0923 22:12:50.018153 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0923 22:12:50.018161 (+ 8us) server_negotiation.cc:407] Connection header received
0923 22:12:50.018186 (+ 25us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 22:12:50.018187 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 22:12:50.018199 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 22:12:50.018257 (+ 58us) server_negotiation.cc:226] Negotiated authn=SASL
0923 22:12:50.019156 (+ 899us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 22:12:50.019949 (+ 793us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 22:12:50.021105 (+ 1156us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 22:12:50.021398 (+ 293us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 22:12:50.021453 (+ 55us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 22:12:50.023268 (+ 1815us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0923 22:12:50.023269 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0923 22:12:50.023269 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0923 22:12:50.023276 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0923 22:12:50.029523 (+ 6247us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0923 22:12:50.029871 (+ 348us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0923 22:12:50.029872 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0923 22:12:50.029873 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0923 22:12:50.029924 (+ 51us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0923 22:12:50.030097 (+ 173us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0923 22:12:50.030098 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0923 22:12:50.030098 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0923 22:12:50.030232 (+ 134us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0923 22:12:50.030310 (+ 78us) server_negotiation.cc:1035] Waiting for connection context
0923 22:12:50.031419 (+ 1109us) server_negotiation.cc:299] Negotiation successful
0923 22:12:50.031815 (+ 396us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":75,"thread_start_us":53,"threads_started":1}
I20260923 22:12:50.032187 7666 ts_manager.cc:194] Re-registered known tserver with Master: 4d0b914e7ab746e7b17789e7b32aca0a (127.1.135.195:33563)
I20260923 22:12:50.032582 8415 sys_catalog.cc:455] T 00000000000000000000000000000000 P 6378d422392b4a74831ae1c0f0692998 [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "6378d422392b4a74831ae1c0f0692998" committed_config { opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "6378d422392b4a74831ae1c0f0692998" member_type: VOTER last_known_addr { host: "127.1.135.254" port: 35949 } } peers { permanent_uuid: "b32070aba06045cabad174bee345066a" member_type: VOTER last_known_addr { host: "127.1.135.253" port: 39445 } attrs { promote: false } } peers { permanent_uuid: "d2a5b389b5d04491a85f778e638e056d" member_type: NON_VOTER last_known_addr { host: "127.1.135.252" port: 42799 } attrs { promote: true } } }
I20260923 22:12:50.032630 7666 master_service.cc:496] Signed X509 certificate for tserver {username='oryx', principal='oryx/127.1.135.195@KRBTEST.COM'} at 127.1.135.195:32953
I20260923 22:12:50.032684 8415 sys_catalog.cc:458] T 00000000000000000000000000000000 P 6378d422392b4a74831ae1c0f0692998 [sys.catalog]: This master's current role is: LEADER
I20260923 22:12:50.032042 8265 ts_manager.cc:194] Registered new tserver with Master: 4d0b914e7ab746e7b17789e7b32aca0a (127.1.135.195:33563)
I20260923 22:12:50.033828 8227 raft_consensus.cc:1050] T 00000000000000000000000000000000 P 6378d422392b4a74831ae1c0f0692998: attempting to promote NON_VOTER d2a5b389b5d04491a85f778e638e056d to VOTER
I20260923 22:12:50.033946 8227 consensus_queue.cc:237] T 00000000000000000000000000000000 P 6378d422392b4a74831ae1c0f0692998 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 1.8, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 9 OBSOLETE_local: true peers { permanent_uuid: "6378d422392b4a74831ae1c0f0692998" member_type: VOTER last_known_addr { host: "127.1.135.254" port: 35949 } } peers { permanent_uuid: "b32070aba06045cabad174bee345066a" member_type: VOTER last_known_addr { host: "127.1.135.253" port: 39445 } attrs { promote: false } } peers { permanent_uuid: "d2a5b389b5d04491a85f778e638e056d" member_type: VOTER last_known_addr { host: "127.1.135.252" port: 42799 } attrs { promote: false } }
I20260923 22:12:50.034258 8275 raft_consensus.cc:1261] T 00000000000000000000000000000000 P b32070aba06045cabad174bee345066a [term 1 FOLLOWER]: Refusing update from remote peer 6378d422392b4a74831ae1c0f0692998: Log matching property violated. Preceding OpId in replica: term: 1 index: 8. Preceding OpId from leader: term: 1 index: 9. (index mismatch)
I20260923 22:12:50.034415 8415 consensus_queue.cc:1035] T 00000000000000000000000000000000 P 6378d422392b4a74831ae1c0f0692998 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b32070aba06045cabad174bee345066a" member_type: VOTER last_known_addr { host: "127.1.135.253" port: 39445 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.000s
I20260923 22:12:50.034849 8460 raft_consensus.cc:1261] T 00000000000000000000000000000000 P d2a5b389b5d04491a85f778e638e056d [term 1 LEARNER]: Refusing update from remote peer 6378d422392b4a74831ae1c0f0692998: Log matching property violated. Preceding OpId in replica: term: 1 index: 8. Preceding OpId from leader: term: 1 index: 9. (index mismatch)
I20260923 22:12:50.034909 8415 sys_catalog.cc:455] T 00000000000000000000000000000000 P 6378d422392b4a74831ae1c0f0692998 [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "6378d422392b4a74831ae1c0f0692998" committed_config { opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "6378d422392b4a74831ae1c0f0692998" member_type: VOTER last_known_addr { host: "127.1.135.254" port: 35949 } } peers { permanent_uuid: "b32070aba06045cabad174bee345066a" member_type: VOTER last_known_addr { host: "127.1.135.253" port: 39445 } attrs { promote: false } } peers { permanent_uuid: "d2a5b389b5d04491a85f778e638e056d" member_type: NON_VOTER last_known_addr { host: "127.1.135.252" port: 42799 } attrs { promote: true } } } pending_config { opid_index: 9 OBSOLETE_local: true peers { permanent_uuid: "6378d422392b4a74831ae1c0f0692998" member_type: VOTER last_known_addr { host: "127.1.135.254" port: 35949 } } peers { permanent_uuid: "b32070aba06045cabad174bee345066a" member_type: VOTER last_known_addr { host: "127.1.135.253" port: 39445 } attrs { promote: false } } peers { permanent_uuid: "d2a5b389b5d04491a85f778e638e056d" member_type: VOTER last_known_addr { host: "127.1.135.252" port: 42799 } attrs { promote: false } } }
I20260923 22:12:50.035020 8415 sys_catalog.cc:458] T 00000000000000000000000000000000 P 6378d422392b4a74831ae1c0f0692998 [sys.catalog]: This master's current role is: LEADER
I20260923 22:12:50.035125 8415 consensus_queue.cc:1035] T 00000000000000000000000000000000 P 6378d422392b4a74831ae1c0f0692998 [LEADER]: Connected to new peer: Peer: permanent_uuid: "d2a5b389b5d04491a85f778e638e056d" member_type: VOTER last_known_addr { host: "127.1.135.252" port: 42799 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.000s
I20260923 22:12:50.035243 8415 raft_consensus.cc:2932] T 00000000000000000000000000000000 P 6378d422392b4a74831ae1c0f0692998 [term 1 LEADER]: Committing config change with OpId 1.9: config changed from index 7 to 9, d2a5b389b5d04491a85f778e638e056d (127.1.135.252) changed from NON_VOTER to VOTER. New config: { opid_index: 9 OBSOLETE_local: true peers { permanent_uuid: "6378d422392b4a74831ae1c0f0692998" member_type: VOTER last_known_addr { host: "127.1.135.254" port: 35949 } } peers { permanent_uuid: "b32070aba06045cabad174bee345066a" member_type: VOTER last_known_addr { host: "127.1.135.253" port: 39445 } attrs { promote: false } } peers { permanent_uuid: "d2a5b389b5d04491a85f778e638e056d" member_type: VOTER last_known_addr { host: "127.1.135.252" port: 42799 } attrs { promote: false } } }
Sep 23 22:12:50 dist-test-slave-s2fj krb5kdc[7622](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790201569, etypes {rep=17 tkt=17 ses=17}, oryx/127.1.135.195@KRBTEST.COM for oryx/127.1.135.252@KRBTEST.COM
I20260923 22:12:50.035985 8460 raft_consensus.cc:2932] T 00000000000000000000000000000000 P d2a5b389b5d04491a85f778e638e056d [term 1 FOLLOWER]: Committing config change with OpId 1.9: config changed from index 7 to 9, d2a5b389b5d04491a85f778e638e056d (127.1.135.252) changed from NON_VOTER to VOTER. New config: { opid_index: 9 OBSOLETE_local: true peers { permanent_uuid: "6378d422392b4a74831ae1c0f0692998" member_type: VOTER last_known_addr { host: "127.1.135.254" port: 35949 } } peers { permanent_uuid: "b32070aba06045cabad174bee345066a" member_type: VOTER last_known_addr { host: "127.1.135.253" port: 39445 } attrs { promote: false } } peers { permanent_uuid: "d2a5b389b5d04491a85f778e638e056d" member_type: VOTER last_known_addr { host: "127.1.135.252" port: 42799 } attrs { promote: false } } }
I20260923 22:12:50.036290 8415 sys_catalog.cc:455] T 00000000000000000000000000000000 P 6378d422392b4a74831ae1c0f0692998 [sys.catalog]: SysCatalogTable state changed. Reason: Config change replication complete. Latest consensus state: current_term: 1 leader_uuid: "6378d422392b4a74831ae1c0f0692998" committed_config { opid_index: 9 OBSOLETE_local: true peers { permanent_uuid: "6378d422392b4a74831ae1c0f0692998" member_type: VOTER last_known_addr { host: "127.1.135.254" port: 35949 } } peers { permanent_uuid: "b32070aba06045cabad174bee345066a" member_type: VOTER last_known_addr { host: "127.1.135.253" port: 39445 } attrs { promote: false } } peers { permanent_uuid: "d2a5b389b5d04491a85f778e638e056d" member_type: VOTER last_known_addr { host: "127.1.135.252" port: 42799 } attrs { promote: false } } }
I20260923 22:12:50.036387 8415 sys_catalog.cc:458] T 00000000000000000000000000000000 P 6378d422392b4a74831ae1c0f0692998 [sys.catalog]: This master's current role is: LEADER
I20260923 22:12:50.036445 8226 sys_catalog.cc:455] T 00000000000000000000000000000000 P 6378d422392b4a74831ae1c0f0692998 [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "6378d422392b4a74831ae1c0f0692998" committed_config { opid_index: 9 OBSOLETE_local: true peers { permanent_uuid: "6378d422392b4a74831ae1c0f0692998" member_type: VOTER last_known_addr { host: "127.1.135.254" port: 35949 } } peers { permanent_uuid: "b32070aba06045cabad174bee345066a" member_type: VOTER last_known_addr { host: "127.1.135.253" port: 39445 } attrs { promote: false } } peers { permanent_uuid: "d2a5b389b5d04491a85f778e638e056d" member_type: VOTER last_known_addr { host: "127.1.135.252" port: 42799 } attrs { promote: false } } }
I20260923 22:12:50.036543 8226 sys_catalog.cc:458] T 00000000000000000000000000000000 P 6378d422392b4a74831ae1c0f0692998 [sys.catalog]: This master's current role is: LEADER
I20260923 22:12:50.036711 8275 raft_consensus.cc:2932] T 00000000000000000000000000000000 P b32070aba06045cabad174bee345066a [term 1 FOLLOWER]: Committing config change with OpId 1.9: config changed from index 7 to 9, d2a5b389b5d04491a85f778e638e056d (127.1.135.252) changed from NON_VOTER to VOTER. New config: { opid_index: 9 OBSOLETE_local: true peers { permanent_uuid: "6378d422392b4a74831ae1c0f0692998" member_type: VOTER last_known_addr { host: "127.1.135.254" port: 35949 } } peers { permanent_uuid: "b32070aba06045cabad174bee345066a" member_type: VOTER last_known_addr { host: "127.1.135.253" port: 39445 } attrs { promote: false } } peers { permanent_uuid: "d2a5b389b5d04491a85f778e638e056d" member_type: VOTER last_known_addr { host: "127.1.135.252" port: 42799 } attrs { promote: false } } }
I20260923 22:12:50.037163 8490 sys_catalog.cc:455] T 00000000000000000000000000000000 P d2a5b389b5d04491a85f778e638e056d [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "6378d422392b4a74831ae1c0f0692998" committed_config { opid_index: 9 OBSOLETE_local: true peers { permanent_uuid: "6378d422392b4a74831ae1c0f0692998" member_type: VOTER last_known_addr { host: "127.1.135.254" port: 35949 } } peers { permanent_uuid: "b32070aba06045cabad174bee345066a" member_type: VOTER last_known_addr { host: "127.1.135.253" port: 39445 } attrs { promote: false } } peers { permanent_uuid: "d2a5b389b5d04491a85f778e638e056d" member_type: VOTER last_known_addr { host: "127.1.135.252" port: 42799 } attrs { promote: false } } }
I20260923 22:12:50.037254 8490 sys_catalog.cc:458] T 00000000000000000000000000000000 P d2a5b389b5d04491a85f778e638e056d [sys.catalog]: This master's current role is: FOLLOWER
I20260923 22:12:50.037657 8414 sys_catalog.cc:455] T 00000000000000000000000000000000 P b32070aba06045cabad174bee345066a [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "6378d422392b4a74831ae1c0f0692998" committed_config { opid_index: 9 OBSOLETE_local: true peers { permanent_uuid: "6378d422392b4a74831ae1c0f0692998" member_type: VOTER last_known_addr { host: "127.1.135.254" port: 35949 } } peers { permanent_uuid: "b32070aba06045cabad174bee345066a" member_type: VOTER last_known_addr { host: "127.1.135.253" port: 39445 } attrs { promote: false } } peers { permanent_uuid: "d2a5b389b5d04491a85f778e638e056d" member_type: VOTER last_known_addr { host: "127.1.135.252" port: 42799 } attrs { promote: false } } }
I20260923 22:12:50.037725 8414 sys_catalog.cc:458] T 00000000000000000000000000000000 P b32070aba06045cabad174bee345066a [sys.catalog]: This master's current role is: FOLLOWER
I20260923 22:12:50.041121 8634 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 22:12:50.016759 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.135.195:41895 (local address 127.1.135.252:42799)
0923 22:12:50.016864 (+ 105us) server_negotiation.cc:206] Beginning negotiation
0923 22:12:50.016866 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0923 22:12:50.016987 (+ 121us) server_negotiation.cc:407] Connection header received
0923 22:12:50.024235 (+ 7248us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 22:12:50.024236 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 22:12:50.024250 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 22:12:50.024280 (+ 30us) server_negotiation.cc:226] Negotiated authn=SASL
0923 22:12:50.025860 (+ 1580us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 22:12:50.026538 (+ 678us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 22:12:50.028708 (+ 2170us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 22:12:50.028902 (+ 194us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 22:12:50.028983 (+ 81us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 22:12:50.038332 (+ 9349us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0923 22:12:50.038333 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0923 22:12:50.038334 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0923 22:12:50.038340 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0923 22:12:50.039870 (+ 1530us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0923 22:12:50.040298 (+ 428us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0923 22:12:50.040299 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0923 22:12:50.040299 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0923 22:12:50.040355 (+ 56us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0923 22:12:50.040563 (+ 208us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0923 22:12:50.040564 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0923 22:12:50.040564 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0923 22:12:50.040684 (+ 120us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0923 22:12:50.040763 (+ 79us) server_negotiation.cc:1035] Waiting for connection context
0923 22:12:50.040926 (+ 163us) server_negotiation.cc:299] Negotiation successful
0923 22:12:50.041004 (+ 78us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":84}
I20260923 22:12:50.041491 8908 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 22:12:50.016627 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.135.252:42799 (local address 127.1.135.195:41895)
0923 22:12:50.016867 (+ 240us) negotiation.cc:107] Waiting for socket to connect
0923 22:12:50.016908 (+ 41us) client_negotiation.cc:174] Beginning negotiation
0923 22:12:50.024153 (+ 7245us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0923 22:12:50.024356 (+ 203us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0923 22:12:50.024357 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0923 22:12:50.025716 (+ 1359us) client_negotiation.cc:189] Negotiated authn=SASL
0923 22:12:50.025777 (+ 61us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 22:12:50.025779 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 22:12:50.027799 (+ 2020us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 22:12:50.027799 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 22:12:50.028612 (+ 813us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 22:12:50.028615 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 22:12:50.028985 (+ 370us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 22:12:50.028986 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 22:12:50.033667 (+ 4681us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 22:12:50.034749 (+ 1082us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0923 22:12:50.034837 (+ 88us) client_negotiation.cc:656] calling sasl_client_start()
0923 22:12:50.038223 (+ 3386us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0923 22:12:50.039974 (+ 1751us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0923 22:12:50.039975 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0923 22:12:50.039976 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0923 22:12:50.040221 (+ 245us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0923 22:12:50.040440 (+ 219us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0923 22:12:50.040441 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0923 22:12:50.040441 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0923 22:12:50.040494 (+ 53us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0923 22:12:50.040769 (+ 275us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0923 22:12:50.040770 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0923 22:12:50.040809 (+ 39us) client_negotiation.cc:769] Sending connection context
0923 22:12:50.041231 (+ 422us) client_negotiation.cc:240] Negotiation successful
0923 22:12:50.041337 (+ 106us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":203,"mutex_wait_us":316,"thread_start_us":78,"threads_started":1}
I20260923 22:12:50.042066 8904 heartbeater.cc:347] Connected to a master server at 127.1.135.252:42799
I20260923 22:12:50.042124 8904 heartbeater.cc:464] Registering TS with master...
I20260923 22:12:50.042196 8904 heartbeater.cc:511] Master 127.1.135.252:42799 requested a full tablet report, sending...
I20260923 22:12:50.042434 8451 ts_manager.cc:194] Registered new tserver with Master: 4d0b914e7ab746e7b17789e7b32aca0a (127.1.135.195:33563)
I20260923 22:12:50.043093 8929 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 22:12:50.037237 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.135.193:44597 (local address 127.0.0.1:45716)
0923 22:12:50.037573 (+ 336us) negotiation.cc:107] Waiting for socket to connect
0923 22:12:50.037581 (+ 8us) client_negotiation.cc:174] Beginning negotiation
0923 22:12:50.037622 (+ 41us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0923 22:12:50.038031 (+ 409us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0923 22:12:50.038032 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0923 22:12:50.038034 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0923 22:12:50.038104 (+ 70us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 22:12:50.038106 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 22:12:50.039696 (+ 1590us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 22:12:50.039697 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 22:12:50.041419 (+ 1722us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 22:12:50.041423 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 22:12:50.042839 (+ 1416us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 22:12:50.042840 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 22:12:50.042914 (+ 74us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 22:12:50.042919 (+ 5us) client_negotiation.cc:769] Sending connection context
0923 22:12:50.042976 (+ 57us) client_negotiation.cc:240] Negotiation successful
0923 22:12:50.043015 (+ 39us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":319,"thread_start_us":71,"threads_started":1}
I20260923 22:12:50.043107 8928 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 22:12:50.037329 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45716 (local address 127.1.135.193:44597)
0923 22:12:50.037854 (+ 525us) server_negotiation.cc:206] Beginning negotiation
0923 22:12:50.037859 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0923 22:12:50.037868 (+ 9us) server_negotiation.cc:407] Connection header received
0923 22:12:50.037908 (+ 40us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 22:12:50.037910 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 22:12:50.037925 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 22:12:50.038184 (+ 259us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0923 22:12:50.038246 (+ 62us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 22:12:50.039602 (+ 1356us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 22:12:50.041497 (+ 1895us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 22:12:50.042743 (+ 1246us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 22:12:50.042837 (+ 94us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 22:12:50.042906 (+ 69us) server_negotiation.cc:1035] Waiting for connection context
0923 22:12:50.042990 (+ 84us) server_negotiation.cc:299] Negotiation successful
0923 22:12:50.043023 (+ 33us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":485,"thread_start_us":79,"threads_started":1}
I20260923 22:12:50.043992 8930 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 22:12:50.037758 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34786 (local address 127.1.135.194:41925)
0923 22:12:50.039760 (+ 2002us) server_negotiation.cc:206] Beginning negotiation
0923 22:12:50.039765 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0923 22:12:50.039774 (+ 9us) server_negotiation.cc:407] Connection header received
0923 22:12:50.039821 (+ 47us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 22:12:50.039822 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 22:12:50.039837 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 22:12:50.040044 (+ 207us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0923 22:12:50.040129 (+ 85us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 22:12:50.041871 (+ 1742us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 22:12:50.043155 (+ 1284us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 22:12:50.043706 (+ 551us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 22:12:50.043757 (+ 51us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 22:12:50.043801 (+ 44us) server_negotiation.cc:1035] Waiting for connection context
0923 22:12:50.043914 (+ 113us) server_negotiation.cc:299] Negotiation successful
0923 22:12:50.043937 (+ 23us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":1968,"thread_start_us":85,"threads_started":1}
I20260923 22:12:50.044000 8225 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 22:12:50.037147 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.135.194:41925 (local address 127.0.0.1:34786)
0923 22:12:50.037534 (+ 387us) negotiation.cc:107] Waiting for socket to connect
0923 22:12:50.037546 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0923 22:12:50.037604 (+ 58us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0923 22:12:50.039931 (+ 2327us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0923 22:12:50.039932 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0923 22:12:50.039934 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0923 22:12:50.039981 (+ 47us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 22:12:50.039982 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 22:12:50.041960 (+ 1978us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 22:12:50.041961 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 22:12:50.043039 (+ 1078us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 22:12:50.043042 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 22:12:50.043777 (+ 735us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 22:12:50.043779 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 22:12:50.043838 (+ 59us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 22:12:50.043841 (+ 3us) client_negotiation.cc:769] Sending connection context
0923 22:12:50.043905 (+ 64us) client_negotiation.cc:240] Negotiation successful
0923 22:12:50.043938 (+ 33us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":372}
I20260923 22:12:50.044703 8701 tablet_service.cc:1467] Processing CreateTablet for tablet c9ec6940f6f240c692d10a75e3a23cf6 (DEFAULT_TABLE table=test-table [id=163c8be33d404351ba5ff430c735e181]), partition=RANGE (key) PARTITION UNBOUNDED
I20260923 22:12:50.044705 8564 tablet_service.cc:1467] Processing CreateTablet for tablet c9ec6940f6f240c692d10a75e3a23cf6 (DEFAULT_TABLE table=test-table [id=163c8be33d404351ba5ff430c735e181]), partition=RANGE (key) PARTITION UNBOUNDED
I20260923 22:12:50.045035 8564 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet c9ec6940f6f240c692d10a75e3a23cf6. 1 dirs total, 0 dirs full, 0 dirs failed
I20260923 22:12:50.045035 8701 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet c9ec6940f6f240c692d10a75e3a23cf6. 1 dirs total, 0 dirs full, 0 dirs failed
I20260923 22:12:50.048543 8933 tablet_bootstrap.cc:492] T c9ec6940f6f240c692d10a75e3a23cf6 P e01531ae61ec4025942988b1a6dd62f0: Bootstrap starting.
I20260923 22:12:50.048698 8932 tablet_bootstrap.cc:492] T c9ec6940f6f240c692d10a75e3a23cf6 P 8a9da5f92d7c4186877987e0ddd62caf: Bootstrap starting.
I20260923 22:12:50.049191 8931 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 22:12:50.037936 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57666 (local address 127.1.135.195:33563)
0923 22:12:50.041491 (+ 3555us) server_negotiation.cc:206] Beginning negotiation
0923 22:12:50.041496 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0923 22:12:50.041504 (+ 8us) server_negotiation.cc:407] Connection header received
0923 22:12:50.041533 (+ 29us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 22:12:50.041535 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 22:12:50.041548 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 22:12:50.041618 (+ 70us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0923 22:12:50.041891 (+ 273us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 22:12:50.043300 (+ 1409us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 22:12:50.048161 (+ 4861us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 22:12:50.048872 (+ 711us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 22:12:50.048950 (+ 78us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 22:12:50.049015 (+ 65us) server_negotiation.cc:1035] Waiting for connection context
0923 22:12:50.049086 (+ 71us) server_negotiation.cc:299] Negotiation successful
0923 22:12:50.049120 (+ 34us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":3525,"thread_start_us":68,"threads_started":1}
I20260923 22:12:50.049187 8927 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 22:12:50.037385 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.135.195:33563 (local address 127.0.0.1:57666)
0923 22:12:50.037696 (+ 311us) negotiation.cc:107] Waiting for socket to connect
0923 22:12:50.037704 (+ 8us) client_negotiation.cc:174] Beginning negotiation
0923 22:12:50.037738 (+ 34us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0923 22:12:50.041626 (+ 3888us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0923 22:12:50.041627 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0923 22:12:50.041630 (+ 3us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0923 22:12:50.041707 (+ 77us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 22:12:50.041709 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 22:12:50.043382 (+ 1673us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 22:12:50.043384 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 22:12:50.044684 (+ 1300us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 22:12:50.044686 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 22:12:50.048949 (+ 4263us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 22:12:50.048950 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 22:12:50.049011 (+ 61us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 22:12:50.049015 (+ 4us) client_negotiation.cc:769] Sending connection context
0923 22:12:50.049074 (+ 59us) client_negotiation.cc:240] Negotiation successful
0923 22:12:50.049120 (+ 46us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":299,"thread_start_us":65,"threads_started":1}
I20260923 22:12:50.049221 8932 tablet_bootstrap.cc:654] T c9ec6940f6f240c692d10a75e3a23cf6 P 8a9da5f92d7c4186877987e0ddd62caf: Neither blocks nor log segments found. Creating new log.
I20260923 22:12:50.049206 8933 tablet_bootstrap.cc:654] T c9ec6940f6f240c692d10a75e3a23cf6 P e01531ae61ec4025942988b1a6dd62f0: Neither blocks nor log segments found. Creating new log.
I20260923 22:12:50.049525 8932 log.cc:824] T c9ec6940f6f240c692d10a75e3a23cf6 P 8a9da5f92d7c4186877987e0ddd62caf: Log is configured to *not* fsync() on all Append() calls
I20260923 22:12:50.049829 8837 tablet_service.cc:1467] Processing CreateTablet for tablet c9ec6940f6f240c692d10a75e3a23cf6 (DEFAULT_TABLE table=test-table [id=163c8be33d404351ba5ff430c735e181]), partition=RANGE (key) PARTITION UNBOUNDED
I20260923 22:12:50.050127 8837 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet c9ec6940f6f240c692d10a75e3a23cf6. 1 dirs total, 0 dirs full, 0 dirs failed
I20260923 22:12:50.050284 8932 tablet_bootstrap.cc:492] T c9ec6940f6f240c692d10a75e3a23cf6 P 8a9da5f92d7c4186877987e0ddd62caf: No bootstrap required, opened a new log
I20260923 22:12:50.050359 8932 ts_tablet_manager.cc:1397] T c9ec6940f6f240c692d10a75e3a23cf6 P 8a9da5f92d7c4186877987e0ddd62caf: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20260923 22:12:50.052569 8933 log.cc:824] T c9ec6940f6f240c692d10a75e3a23cf6 P e01531ae61ec4025942988b1a6dd62f0: Log is configured to *not* fsync() on all Append() calls
I20260923 22:12:50.052800 8932 raft_consensus.cc:348] T c9ec6940f6f240c692d10a75e3a23cf6 P 8a9da5f92d7c4186877987e0ddd62caf [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8a9da5f92d7c4186877987e0ddd62caf" member_type: VOTER last_known_addr { host: "127.1.135.194" port: 41925 } } peers { permanent_uuid: "4d0b914e7ab746e7b17789e7b32aca0a" member_type: VOTER last_known_addr { host: "127.1.135.195" port: 33563 } } peers { permanent_uuid: "e01531ae61ec4025942988b1a6dd62f0" member_type: VOTER last_known_addr { host: "127.1.135.193" port: 44597 } }
I20260923 22:12:50.052959 8932 raft_consensus.cc:374] T c9ec6940f6f240c692d10a75e3a23cf6 P 8a9da5f92d7c4186877987e0ddd62caf [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260923 22:12:50.053006 8932 raft_consensus.cc:729] T c9ec6940f6f240c692d10a75e3a23cf6 P 8a9da5f92d7c4186877987e0ddd62caf [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 8a9da5f92d7c4186877987e0ddd62caf, State: Initialized, Role: FOLLOWER
I20260923 22:12:50.053138 8932 consensus_queue.cc:260] T c9ec6940f6f240c692d10a75e3a23cf6 P 8a9da5f92d7c4186877987e0ddd62caf [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8a9da5f92d7c4186877987e0ddd62caf" member_type: VOTER last_known_addr { host: "127.1.135.194" port: 41925 } } peers { permanent_uuid: "4d0b914e7ab746e7b17789e7b32aca0a" member_type: VOTER last_known_addr { host: "127.1.135.195" port: 33563 } } peers { permanent_uuid: "e01531ae61ec4025942988b1a6dd62f0" member_type: VOTER last_known_addr { host: "127.1.135.193" port: 44597 } }
I20260923 22:12:50.053411 8932 ts_tablet_manager.cc:1428] T c9ec6940f6f240c692d10a75e3a23cf6 P 8a9da5f92d7c4186877987e0ddd62caf: Time spent starting tablet: real 0.003s user 0.003s sys 0.000s
I20260923 22:12:50.053413 8933 tablet_bootstrap.cc:492] T c9ec6940f6f240c692d10a75e3a23cf6 P e01531ae61ec4025942988b1a6dd62f0: No bootstrap required, opened a new log
I20260923 22:12:50.053483 8933 ts_tablet_manager.cc:1397] T c9ec6940f6f240c692d10a75e3a23cf6 P e01531ae61ec4025942988b1a6dd62f0: Time spent bootstrapping tablet: real 0.005s user 0.001s sys 0.000s
I20260923 22:12:50.053532 8767 heartbeater.cc:503] Master 127.1.135.254:35949 was elected leader, sending a full tablet report...
I20260923 22:12:50.053511 8937 tablet_bootstrap.cc:492] T c9ec6940f6f240c692d10a75e3a23cf6 P 4d0b914e7ab746e7b17789e7b32aca0a: Bootstrap starting.
I20260923 22:12:50.054603 8937 tablet_bootstrap.cc:654] T c9ec6940f6f240c692d10a75e3a23cf6 P 4d0b914e7ab746e7b17789e7b32aca0a: Neither blocks nor log segments found. Creating new log.
I20260923 22:12:50.054868 8937 log.cc:824] T c9ec6940f6f240c692d10a75e3a23cf6 P 4d0b914e7ab746e7b17789e7b32aca0a: Log is configured to *not* fsync() on all Append() calls
I20260923 22:12:50.055502 8937 tablet_bootstrap.cc:492] T c9ec6940f6f240c692d10a75e3a23cf6 P 4d0b914e7ab746e7b17789e7b32aca0a: No bootstrap required, opened a new log
I20260923 22:12:50.055590 8937 ts_tablet_manager.cc:1397] T c9ec6940f6f240c692d10a75e3a23cf6 P 4d0b914e7ab746e7b17789e7b32aca0a: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20260923 22:12:50.055745 8933 raft_consensus.cc:348] T c9ec6940f6f240c692d10a75e3a23cf6 P e01531ae61ec4025942988b1a6dd62f0 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8a9da5f92d7c4186877987e0ddd62caf" member_type: VOTER last_known_addr { host: "127.1.135.194" port: 41925 } } peers { permanent_uuid: "4d0b914e7ab746e7b17789e7b32aca0a" member_type: VOTER last_known_addr { host: "127.1.135.195" port: 33563 } } peers { permanent_uuid: "e01531ae61ec4025942988b1a6dd62f0" member_type: VOTER last_known_addr { host: "127.1.135.193" port: 44597 } }
I20260923 22:12:50.055860 8933 raft_consensus.cc:374] T c9ec6940f6f240c692d10a75e3a23cf6 P e01531ae61ec4025942988b1a6dd62f0 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260923 22:12:50.055881 8933 raft_consensus.cc:729] T c9ec6940f6f240c692d10a75e3a23cf6 P e01531ae61ec4025942988b1a6dd62f0 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e01531ae61ec4025942988b1a6dd62f0, State: Initialized, Role: FOLLOWER
I20260923 22:12:50.055989 8933 consensus_queue.cc:260] T c9ec6940f6f240c692d10a75e3a23cf6 P e01531ae61ec4025942988b1a6dd62f0 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8a9da5f92d7c4186877987e0ddd62caf" member_type: VOTER last_known_addr { host: "127.1.135.194" port: 41925 } } peers { permanent_uuid: "4d0b914e7ab746e7b17789e7b32aca0a" member_type: VOTER last_known_addr { host: "127.1.135.195" port: 33563 } } peers { permanent_uuid: "e01531ae61ec4025942988b1a6dd62f0" member_type: VOTER last_known_addr { host: "127.1.135.193" port: 44597 } }
I20260923 22:12:50.056262 8933 ts_tablet_manager.cc:1428] T c9ec6940f6f240c692d10a75e3a23cf6 P e01531ae61ec4025942988b1a6dd62f0: Time spent starting tablet: real 0.003s user 0.003s sys 0.000s
I20260923 22:12:50.056275 8630 heartbeater.cc:503] Master 127.1.135.254:35949 was elected leader, sending a full tablet report...
I20260923 22:12:50.058094 8937 raft_consensus.cc:348] T c9ec6940f6f240c692d10a75e3a23cf6 P 4d0b914e7ab746e7b17789e7b32aca0a [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8a9da5f92d7c4186877987e0ddd62caf" member_type: VOTER last_known_addr { host: "127.1.135.194" port: 41925 } } peers { permanent_uuid: "4d0b914e7ab746e7b17789e7b32aca0a" member_type: VOTER last_known_addr { host: "127.1.135.195" port: 33563 } } peers { permanent_uuid: "e01531ae61ec4025942988b1a6dd62f0" member_type: VOTER last_known_addr { host: "127.1.135.193" port: 44597 } }
I20260923 22:12:50.058234 8937 raft_consensus.cc:374] T c9ec6940f6f240c692d10a75e3a23cf6 P 4d0b914e7ab746e7b17789e7b32aca0a [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260923 22:12:50.058277 8937 raft_consensus.cc:729] T c9ec6940f6f240c692d10a75e3a23cf6 P 4d0b914e7ab746e7b17789e7b32aca0a [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 4d0b914e7ab746e7b17789e7b32aca0a, State: Initialized, Role: FOLLOWER
I20260923 22:12:50.058383 8937 consensus_queue.cc:260] T c9ec6940f6f240c692d10a75e3a23cf6 P 4d0b914e7ab746e7b17789e7b32aca0a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8a9da5f92d7c4186877987e0ddd62caf" member_type: VOTER last_known_addr { host: "127.1.135.194" port: 41925 } } peers { permanent_uuid: "4d0b914e7ab746e7b17789e7b32aca0a" member_type: VOTER last_known_addr { host: "127.1.135.195" port: 33563 } } peers { permanent_uuid: "e01531ae61ec4025942988b1a6dd62f0" member_type: VOTER last_known_addr { host: "127.1.135.193" port: 44597 } }
I20260923 22:12:50.058625 8937 ts_tablet_manager.cc:1428] T c9ec6940f6f240c692d10a75e3a23cf6 P 4d0b914e7ab746e7b17789e7b32aca0a: Time spent starting tablet: real 0.003s user 0.003s sys 0.000s
I20260923 22:12:50.058676 8903 heartbeater.cc:503] Master 127.1.135.254:35949 was elected leader, sending a full tablet report...
W20260923 22:12:50.127892 8770 tablet.cc:2369] T c9ec6940f6f240c692d10a75e3a23cf6 P 8a9da5f92d7c4186877987e0ddd62caf: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260923 22:12:50.167716 8939 raft_consensus.cc:482] T c9ec6940f6f240c692d10a75e3a23cf6 P e01531ae61ec4025942988b1a6dd62f0 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260923 22:12:50.167824 8939 raft_consensus.cc:504] T c9ec6940f6f240c692d10a75e3a23cf6 P e01531ae61ec4025942988b1a6dd62f0 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8a9da5f92d7c4186877987e0ddd62caf" member_type: VOTER last_known_addr { host: "127.1.135.194" port: 41925 } } peers { permanent_uuid: "4d0b914e7ab746e7b17789e7b32aca0a" member_type: VOTER last_known_addr { host: "127.1.135.195" port: 33563 } } peers { permanent_uuid: "e01531ae61ec4025942988b1a6dd62f0" member_type: VOTER last_known_addr { host: "127.1.135.193" port: 44597 } }
I20260923 22:12:50.168231 8939 leader_election.cc:288] T c9ec6940f6f240c692d10a75e3a23cf6 P e01531ae61ec4025942988b1a6dd62f0 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 8a9da5f92d7c4186877987e0ddd62caf (127.1.135.194:41925), 4d0b914e7ab746e7b17789e7b32aca0a (127.1.135.195:33563)
I20260923 22:12:50.171028 8637 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 22:12:50.168410 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.135.194:41925 (local address 127.1.135.193:53583)
0923 22:12:50.168519 (+ 109us) negotiation.cc:107] Waiting for socket to connect
0923 22:12:50.168531 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0923 22:12:50.168586 (+ 55us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0923 22:12:50.168749 (+ 163us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0923 22:12:50.168750 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0923 22:12:50.168751 (+ 1us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0923 22:12:50.168831 (+ 80us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 22:12:50.168832 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 22:12:50.169465 (+ 633us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 22:12:50.169466 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 22:12:50.170239 (+ 773us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 22:12:50.170241 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 22:12:50.170803 (+ 562us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 22:12:50.170804 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 22:12:50.170872 (+ 68us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 22:12:50.170877 (+ 5us) client_negotiation.cc:769] Sending connection context
0923 22:12:50.170928 (+ 51us) client_negotiation.cc:240] Negotiation successful
0923 22:12:50.170977 (+ 49us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":91,"mutex_wait_us":98}
I20260923 22:12:50.171060 8930 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 22:12:50.168468 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.135.193:53583 (local address 127.1.135.194:41925)
0923 22:12:50.168604 (+ 136us) server_negotiation.cc:206] Beginning negotiation
0923 22:12:50.168606 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0923 22:12:50.168612 (+ 6us) server_negotiation.cc:407] Connection header received
0923 22:12:50.168637 (+ 25us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 22:12:50.168637 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 22:12:50.168672 (+ 35us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 22:12:50.168724 (+ 52us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0923 22:12:50.168913 (+ 189us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 22:12:50.169362 (+ 449us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 22:12:50.170316 (+ 954us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 22:12:50.170726 (+ 410us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 22:12:50.170781 (+ 55us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 22:12:50.170787 (+ 6us) server_negotiation.cc:1035] Waiting for connection context
0923 22:12:50.170945 (+ 158us) server_negotiation.cc:299] Negotiation successful
0923 22:12:50.170979 (+ 34us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":117}
I20260923 22:12:50.171231 8636 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 22:12:50.168410 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.135.195:33563 (local address 127.1.135.193:58497)
0923 22:12:50.168471 (+ 61us) negotiation.cc:107] Waiting for socket to connect
0923 22:12:50.168481 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0923 22:12:50.168525 (+ 44us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0923 22:12:50.168692 (+ 167us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0923 22:12:50.168693 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0923 22:12:50.168695 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0923 22:12:50.168774 (+ 79us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 22:12:50.168781 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 22:12:50.169439 (+ 658us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 22:12:50.169441 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 22:12:50.170347 (+ 906us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 22:12:50.170349 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 22:12:50.170950 (+ 601us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 22:12:50.170951 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 22:12:50.171026 (+ 75us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 22:12:50.171030 (+ 4us) client_negotiation.cc:769] Sending connection context
0923 22:12:50.171096 (+ 66us) client_negotiation.cc:240] Negotiation successful
0923 22:12:50.171136 (+ 40us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":42,"mutex_wait_us":28}
I20260923 22:12:50.171370 8931 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 22:12:50.168485 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.135.193:58497 (local address 127.1.135.195:33563)
0923 22:12:50.168524 (+ 39us) server_negotiation.cc:206] Beginning negotiation
0923 22:12:50.168526 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0923 22:12:50.168538 (+ 12us) server_negotiation.cc:407] Connection header received
0923 22:12:50.168598 (+ 60us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 22:12:50.168599 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 22:12:50.168609 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 22:12:50.168664 (+ 55us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0923 22:12:50.168848 (+ 184us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 22:12:50.169359 (+ 511us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 22:12:50.170431 (+ 1072us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 22:12:50.170848 (+ 417us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 22:12:50.170921 (+ 73us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 22:12:50.170930 (+ 9us) server_negotiation.cc:1035] Waiting for connection context
0923 22:12:50.171258 (+ 328us) server_negotiation.cc:299] Negotiation successful
0923 22:12:50.171303 (+ 45us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":21}
I20260923 22:12:50.171427 8721 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "c9ec6940f6f240c692d10a75e3a23cf6" candidate_uuid: "e01531ae61ec4025942988b1a6dd62f0" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8a9da5f92d7c4186877987e0ddd62caf" is_pre_election: true
I20260923 22:12:50.171581 8721 raft_consensus.cc:2454] T c9ec6940f6f240c692d10a75e3a23cf6 P 8a9da5f92d7c4186877987e0ddd62caf [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e01531ae61ec4025942988b1a6dd62f0 in term 0.
I20260923 22:12:50.171693 8857 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "c9ec6940f6f240c692d10a75e3a23cf6" candidate_uuid: "e01531ae61ec4025942988b1a6dd62f0" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4d0b914e7ab746e7b17789e7b32aca0a" is_pre_election: true
I20260923 22:12:50.171835 8857 raft_consensus.cc:2454] T c9ec6940f6f240c692d10a75e3a23cf6 P 4d0b914e7ab746e7b17789e7b32aca0a [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e01531ae61ec4025942988b1a6dd62f0 in term 0.
I20260923 22:12:50.171837 8518 leader_election.cc:302] T c9ec6940f6f240c692d10a75e3a23cf6 P e01531ae61ec4025942988b1a6dd62f0 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8a9da5f92d7c4186877987e0ddd62caf, e01531ae61ec4025942988b1a6dd62f0; no voters:
I20260923 22:12:50.171968 8939 raft_consensus.cc:2781] T c9ec6940f6f240c692d10a75e3a23cf6 P e01531ae61ec4025942988b1a6dd62f0 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260923 22:12:50.172039 8939 raft_consensus.cc:482] T c9ec6940f6f240c692d10a75e3a23cf6 P e01531ae61ec4025942988b1a6dd62f0 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260923 22:12:50.172087 8939 raft_consensus.cc:3037] T c9ec6940f6f240c692d10a75e3a23cf6 P e01531ae61ec4025942988b1a6dd62f0 [term 0 FOLLOWER]: Advancing to term 1
I20260923 22:12:50.172766 8939 raft_consensus.cc:504] T c9ec6940f6f240c692d10a75e3a23cf6 P e01531ae61ec4025942988b1a6dd62f0 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8a9da5f92d7c4186877987e0ddd62caf" member_type: VOTER last_known_addr { host: "127.1.135.194" port: 41925 } } peers { permanent_uuid: "4d0b914e7ab746e7b17789e7b32aca0a" member_type: VOTER last_known_addr { host: "127.1.135.195" port: 33563 } } peers { permanent_uuid: "e01531ae61ec4025942988b1a6dd62f0" member_type: VOTER last_known_addr { host: "127.1.135.193" port: 44597 } }
I20260923 22:12:50.172923 8939 leader_election.cc:288] T c9ec6940f6f240c692d10a75e3a23cf6 P e01531ae61ec4025942988b1a6dd62f0 [CANDIDATE]: Term 1 election: Requested vote from peers 8a9da5f92d7c4186877987e0ddd62caf (127.1.135.194:41925), 4d0b914e7ab746e7b17789e7b32aca0a (127.1.135.195:33563)
I20260923 22:12:50.173077 8721 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "c9ec6940f6f240c692d10a75e3a23cf6" candidate_uuid: "e01531ae61ec4025942988b1a6dd62f0" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8a9da5f92d7c4186877987e0ddd62caf"
I20260923 22:12:50.173134 8857 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "c9ec6940f6f240c692d10a75e3a23cf6" candidate_uuid: "e01531ae61ec4025942988b1a6dd62f0" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4d0b914e7ab746e7b17789e7b32aca0a"
I20260923 22:12:50.173185 8721 raft_consensus.cc:3037] T c9ec6940f6f240c692d10a75e3a23cf6 P 8a9da5f92d7c4186877987e0ddd62caf [term 0 FOLLOWER]: Advancing to term 1
I20260923 22:12:50.173219 8857 raft_consensus.cc:3037] T c9ec6940f6f240c692d10a75e3a23cf6 P 4d0b914e7ab746e7b17789e7b32aca0a [term 0 FOLLOWER]: Advancing to term 1
I20260923 22:12:50.173851 8721 raft_consensus.cc:2454] T c9ec6940f6f240c692d10a75e3a23cf6 P 8a9da5f92d7c4186877987e0ddd62caf [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e01531ae61ec4025942988b1a6dd62f0 in term 1.
I20260923 22:12:50.173911 8857 raft_consensus.cc:2454] T c9ec6940f6f240c692d10a75e3a23cf6 P 4d0b914e7ab746e7b17789e7b32aca0a [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e01531ae61ec4025942988b1a6dd62f0 in term 1.
I20260923 22:12:50.174027 8518 leader_election.cc:302] T c9ec6940f6f240c692d10a75e3a23cf6 P e01531ae61ec4025942988b1a6dd62f0 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8a9da5f92d7c4186877987e0ddd62caf, e01531ae61ec4025942988b1a6dd62f0; no voters:
I20260923 22:12:50.174142 8939 raft_consensus.cc:2781] T c9ec6940f6f240c692d10a75e3a23cf6 P e01531ae61ec4025942988b1a6dd62f0 [term 1 FOLLOWER]: Leader election won for term 1
I20260923 22:12:50.174306 8939 raft_consensus.cc:686] T c9ec6940f6f240c692d10a75e3a23cf6 P e01531ae61ec4025942988b1a6dd62f0 [term 1 LEADER]: Becoming Leader. State: Replica: e01531ae61ec4025942988b1a6dd62f0, State: Running, Role: LEADER
I20260923 22:12:50.174415 8939 consensus_queue.cc:237] T c9ec6940f6f240c692d10a75e3a23cf6 P e01531ae61ec4025942988b1a6dd62f0 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8a9da5f92d7c4186877987e0ddd62caf" member_type: VOTER last_known_addr { host: "127.1.135.194" port: 41925 } } peers { permanent_uuid: "4d0b914e7ab746e7b17789e7b32aca0a" member_type: VOTER last_known_addr { host: "127.1.135.195" port: 33563 } } peers { permanent_uuid: "e01531ae61ec4025942988b1a6dd62f0" member_type: VOTER last_known_addr { host: "127.1.135.193" port: 44597 } }
I20260923 22:12:50.175261 7667 catalog_manager.cc:5495] T c9ec6940f6f240c692d10a75e3a23cf6 P e01531ae61ec4025942988b1a6dd62f0 reported cstate change: term changed from 0 to 1, leader changed from <none> to e01531ae61ec4025942988b1a6dd62f0 (127.1.135.193). New cstate: current_term: 1 leader_uuid: "e01531ae61ec4025942988b1a6dd62f0" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8a9da5f92d7c4186877987e0ddd62caf" member_type: VOTER last_known_addr { host: "127.1.135.194" port: 41925 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "4d0b914e7ab746e7b17789e7b32aca0a" member_type: VOTER last_known_addr { host: "127.1.135.195" port: 33563 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "e01531ae61ec4025942988b1a6dd62f0" member_type: VOTER last_known_addr { host: "127.1.135.193" port: 44597 } health_report { overall_health: HEALTHY } } }
I20260923 22:12:50.182934 8928 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 22:12:50.180982 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45724 (local address 127.1.135.193:44597)
0923 22:12:50.181022 (+ 40us) server_negotiation.cc:206] Beginning negotiation
0923 22:12:50.181023 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0923 22:12:50.181030 (+ 7us) server_negotiation.cc:407] Connection header received
0923 22:12:50.181100 (+ 70us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 22:12:50.181101 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 22:12:50.181115 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 22:12:50.181169 (+ 54us) server_negotiation.cc:226] Negotiated authn=TOKEN
0923 22:12:50.181268 (+ 99us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 22:12:50.181686 (+ 418us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 22:12:50.182343 (+ 657us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 22:12:50.182523 (+ 180us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 22:12:50.182568 (+ 45us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 22:12:50.182678 (+ 110us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0923 22:12:50.182760 (+ 82us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0923 22:12:50.182815 (+ 55us) server_negotiation.cc:1035] Waiting for connection context
0923 22:12:50.182844 (+ 29us) server_negotiation.cc:299] Negotiation successful
0923 22:12:50.182856 (+ 12us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":21}
I20260923 22:12:50.184612 8721 raft_consensus.cc:1261] T c9ec6940f6f240c692d10a75e3a23cf6 P 8a9da5f92d7c4186877987e0ddd62caf [term 1 FOLLOWER]: Refusing update from remote peer e01531ae61ec4025942988b1a6dd62f0: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260923 22:12:50.184619 8857 raft_consensus.cc:1261] T c9ec6940f6f240c692d10a75e3a23cf6 P 4d0b914e7ab746e7b17789e7b32aca0a [term 1 FOLLOWER]: Refusing update from remote peer e01531ae61ec4025942988b1a6dd62f0: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260923 22:12:50.184933 8939 consensus_queue.cc:1035] T c9ec6940f6f240c692d10a75e3a23cf6 P e01531ae61ec4025942988b1a6dd62f0 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4d0b914e7ab746e7b17789e7b32aca0a" member_type: VOTER last_known_addr { host: "127.1.135.195" port: 33563 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260923 22:12:50.185073 8941 consensus_queue.cc:1035] T c9ec6940f6f240c692d10a75e3a23cf6 P e01531ae61ec4025942988b1a6dd62f0 [LEADER]: Connected to new peer: Peer: permanent_uuid: "8a9da5f92d7c4186877987e0ddd62caf" member_type: VOTER last_known_addr { host: "127.1.135.194" port: 41925 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260923 22:12:50.189970 7667 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:52072:
table { table_name: "test-table" } modify_external_catalogs: true
I20260923 22:12:50.190060 7667 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:52072:
table { table_name: "test-table" } modify_external_catalogs: true
I20260923 22:12:50.191490 7667 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 6378d422392b4a74831ae1c0f0692998: Sending DeleteTablet for 3 replicas of tablet c9ec6940f6f240c692d10a75e3a23cf6
I20260923 22:12:50.191898 8837 tablet_service.cc:1514] Processing DeleteTablet for tablet c9ec6940f6f240c692d10a75e3a23cf6 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-23 22:12:50 UTC) from {username='oryx', principal='oryx/127.1.135.254@KRBTEST.COM'} at 127.0.0.1:57666
I20260923 22:12:50.192180 8954 tablet_replica.cc:330] T c9ec6940f6f240c692d10a75e3a23cf6 P 4d0b914e7ab746e7b17789e7b32aca0a: stopping tablet replica
I20260923 22:12:50.192281 8954 raft_consensus.cc:2229] T c9ec6940f6f240c692d10a75e3a23cf6 P 4d0b914e7ab746e7b17789e7b32aca0a [term 1 FOLLOWER]: Raft consensus shutting down.
I20260923 22:12:50.192374 8954 raft_consensus.cc:2258] T c9ec6940f6f240c692d10a75e3a23cf6 P 4d0b914e7ab746e7b17789e7b32aca0a [term 1 FOLLOWER]: Raft consensus is shut down!
I20260923 22:12:50.192360 8701 tablet_service.cc:1514] Processing DeleteTablet for tablet c9ec6940f6f240c692d10a75e3a23cf6 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-23 22:12:50 UTC) from {username='oryx', principal='oryx/127.1.135.254@KRBTEST.COM'} at 127.0.0.1:34786
I20260923 22:12:50.192548 8956 tablet_replica.cc:330] T c9ec6940f6f240c692d10a75e3a23cf6 P 8a9da5f92d7c4186877987e0ddd62caf: stopping tablet replica
I20260923 22:12:50.192565 8564 tablet_service.cc:1514] Processing DeleteTablet for tablet c9ec6940f6f240c692d10a75e3a23cf6 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-23 22:12:50 UTC) from {username='oryx', principal='oryx/127.1.135.254@KRBTEST.COM'} at 127.0.0.1:45716
I20260923 22:12:50.192653 8956 raft_consensus.cc:2229] T c9ec6940f6f240c692d10a75e3a23cf6 P 8a9da5f92d7c4186877987e0ddd62caf [term 1 FOLLOWER]: Raft consensus shutting down.
I20260923 22:12:50.192749 8956 raft_consensus.cc:2258] T c9ec6940f6f240c692d10a75e3a23cf6 P 8a9da5f92d7c4186877987e0ddd62caf [term 1 FOLLOWER]: Raft consensus is shut down!
I20260923 22:12:50.192768 8957 tablet_replica.cc:330] T c9ec6940f6f240c692d10a75e3a23cf6 P e01531ae61ec4025942988b1a6dd62f0: stopping tablet replica
I20260923 22:12:50.192957 8957 raft_consensus.cc:2229] T c9ec6940f6f240c692d10a75e3a23cf6 P e01531ae61ec4025942988b1a6dd62f0 [term 1 LEADER]: Raft consensus shutting down.
I20260923 22:12:50.193096 8957 raft_consensus.cc:2258] T c9ec6940f6f240c692d10a75e3a23cf6 P e01531ae61ec4025942988b1a6dd62f0 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260923 22:12:50.193177 8956 ts_tablet_manager.cc:1905] T c9ec6940f6f240c692d10a75e3a23cf6 P 8a9da5f92d7c4186877987e0ddd62caf: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260923 22:12:50.193437 1567 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskRiQe_a/build/release/bin/kudu with pid 7635
I20260923 22:12:50.195220 8957 ts_tablet_manager.cc:1905] T c9ec6940f6f240c692d10a75e3a23cf6 P e01531ae61ec4025942988b1a6dd62f0: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260923 22:12:50.196398 8956 ts_tablet_manager.cc:1918] T c9ec6940f6f240c692d10a75e3a23cf6 P 8a9da5f92d7c4186877987e0ddd62caf: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260923 22:12:50.196498 8956 log.cc:1196] T c9ec6940f6f240c692d10a75e3a23cf6 P 8a9da5f92d7c4186877987e0ddd62caf: Deleting WAL directory at /tmp/dist-test-taskRiQe_a/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790201539513534-1567-0/minicluster-data/ts-1/wal/wals/c9ec6940f6f240c692d10a75e3a23cf6
I20260923 22:12:50.196746 8956 ts_tablet_manager.cc:1939] T c9ec6940f6f240c692d10a75e3a23cf6 P 8a9da5f92d7c4186877987e0ddd62caf: Deleting consensus metadata
I20260923 22:12:50.197111 8957 ts_tablet_manager.cc:1918] T c9ec6940f6f240c692d10a75e3a23cf6 P e01531ae61ec4025942988b1a6dd62f0: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260923 22:12:50.197184 8957 log.cc:1196] T c9ec6940f6f240c692d10a75e3a23cf6 P e01531ae61ec4025942988b1a6dd62f0: Deleting WAL directory at /tmp/dist-test-taskRiQe_a/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790201539513534-1567-0/minicluster-data/ts-0/wal/wals/c9ec6940f6f240c692d10a75e3a23cf6
I20260923 22:12:50.197415 8957 ts_tablet_manager.cc:1939] T c9ec6940f6f240c692d10a75e3a23cf6 P e01531ae61ec4025942988b1a6dd62f0: Deleting consensus metadata
W20260923 22:12:50.199494 8518 connection.cc:535] server connection from 127.0.0.1:45716 recv error: Network error: recv error from unknown peer: Transport endpoint is not connected (error 107)
W20260923 22:12:50.199643 8654 connection.cc:535] server connection from 127.0.0.1:34786 recv error: Network error: recv error from unknown peer: Transport endpoint is not connected (error 107)
I20260923 22:12:50.200227 8954 ts_tablet_manager.cc:1905] T c9ec6940f6f240c692d10a75e3a23cf6 P 4d0b914e7ab746e7b17789e7b32aca0a: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260923 22:12:50.201848 8954 ts_tablet_manager.cc:1918] T c9ec6940f6f240c692d10a75e3a23cf6 P 4d0b914e7ab746e7b17789e7b32aca0a: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260923 22:12:50.201921 8954 log.cc:1196] T c9ec6940f6f240c692d10a75e3a23cf6 P 4d0b914e7ab746e7b17789e7b32aca0a: Deleting WAL directory at /tmp/dist-test-taskRiQe_a/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790201539513534-1567-0/minicluster-data/ts-2/wal/wals/c9ec6940f6f240c692d10a75e3a23cf6
I20260923 22:12:50.202195 8954 ts_tablet_manager.cc:1939] T c9ec6940f6f240c692d10a75e3a23cf6 P 4d0b914e7ab746e7b17789e7b32aca0a: Deleting consensus metadata
W20260923 22:12:50.202519 8793 connection.cc:441] server connection from 127.0.0.1:57666 torn down before Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:57666 (request call id 1) could send its response
I20260923 22:12:50.209014 8919 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 22:12:50.201219 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53534 (local address 127.1.135.253:39445)
0923 22:12:50.201259 (+ 40us) server_negotiation.cc:206] Beginning negotiation
0923 22:12:50.201261 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0923 22:12:50.201329 (+ 68us) server_negotiation.cc:407] Connection header received
0923 22:12:50.201438 (+ 109us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 22:12:50.201439 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 22:12:50.201449 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 22:12:50.201518 (+ 69us) server_negotiation.cc:226] Negotiated authn=SASL
0923 22:12:50.202325 (+ 807us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 22:12:50.203100 (+ 775us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 22:12:50.203901 (+ 801us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 22:12:50.204195 (+ 294us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 22:12:50.204279 (+ 84us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 22:12:50.205645 (+ 1366us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0923 22:12:50.205646 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0923 22:12:50.205647 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0923 22:12:50.205654 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0923 22:12:50.207540 (+ 1886us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0923 22:12:50.207918 (+ 378us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0923 22:12:50.207920 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0923 22:12:50.207920 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0923 22:12:50.207982 (+ 62us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0923 22:12:50.208362 (+ 380us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0923 22:12:50.208364 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0923 22:12:50.208364 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0923 22:12:50.208514 (+ 150us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0923 22:12:50.208597 (+ 83us) server_negotiation.cc:1035] Waiting for connection context
0923 22:12:50.208814 (+ 217us) server_negotiation.cc:299] Negotiation successful
0923 22:12:50.208908 (+ 94us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":21}
I20260923 22:12:50.210080 8634 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 22:12:50.192720 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43386 (local address 127.1.135.252:42799)
0923 22:12:50.192820 (+ 100us) server_negotiation.cc:206] Beginning negotiation
0923 22:12:50.192822 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0923 22:12:50.202606 (+ 9784us) server_negotiation.cc:407] Connection header received
0923 22:12:50.202722 (+ 116us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 22:12:50.202723 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 22:12:50.202738 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 22:12:50.202794 (+ 56us) server_negotiation.cc:226] Negotiated authn=SASL
0923 22:12:50.203335 (+ 541us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 22:12:50.204116 (+ 781us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 22:12:50.205556 (+ 1440us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 22:12:50.205806 (+ 250us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 22:12:50.205883 (+ 77us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 22:12:50.207179 (+ 1296us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0923 22:12:50.207180 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0923 22:12:50.207180 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0923 22:12:50.207187 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0923 22:12:50.208725 (+ 1538us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0923 22:12:50.209243 (+ 518us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0923 22:12:50.209244 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0923 22:12:50.209245 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0923 22:12:50.209330 (+ 85us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0923 22:12:50.209559 (+ 229us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0923 22:12:50.209560 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0923 22:12:50.209561 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0923 22:12:50.209691 (+ 130us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0923 22:12:50.209788 (+ 97us) server_negotiation.cc:1035] Waiting for connection context
0923 22:12:50.209915 (+ 127us) server_negotiation.cc:299] Negotiation successful
0923 22:12:50.209988 (+ 73us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":79}
I20260923 22:12:50.215015 8915 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 22:12:50.208256 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43402 (local address 127.1.135.252:42799)
0923 22:12:50.208828 (+ 572us) server_negotiation.cc:206] Beginning negotiation
0923 22:12:50.208829 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0923 22:12:50.208835 (+ 6us) server_negotiation.cc:407] Connection header received
0923 22:12:50.208851 (+ 16us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 22:12:50.208851 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 22:12:50.208860 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 22:12:50.208905 (+ 45us) server_negotiation.cc:226] Negotiated authn=SASL
0923 22:12:50.209913 (+ 1008us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 22:12:50.210454 (+ 541us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 22:12:50.211339 (+ 885us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 22:12:50.211538 (+ 199us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 22:12:50.211620 (+ 82us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 22:12:50.212847 (+ 1227us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0923 22:12:50.212848 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0923 22:12:50.212848 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0923 22:12:50.212854 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0923 22:12:50.214017 (+ 1163us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0923 22:12:50.214324 (+ 307us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0923 22:12:50.214325 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0923 22:12:50.214325 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0923 22:12:50.214376 (+ 51us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0923 22:12:50.214560 (+ 184us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0923 22:12:50.214561 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0923 22:12:50.214561 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0923 22:12:50.214664 (+ 103us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0923 22:12:50.214755 (+ 91us) server_negotiation.cc:1035] Waiting for connection context
0923 22:12:50.214855 (+ 100us) server_negotiation.cc:299] Negotiation successful
0923 22:12:50.214925 (+ 70us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":552}
I20260923 22:12:51.190232 8964 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 22:12:51.189981 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.135.254:35949 (local address 127.1.135.195:55465)
0923 22:12:51.190094 (+ 113us) negotiation.cc:107] Waiting for socket to connect
0923 22:12:51.190172 (+ 78us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.135.254:35949: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":93,"thread_start_us":66,"threads_started":1}
W20260923 22:12:51.190641 8903 heartbeater.cc:644] Failed to heartbeat to 127.1.135.254:35949 (0 consecutive failures): Network error: Failed to send heartbeat to master: Client connection negotiation failed: client connection to 127.1.135.254:35949: connect: Connection refused (error 111)
I20260923 22:12:51.192637 8966 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 22:12:51.192414 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.135.254:35949 (local address 127.1.135.194:44725)
0923 22:12:51.192512 (+ 98us) negotiation.cc:107] Waiting for socket to connect
0923 22:12:51.192584 (+ 72us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.135.254:35949: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":81,"thread_start_us":53,"threads_started":1}
I20260923 22:12:51.192615 8965 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 22:12:51.190169 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.135.254:35949 (local address 127.1.135.193:35421)
0923 22:12:51.192493 (+ 2324us) negotiation.cc:107] Waiting for socket to connect
0923 22:12:51.192569 (+ 76us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.135.254:35949: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":2306,"thread_start_us":45,"threads_started":1}
W20260923 22:12:51.193061 8630 heartbeater.cc:644] Failed to heartbeat to 127.1.135.254:35949 (0 consecutive failures): Network error: Failed to send heartbeat to master: Client connection negotiation failed: client connection to 127.1.135.254:35949: connect: Connection refused (error 111)
W20260923 22:12:51.195201 8767 heartbeater.cc:644] Failed to heartbeat to 127.1.135.254:35949 (0 consecutive failures): Network error: Failed to send heartbeat to master: Client connection negotiation failed: client connection to 127.1.135.254:35949: connect: Connection refused (error 111)
I20260923 22:12:51.692006 8967 raft_consensus.cc:482] T 00000000000000000000000000000000 P b32070aba06045cabad174bee345066a [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 6378d422392b4a74831ae1c0f0692998)
I20260923 22:12:51.692207 8967 raft_consensus.cc:504] T 00000000000000000000000000000000 P b32070aba06045cabad174bee345066a [term 1 FOLLOWER]: Starting pre-election with config: opid_index: 9 OBSOLETE_local: true peers { permanent_uuid: "6378d422392b4a74831ae1c0f0692998" member_type: VOTER last_known_addr { host: "127.1.135.254" port: 35949 } } peers { permanent_uuid: "b32070aba06045cabad174bee345066a" member_type: VOTER last_known_addr { host: "127.1.135.253" port: 39445 } attrs { promote: false } } peers { permanent_uuid: "d2a5b389b5d04491a85f778e638e056d" member_type: VOTER last_known_addr { host: "127.1.135.252" port: 42799 } attrs { promote: false } }
I20260923 22:12:51.692411 8968 raft_consensus.cc:482] T 00000000000000000000000000000000 P d2a5b389b5d04491a85f778e638e056d [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 6378d422392b4a74831ae1c0f0692998)
I20260923 22:12:51.692528 8967 leader_election.cc:288] T 00000000000000000000000000000000 P b32070aba06045cabad174bee345066a [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 6378d422392b4a74831ae1c0f0692998 (127.1.135.254:35949), d2a5b389b5d04491a85f778e638e056d (127.1.135.252:42799)
I20260923 22:12:51.692783 8968 raft_consensus.cc:504] T 00000000000000000000000000000000 P d2a5b389b5d04491a85f778e638e056d [term 1 FOLLOWER]: Starting pre-election with config: opid_index: 9 OBSOLETE_local: true peers { permanent_uuid: "6378d422392b4a74831ae1c0f0692998" member_type: VOTER last_known_addr { host: "127.1.135.254" port: 35949 } } peers { permanent_uuid: "b32070aba06045cabad174bee345066a" member_type: VOTER last_known_addr { host: "127.1.135.253" port: 39445 } attrs { promote: false } } peers { permanent_uuid: "d2a5b389b5d04491a85f778e638e056d" member_type: VOTER last_known_addr { host: "127.1.135.252" port: 42799 } attrs { promote: false } }
I20260923 22:12:51.692951 8969 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 22:12:51.692644 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.135.254:35949 (local address 127.0.0.1:52514)
0923 22:12:51.692755 (+ 111us) negotiation.cc:107] Waiting for socket to connect
0923 22:12:51.692913 (+ 158us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.135.254:35949: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":96,"thread_start_us":69,"threads_started":1}
W20260923 22:12:51.693053 8250 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.1.135.254:35949: connect: Connection refused (error 111)
I20260923 22:12:51.693463 8973 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 22:12:51.693222 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.135.254:35949 (local address 127.0.0.1:52516)
0923 22:12:51.693382 (+ 160us) negotiation.cc:107] Waiting for socket to connect
0923 22:12:51.693429 (+ 47us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.135.254:35949: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":148,"thread_start_us":68,"threads_started":1}
I20260923 22:12:51.693492 8969 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 22:12:51.693399 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.135.254:35949 (local address 127.0.0.1:52518)
0923 22:12:51.693438 (+ 39us) negotiation.cc:107] Waiting for socket to connect
0923 22:12:51.693464 (+ 26us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.135.254:35949: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":24}
W20260923 22:12:51.693583 8250 leader_election.cc:334] T 00000000000000000000000000000000 P b32070aba06045cabad174bee345066a [CANDIDATE]: Term 2 pre-election: RPC error from VoteRequest() call to peer 6378d422392b4a74831ae1c0f0692998 (127.1.135.254:35949): Network error: Client connection negotiation failed: client connection to 127.1.135.254:35949: connect: Connection refused (error 111)
W20260923 22:12:51.693681 8436 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.1.135.254:35949: connect: Connection refused (error 111)
I20260923 22:12:51.693845 8968 leader_election.cc:288] T 00000000000000000000000000000000 P d2a5b389b5d04491a85f778e638e056d [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 6378d422392b4a74831ae1c0f0692998 (127.1.135.254:35949), b32070aba06045cabad174bee345066a (127.1.135.253:39445)
I20260923 22:12:51.694171 8974 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 22:12:51.694050 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.135.254:35949 (local address 127.0.0.1:52540)
0923 22:12:51.694104 (+ 54us) negotiation.cc:107] Waiting for socket to connect
0923 22:12:51.694143 (+ 39us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.135.254:35949: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":39}
W20260923 22:12:51.694244 8436 leader_election.cc:334] T 00000000000000000000000000000000 P d2a5b389b5d04491a85f778e638e056d [CANDIDATE]: Term 2 pre-election: RPC error from VoteRequest() call to peer 6378d422392b4a74831ae1c0f0692998 (127.1.135.254:35949): Network error: Client connection negotiation failed: client connection to 127.1.135.254:35949: connect: Connection refused (error 111)
I20260923 22:12:51.696528 8971 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 22:12:51.692823 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43404 (local address 127.1.135.252:42799)
0923 22:12:51.692925 (+ 102us) server_negotiation.cc:206] Beginning negotiation
0923 22:12:51.692927 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0923 22:12:51.692934 (+ 7us) server_negotiation.cc:407] Connection header received
0923 22:12:51.692959 (+ 25us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 22:12:51.692959 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 22:12:51.692968 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 22:12:51.693051 (+ 83us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0923 22:12:51.693213 (+ 162us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 22:12:51.694515 (+ 1302us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 22:12:51.695690 (+ 1175us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 22:12:51.696211 (+ 521us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 22:12:51.696284 (+ 73us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 22:12:51.696297 (+ 13us) server_negotiation.cc:1035] Waiting for connection context
0923 22:12:51.696453 (+ 156us) server_negotiation.cc:299] Negotiation successful
0923 22:12:51.696486 (+ 33us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":84,"thread_start_us":49,"threads_started":1}
I20260923 22:12:51.696573 8970 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 22:12:51.692653 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.135.252:42799 (local address 127.0.0.1:43404)
0923 22:12:51.692763 (+ 110us) negotiation.cc:107] Waiting for socket to connect
0923 22:12:51.692775 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0923 22:12:51.692849 (+ 74us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0923 22:12:51.693099 (+ 250us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0923 22:12:51.693100 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0923 22:12:51.693102 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0923 22:12:51.693159 (+ 57us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 22:12:51.693161 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 22:12:51.694592 (+ 1431us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 22:12:51.694593 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 22:12:51.695611 (+ 1018us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 22:12:51.695612 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 22:12:51.696286 (+ 674us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 22:12:51.696287 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 22:12:51.696378 (+ 91us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 22:12:51.696382 (+ 4us) client_negotiation.cc:769] Sending connection context
0923 22:12:51.696442 (+ 60us) client_negotiation.cc:240] Negotiation successful
0923 22:12:51.696506 (+ 64us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":92,"mutex_wait_us":21,"thread_start_us":50,"threads_started":1}
I20260923 22:12:51.696774 8460 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "b32070aba06045cabad174bee345066a" candidate_term: 2 candidate_status { last_received { term: 1 index: 12 } } ignore_live_leader: false dest_uuid: "d2a5b389b5d04491a85f778e638e056d" is_pre_election: true
I20260923 22:12:51.696877 8460 raft_consensus.cc:2454] T 00000000000000000000000000000000 P d2a5b389b5d04491a85f778e638e056d [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b32070aba06045cabad174bee345066a in term 1.
I20260923 22:12:51.697036 8251 leader_election.cc:302] T 00000000000000000000000000000000 P b32070aba06045cabad174bee345066a [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: b32070aba06045cabad174bee345066a, d2a5b389b5d04491a85f778e638e056d; no voters: 6378d422392b4a74831ae1c0f0692998
I20260923 22:12:51.697139 8967 raft_consensus.cc:2781] T 00000000000000000000000000000000 P b32070aba06045cabad174bee345066a [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260923 22:12:51.697182 8967 raft_consensus.cc:482] T 00000000000000000000000000000000 P b32070aba06045cabad174bee345066a [term 1 FOLLOWER]: Starting leader election (detected failure of leader 6378d422392b4a74831ae1c0f0692998)
I20260923 22:12:51.697212 8967 raft_consensus.cc:3037] T 00000000000000000000000000000000 P b32070aba06045cabad174bee345066a [term 1 FOLLOWER]: Advancing to term 2
I20260923 22:12:51.697340 8973 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 22:12:51.693421 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.135.253:39445 (local address 127.0.0.1:53538)
0923 22:12:51.693533 (+ 112us) negotiation.cc:107] Waiting for socket to connect
0923 22:12:51.693543 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0923 22:12:51.693606 (+ 63us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0923 22:12:51.694352 (+ 746us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0923 22:12:51.694353 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0923 22:12:51.694355 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0923 22:12:51.694389 (+ 34us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 22:12:51.694391 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 22:12:51.695236 (+ 845us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 22:12:51.695237 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 22:12:51.696268 (+ 1031us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 22:12:51.696270 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 22:12:51.697113 (+ 843us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 22:12:51.697114 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 22:12:51.697178 (+ 64us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 22:12:51.697183 (+ 5us) client_negotiation.cc:769] Sending connection context
0923 22:12:51.697244 (+ 61us) client_negotiation.cc:240] Negotiation successful
0923 22:12:51.697282 (+ 38us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":100,"thread_start_us":40,"threads_started":1}
I20260923 22:12:51.697542 8975 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 22:12:51.693533 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53538 (local address 127.1.135.253:39445)
0923 22:12:51.693688 (+ 155us) server_negotiation.cc:206] Beginning negotiation
0923 22:12:51.693691 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0923 22:12:51.693700 (+ 9us) server_negotiation.cc:407] Connection header received
0923 22:12:51.694266 (+ 566us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 22:12:51.694267 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 22:12:51.694278 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 22:12:51.694347 (+ 69us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0923 22:12:51.694453 (+ 106us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 22:12:51.695159 (+ 706us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 22:12:51.696350 (+ 1191us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 22:12:51.697012 (+ 662us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 22:12:51.697082 (+ 70us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 22:12:51.697093 (+ 11us) server_negotiation.cc:1035] Waiting for connection context
0923 22:12:51.697443 (+ 350us) server_negotiation.cc:299] Negotiation successful
0923 22:12:51.697485 (+ 42us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":137,"thread_start_us":73,"threads_started":1}
I20260923 22:12:51.697731 8275 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "d2a5b389b5d04491a85f778e638e056d" candidate_term: 2 candidate_status { last_received { term: 1 index: 12 } } ignore_live_leader: false dest_uuid: "b32070aba06045cabad174bee345066a" is_pre_election: true
I20260923 22:12:51.698134 8967 raft_consensus.cc:504] T 00000000000000000000000000000000 P b32070aba06045cabad174bee345066a [term 2 FOLLOWER]: Starting leader election with config: opid_index: 9 OBSOLETE_local: true peers { permanent_uuid: "6378d422392b4a74831ae1c0f0692998" member_type: VOTER last_known_addr { host: "127.1.135.254" port: 35949 } } peers { permanent_uuid: "b32070aba06045cabad174bee345066a" member_type: VOTER last_known_addr { host: "127.1.135.253" port: 39445 } attrs { promote: false } } peers { permanent_uuid: "d2a5b389b5d04491a85f778e638e056d" member_type: VOTER last_known_addr { host: "127.1.135.252" port: 42799 } attrs { promote: false } }
I20260923 22:12:51.698258 8275 raft_consensus.cc:2379] T 00000000000000000000000000000000 P b32070aba06045cabad174bee345066a [term 2 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate d2a5b389b5d04491a85f778e638e056d in current term 2: Already voted for candidate b32070aba06045cabad174bee345066a in this term.
I20260923 22:12:51.698285 8967 leader_election.cc:288] T 00000000000000000000000000000000 P b32070aba06045cabad174bee345066a [CANDIDATE]: Term 2 election: Requested vote from peers 6378d422392b4a74831ae1c0f0692998 (127.1.135.254:35949), d2a5b389b5d04491a85f778e638e056d (127.1.135.252:42799)
I20260923 22:12:51.698411 8460 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "b32070aba06045cabad174bee345066a" candidate_term: 2 candidate_status { last_received { term: 1 index: 12 } } ignore_live_leader: false dest_uuid: "d2a5b389b5d04491a85f778e638e056d"
I20260923 22:12:51.698438 8436 leader_election.cc:302] T 00000000000000000000000000000000 P d2a5b389b5d04491a85f778e638e056d [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d2a5b389b5d04491a85f778e638e056d; no voters: 6378d422392b4a74831ae1c0f0692998, b32070aba06045cabad174bee345066a
I20260923 22:12:51.698473 8460 raft_consensus.cc:3037] T 00000000000000000000000000000000 P d2a5b389b5d04491a85f778e638e056d [term 1 FOLLOWER]: Advancing to term 2
I20260923 22:12:51.698642 8970 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 22:12:51.698355 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.135.254:35949 (local address 127.0.0.1:52554)
0923 22:12:51.698593 (+ 238us) negotiation.cc:107] Waiting for socket to connect
0923 22:12:51.698620 (+ 27us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.135.254:35949: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":213}
I20260923 22:12:51.699128 8460 raft_consensus.cc:2454] T 00000000000000000000000000000000 P d2a5b389b5d04491a85f778e638e056d [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b32070aba06045cabad174bee345066a in term 2.
I20260923 22:12:51.699204 8968 raft_consensus.cc:2726] T 00000000000000000000000000000000 P d2a5b389b5d04491a85f778e638e056d [term 2 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260923 22:12:51.699329 8251 leader_election.cc:302] T 00000000000000000000000000000000 P b32070aba06045cabad174bee345066a [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: b32070aba06045cabad174bee345066a, d2a5b389b5d04491a85f778e638e056d; no voters:
I20260923 22:12:51.699445 8967 raft_consensus.cc:2781] T 00000000000000000000000000000000 P b32070aba06045cabad174bee345066a [term 2 FOLLOWER]: Leader election won for term 2
I20260923 22:12:51.699517 8970 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 22:12:51.699435 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.135.254:35949 (local address 127.0.0.1:52556)
0923 22:12:51.699467 (+ 32us) negotiation.cc:107] Waiting for socket to connect
0923 22:12:51.699490 (+ 23us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.135.254:35949: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":17}
I20260923 22:12:51.699596 8967 raft_consensus.cc:686] T 00000000000000000000000000000000 P b32070aba06045cabad174bee345066a [term 2 LEADER]: Becoming Leader. State: Replica: b32070aba06045cabad174bee345066a, State: Running, Role: LEADER
W20260923 22:12:51.699601 8250 leader_election.cc:334] T 00000000000000000000000000000000 P b32070aba06045cabad174bee345066a [CANDIDATE]: Term 2 election: RPC error from VoteRequest() call to peer 6378d422392b4a74831ae1c0f0692998 (127.1.135.254:35949): Network error: Client connection negotiation failed: client connection to 127.1.135.254:35949: connect: Connection refused (error 111)
I20260923 22:12:51.699683 8967 consensus_queue.cc:237] T 00000000000000000000000000000000 P b32070aba06045cabad174bee345066a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 1.12, Last appended by leader: 12, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 9 OBSOLETE_local: true peers { permanent_uuid: "6378d422392b4a74831ae1c0f0692998" member_type: VOTER last_known_addr { host: "127.1.135.254" port: 35949 } } peers { permanent_uuid: "b32070aba06045cabad174bee345066a" member_type: VOTER last_known_addr { host: "127.1.135.253" port: 39445 } attrs { promote: false } } peers { permanent_uuid: "d2a5b389b5d04491a85f778e638e056d" member_type: VOTER last_known_addr { host: "127.1.135.252" port: 42799 } attrs { promote: false } }
I20260923 22:12:51.700021 8977 sys_catalog.cc:455] T 00000000000000000000000000000000 P b32070aba06045cabad174bee345066a [sys.catalog]: SysCatalogTable state changed. Reason: New leader b32070aba06045cabad174bee345066a. Latest consensus state: current_term: 2 leader_uuid: "b32070aba06045cabad174bee345066a" committed_config { opid_index: 9 OBSOLETE_local: true peers { permanent_uuid: "6378d422392b4a74831ae1c0f0692998" member_type: VOTER last_known_addr { host: "127.1.135.254" port: 35949 } } peers { permanent_uuid: "b32070aba06045cabad174bee345066a" member_type: VOTER last_known_addr { host: "127.1.135.253" port: 39445 } attrs { promote: false } } peers { permanent_uuid: "d2a5b389b5d04491a85f778e638e056d" member_type: VOTER last_known_addr { host: "127.1.135.252" port: 42799 } attrs { promote: false } } }
I20260923 22:12:51.700153 8977 sys_catalog.cc:458] T 00000000000000000000000000000000 P b32070aba06045cabad174bee345066a [sys.catalog]: This master's current role is: LEADER
I20260923 22:12:51.700403 8979 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260923 22:12:51.700837 8979 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260923 22:12:51.700918 8979 catalog_manager.cc:1259] Loaded cluster ID: e03b1c9cbcbb49658f49f0be6e6c5546
I20260923 22:12:51.700950 8979 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260923 22:12:51.701148 8979 catalog_manager.cc:1504] Loading token signing keys...
I20260923 22:12:51.701243 8979 catalog_manager.cc:5879] T 00000000000000000000000000000000 P b32070aba06045cabad174bee345066a: Loaded TSK: 0
I20260923 22:12:51.701402 8979 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260923 22:12:51.758455 8265 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:53534:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260923 22:12:51.760015 8265 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260923 22:12:51.761075 8970 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 22:12:51.760944 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.135.254:35949 (local address 127.0.0.1:52574)
0923 22:12:51.761000 (+ 56us) negotiation.cc:107] Waiting for socket to connect
0923 22:12:51.761034 (+ 34us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.135.254:35949: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":37}
I20260923 22:12:51.761058 8460 raft_consensus.cc:1261] T 00000000000000000000000000000000 P d2a5b389b5d04491a85f778e638e056d [term 2 FOLLOWER]: Refusing update from remote peer b32070aba06045cabad174bee345066a: Log matching property violated. Preceding OpId in replica: term: 1 index: 12. Preceding OpId from leader: term: 2 index: 14. (index mismatch)
I20260923 22:12:51.761281 8967 consensus_queue.cc:1035] T 00000000000000000000000000000000 P b32070aba06045cabad174bee345066a [LEADER]: Connected to new peer: Peer: permanent_uuid: "d2a5b389b5d04491a85f778e638e056d" member_type: VOTER last_known_addr { host: "127.1.135.252" port: 42799 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.000s
I20260923 22:12:51.761422 8970 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 22:12:51.761297 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.135.254:35949 (local address 127.0.0.1:52588)
0923 22:12:51.761349 (+ 52us) negotiation.cc:107] Waiting for socket to connect
0923 22:12:51.761389 (+ 40us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.135.254:35949: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":37}
W20260923 22:12:51.761490 8250 consensus_peers.cc:487] T 00000000000000000000000000000000 P b32070aba06045cabad174bee345066a -> Peer 6378d422392b4a74831ae1c0f0692998 (127.1.135.254:35949): Couldn't send request to peer 6378d422392b4a74831ae1c0f0692998. Status: Network error: Client connection negotiation failed: client connection to 127.1.135.254:35949: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
I20260923 22:12:51.762025 8968 sys_catalog.cc:455] T 00000000000000000000000000000000 P d2a5b389b5d04491a85f778e638e056d [sys.catalog]: SysCatalogTable state changed. Reason: New leader b32070aba06045cabad174bee345066a. Latest consensus state: current_term: 2 leader_uuid: "b32070aba06045cabad174bee345066a" committed_config { opid_index: 9 OBSOLETE_local: true peers { permanent_uuid: "6378d422392b4a74831ae1c0f0692998" member_type: VOTER last_known_addr { host: "127.1.135.254" port: 35949 } } peers { permanent_uuid: "b32070aba06045cabad174bee345066a" member_type: VOTER last_known_addr { host: "127.1.135.253" port: 39445 } attrs { promote: false } } peers { permanent_uuid: "d2a5b389b5d04491a85f778e638e056d" member_type: VOTER last_known_addr { host: "127.1.135.252" port: 42799 } attrs { promote: false } } }
I20260923 22:12:51.762132 8968 sys_catalog.cc:458] T 00000000000000000000000000000000 P d2a5b389b5d04491a85f778e638e056d [sys.catalog]: This master's current role is: FOLLOWER
I20260923 22:12:51.762570 8967 sys_catalog.cc:455] T 00000000000000000000000000000000 P b32070aba06045cabad174bee345066a [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 2 leader_uuid: "b32070aba06045cabad174bee345066a" committed_config { opid_index: 9 OBSOLETE_local: true peers { permanent_uuid: "6378d422392b4a74831ae1c0f0692998" member_type: VOTER last_known_addr { host: "127.1.135.254" port: 35949 } } peers { permanent_uuid: "b32070aba06045cabad174bee345066a" member_type: VOTER last_known_addr { host: "127.1.135.253" port: 39445 } attrs { promote: false } } peers { permanent_uuid: "d2a5b389b5d04491a85f778e638e056d" member_type: VOTER last_known_addr { host: "127.1.135.252" port: 42799 } attrs { promote: false } } }
I20260923 22:12:51.762660 8967 sys_catalog.cc:458] T 00000000000000000000000000000000 P b32070aba06045cabad174bee345066a [sys.catalog]: This master's current role is: LEADER
I20260923 22:12:51.762678 8968 sys_catalog.cc:455] T 00000000000000000000000000000000 P d2a5b389b5d04491a85f778e638e056d [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 2 leader_uuid: "b32070aba06045cabad174bee345066a" committed_config { opid_index: 9 OBSOLETE_local: true peers { permanent_uuid: "6378d422392b4a74831ae1c0f0692998" member_type: VOTER last_known_addr { host: "127.1.135.254" port: 35949 } } peers { permanent_uuid: "b32070aba06045cabad174bee345066a" member_type: VOTER last_known_addr { host: "127.1.135.253" port: 39445 } attrs { promote: false } } peers { permanent_uuid: "d2a5b389b5d04491a85f778e638e056d" member_type: VOTER last_known_addr { host: "127.1.135.252" port: 42799 } attrs { promote: false } } }
I20260923 22:12:51.762763 8968 sys_catalog.cc:458] T 00000000000000000000000000000000 P d2a5b389b5d04491a85f778e638e056d [sys.catalog]: This master's current role is: FOLLOWER
I20260923 22:12:51.763226 8981 mvcc.cc:205] Tried to move back new op lower bound from 7332665637931102208 to 7332665637682286592. Current Snapshot: MvccSnapshot[applied={T|T < 7332665637931102208}]
I20260923 22:12:51.767601 8969 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 22:12:51.764311 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.135.193:44597 (local address 127.0.0.1:45734)
0923 22:12:51.764346 (+ 35us) negotiation.cc:107] Waiting for socket to connect
0923 22:12:51.764356 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0923 22:12:51.764401 (+ 45us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0923 22:12:51.764859 (+ 458us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0923 22:12:51.764860 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0923 22:12:51.764862 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0923 22:12:51.764903 (+ 41us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 22:12:51.764904 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 22:12:51.765805 (+ 901us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 22:12:51.765806 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 22:12:51.766624 (+ 818us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 22:12:51.766626 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 22:12:51.767358 (+ 732us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 22:12:51.767359 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 22:12:51.767418 (+ 59us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 22:12:51.767421 (+ 3us) client_negotiation.cc:769] Sending connection context
0923 22:12:51.767487 (+ 66us) client_negotiation.cc:240] Negotiation successful
0923 22:12:51.767529 (+ 42us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":17,"mutex_wait_us":3}
I20260923 22:12:51.767714 8989 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 22:12:51.764626 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45734 (local address 127.1.135.193:44597)
0923 22:12:51.764730 (+ 104us) server_negotiation.cc:206] Beginning negotiation
0923 22:12:51.764731 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0923 22:12:51.764738 (+ 7us) server_negotiation.cc:407] Connection header received
0923 22:12:51.764763 (+ 25us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 22:12:51.764764 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 22:12:51.764774 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 22:12:51.764963 (+ 189us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0923 22:12:51.764999 (+ 36us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 22:12:51.765713 (+ 714us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 22:12:51.766726 (+ 1013us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 22:12:51.767260 (+ 534us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 22:12:51.767349 (+ 89us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 22:12:51.767361 (+ 12us) server_negotiation.cc:1035] Waiting for connection context
0923 22:12:51.767587 (+ 226us) server_negotiation.cc:299] Negotiation successful
0923 22:12:51.767630 (+ 43us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":86,"thread_start_us":56,"threads_started":1}
I20260923 22:12:51.767884 8970 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 22:12:51.764246 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.135.195:33563 (local address 127.0.0.1:57678)
0923 22:12:51.764285 (+ 39us) negotiation.cc:107] Waiting for socket to connect
0923 22:12:51.764295 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0923 22:12:51.764332 (+ 37us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0923 22:12:51.764849 (+ 517us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0923 22:12:51.764850 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0923 22:12:51.764851 (+ 1us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0923 22:12:51.764899 (+ 48us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 22:12:51.764900 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 22:12:51.765570 (+ 670us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 22:12:51.765571 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 22:12:51.766777 (+ 1206us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 22:12:51.766779 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 22:12:51.767662 (+ 883us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 22:12:51.767664 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 22:12:51.767736 (+ 72us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 22:12:51.767739 (+ 3us) client_negotiation.cc:769] Sending connection context
0923 22:12:51.767767 (+ 28us) client_negotiation.cc:240] Negotiation successful
0923 22:12:51.767819 (+ 52us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":26,"mutex_wait_us":43}
I20260923 22:12:51.767889 8564 tablet_service.cc:1467] Processing CreateTablet for tablet e9ed3afc39914c9bbb09a3a374df7670 (DEFAULT_TABLE table=test-table [id=d8fabe81fa1a42c28b8b0ad6258d7c11]), partition=RANGE (key) PARTITION UNBOUNDED
I20260923 22:12:51.768039 8564 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet e9ed3afc39914c9bbb09a3a374df7670. 1 dirs total, 0 dirs full, 0 dirs failed
I20260923 22:12:51.768149 8987 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 22:12:51.764424 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.135.194:41925 (local address 127.0.0.1:34800)
0923 22:12:51.764532 (+ 108us) negotiation.cc:107] Waiting for socket to connect
0923 22:12:51.764541 (+ 9us) client_negotiation.cc:174] Beginning negotiation
0923 22:12:51.764578 (+ 37us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0923 22:12:51.764769 (+ 191us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0923 22:12:51.764771 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0923 22:12:51.764773 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0923 22:12:51.764830 (+ 57us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 22:12:51.764832 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 22:12:51.765760 (+ 928us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 22:12:51.765761 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 22:12:51.766883 (+ 1122us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 22:12:51.766885 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 22:12:51.767867 (+ 982us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 22:12:51.767868 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 22:12:51.767924 (+ 56us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 22:12:51.767928 (+ 4us) client_negotiation.cc:769] Sending connection context
0923 22:12:51.767980 (+ 52us) client_negotiation.cc:240] Negotiation successful
0923 22:12:51.768025 (+ 45us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":93,"thread_start_us":76,"threads_started":1}
I20260923 22:12:51.768307 8986 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 22:12:51.764305 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57678 (local address 127.1.135.195:33563)
0923 22:12:51.764710 (+ 405us) server_negotiation.cc:206] Beginning negotiation
0923 22:12:51.764713 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0923 22:12:51.764721 (+ 8us) server_negotiation.cc:407] Connection header received
0923 22:12:51.764747 (+ 26us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 22:12:51.764748 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 22:12:51.764759 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 22:12:51.764965 (+ 206us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0923 22:12:51.764996 (+ 31us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 22:12:51.765468 (+ 472us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 22:12:51.766968 (+ 1500us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 22:12:51.767574 (+ 606us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 22:12:51.768157 (+ 583us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 22:12:51.768172 (+ 15us) server_negotiation.cc:1035] Waiting for connection context
0923 22:12:51.768192 (+ 20us) server_negotiation.cc:299] Negotiation successful
0923 22:12:51.768242 (+ 50us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":383,"thread_start_us":50,"threads_started":1}
I20260923 22:12:51.768409 8988 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 22:12:51.764501 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34800 (local address 127.1.135.194:41925)
0923 22:12:51.764599 (+ 98us) server_negotiation.cc:206] Beginning negotiation
0923 22:12:51.764602 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0923 22:12:51.764612 (+ 10us) server_negotiation.cc:407] Connection header received
0923 22:12:51.764639 (+ 27us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 22:12:51.764640 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 22:12:51.764651 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 22:12:51.764715 (+ 64us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0923 22:12:51.764907 (+ 192us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 22:12:51.765683 (+ 776us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 22:12:51.766954 (+ 1271us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 22:12:51.767529 (+ 575us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 22:12:51.767605 (+ 76us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 22:12:51.767617 (+ 12us) server_negotiation.cc:1035] Waiting for connection context
0923 22:12:51.768280 (+ 663us) server_negotiation.cc:299] Negotiation successful
0923 22:12:51.768328 (+ 48us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":79,"thread_start_us":46,"threads_started":1}
I20260923 22:12:51.768473 8837 tablet_service.cc:1467] Processing CreateTablet for tablet e9ed3afc39914c9bbb09a3a374df7670 (DEFAULT_TABLE table=test-table [id=d8fabe81fa1a42c28b8b0ad6258d7c11]), partition=RANGE (key) PARTITION UNBOUNDED
I20260923 22:12:51.768594 8837 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet e9ed3afc39914c9bbb09a3a374df7670. 1 dirs total, 0 dirs full, 0 dirs failed
I20260923 22:12:51.768646 8701 tablet_service.cc:1467] Processing CreateTablet for tablet e9ed3afc39914c9bbb09a3a374df7670 (DEFAULT_TABLE table=test-table [id=d8fabe81fa1a42c28b8b0ad6258d7c11]), partition=RANGE (key) PARTITION UNBOUNDED
I20260923 22:12:51.768743 8701 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet e9ed3afc39914c9bbb09a3a374df7670. 1 dirs total, 0 dirs full, 0 dirs failed
I20260923 22:12:51.770058 8990 tablet_bootstrap.cc:492] T e9ed3afc39914c9bbb09a3a374df7670 P e01531ae61ec4025942988b1a6dd62f0: Bootstrap starting.
I20260923 22:12:51.770040 8991 tablet_bootstrap.cc:492] T e9ed3afc39914c9bbb09a3a374df7670 P 4d0b914e7ab746e7b17789e7b32aca0a: Bootstrap starting.
I20260923 22:12:51.770390 8991 tablet_bootstrap.cc:654] T e9ed3afc39914c9bbb09a3a374df7670 P 4d0b914e7ab746e7b17789e7b32aca0a: Neither blocks nor log segments found. Creating new log.
I20260923 22:12:51.770452 8990 tablet_bootstrap.cc:654] T e9ed3afc39914c9bbb09a3a374df7670 P e01531ae61ec4025942988b1a6dd62f0: Neither blocks nor log segments found. Creating new log.
I20260923 22:12:51.770630 8992 tablet_bootstrap.cc:492] T e9ed3afc39914c9bbb09a3a374df7670 P 8a9da5f92d7c4186877987e0ddd62caf: Bootstrap starting.
I20260923 22:12:51.771008 8992 tablet_bootstrap.cc:654] T e9ed3afc39914c9bbb09a3a374df7670 P 8a9da5f92d7c4186877987e0ddd62caf: Neither blocks nor log segments found. Creating new log.
I20260923 22:12:51.771140 8991 tablet_bootstrap.cc:492] T e9ed3afc39914c9bbb09a3a374df7670 P 4d0b914e7ab746e7b17789e7b32aca0a: No bootstrap required, opened a new log
I20260923 22:12:51.771201 8991 ts_tablet_manager.cc:1397] T e9ed3afc39914c9bbb09a3a374df7670 P 4d0b914e7ab746e7b17789e7b32aca0a: Time spent bootstrapping tablet: real 0.001s user 0.001s sys 0.000s
I20260923 22:12:51.771323 8991 raft_consensus.cc:348] T e9ed3afc39914c9bbb09a3a374df7670 P 4d0b914e7ab746e7b17789e7b32aca0a [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4d0b914e7ab746e7b17789e7b32aca0a" member_type: VOTER last_known_addr { host: "127.1.135.195" port: 33563 } } peers { permanent_uuid: "e01531ae61ec4025942988b1a6dd62f0" member_type: VOTER last_known_addr { host: "127.1.135.193" port: 44597 } } peers { permanent_uuid: "8a9da5f92d7c4186877987e0ddd62caf" member_type: VOTER last_known_addr { host: "127.1.135.194" port: 41925 } }
I20260923 22:12:51.771409 8991 raft_consensus.cc:374] T e9ed3afc39914c9bbb09a3a374df7670 P 4d0b914e7ab746e7b17789e7b32aca0a [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260923 22:12:51.771435 8991 raft_consensus.cc:729] T e9ed3afc39914c9bbb09a3a374df7670 P 4d0b914e7ab746e7b17789e7b32aca0a [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 4d0b914e7ab746e7b17789e7b32aca0a, State: Initialized, Role: FOLLOWER
I20260923 22:12:51.771503 8991 consensus_queue.cc:260] T e9ed3afc39914c9bbb09a3a374df7670 P 4d0b914e7ab746e7b17789e7b32aca0a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4d0b914e7ab746e7b17789e7b32aca0a" member_type: VOTER last_known_addr { host: "127.1.135.195" port: 33563 } } peers { permanent_uuid: "e01531ae61ec4025942988b1a6dd62f0" member_type: VOTER last_known_addr { host: "127.1.135.193" port: 44597 } } peers { permanent_uuid: "8a9da5f92d7c4186877987e0ddd62caf" member_type: VOTER last_known_addr { host: "127.1.135.194" port: 41925 } }
I20260923 22:12:51.771673 8991 ts_tablet_manager.cc:1428] T e9ed3afc39914c9bbb09a3a374df7670 P 4d0b914e7ab746e7b17789e7b32aca0a: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20260923 22:12:51.771859 8992 tablet_bootstrap.cc:492] T e9ed3afc39914c9bbb09a3a374df7670 P 8a9da5f92d7c4186877987e0ddd62caf: No bootstrap required, opened a new log
I20260923 22:12:51.771955 8992 ts_tablet_manager.cc:1397] T e9ed3afc39914c9bbb09a3a374df7670 P 8a9da5f92d7c4186877987e0ddd62caf: Time spent bootstrapping tablet: real 0.001s user 0.001s sys 0.000s
I20260923 22:12:51.772024 8990 tablet_bootstrap.cc:492] T e9ed3afc39914c9bbb09a3a374df7670 P e01531ae61ec4025942988b1a6dd62f0: No bootstrap required, opened a new log
I20260923 22:12:51.772044 8997 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 22:12:51.771818 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.135.254:35949 (local address 127.1.135.195:38027)
0923 22:12:51.771961 (+ 143us) negotiation.cc:107] Waiting for socket to connect
0923 22:12:51.772001 (+ 40us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.135.254:35949: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":118,"thread_start_us":72,"threads_started":1}
I20260923 22:12:51.772107 8990 ts_tablet_manager.cc:1397] T e9ed3afc39914c9bbb09a3a374df7670 P e01531ae61ec4025942988b1a6dd62f0: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20260923 22:12:51.772220 8990 raft_consensus.cc:348] T e9ed3afc39914c9bbb09a3a374df7670 P e01531ae61ec4025942988b1a6dd62f0 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4d0b914e7ab746e7b17789e7b32aca0a" member_type: VOTER last_known_addr { host: "127.1.135.195" port: 33563 } } peers { permanent_uuid: "e01531ae61ec4025942988b1a6dd62f0" member_type: VOTER last_known_addr { host: "127.1.135.193" port: 44597 } } peers { permanent_uuid: "8a9da5f92d7c4186877987e0ddd62caf" member_type: VOTER last_known_addr { host: "127.1.135.194" port: 41925 } }
I20260923 22:12:51.772289 8990 raft_consensus.cc:374] T e9ed3afc39914c9bbb09a3a374df7670 P e01531ae61ec4025942988b1a6dd62f0 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260923 22:12:51.772313 8990 raft_consensus.cc:729] T e9ed3afc39914c9bbb09a3a374df7670 P e01531ae61ec4025942988b1a6dd62f0 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e01531ae61ec4025942988b1a6dd62f0, State: Initialized, Role: FOLLOWER
I20260923 22:12:51.772373 8990 consensus_queue.cc:260] T e9ed3afc39914c9bbb09a3a374df7670 P e01531ae61ec4025942988b1a6dd62f0 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4d0b914e7ab746e7b17789e7b32aca0a" member_type: VOTER last_known_addr { host: "127.1.135.195" port: 33563 } } peers { permanent_uuid: "e01531ae61ec4025942988b1a6dd62f0" member_type: VOTER last_known_addr { host: "127.1.135.193" port: 44597 } } peers { permanent_uuid: "8a9da5f92d7c4186877987e0ddd62caf" member_type: VOTER last_known_addr { host: "127.1.135.194" port: 41925 } }
I20260923 22:12:51.772408 8992 raft_consensus.cc:348] T e9ed3afc39914c9bbb09a3a374df7670 P 8a9da5f92d7c4186877987e0ddd62caf [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4d0b914e7ab746e7b17789e7b32aca0a" member_type: VOTER last_known_addr { host: "127.1.135.195" port: 33563 } } peers { permanent_uuid: "e01531ae61ec4025942988b1a6dd62f0" member_type: VOTER last_known_addr { host: "127.1.135.193" port: 44597 } } peers { permanent_uuid: "8a9da5f92d7c4186877987e0ddd62caf" member_type: VOTER last_known_addr { host: "127.1.135.194" port: 41925 } }
I20260923 22:12:51.772524 8990 ts_tablet_manager.cc:1428] T e9ed3afc39914c9bbb09a3a374df7670 P e01531ae61ec4025942988b1a6dd62f0: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20260923 22:12:51.772590 8992 raft_consensus.cc:374] T e9ed3afc39914c9bbb09a3a374df7670 P 8a9da5f92d7c4186877987e0ddd62caf [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260923 22:12:51.772656 8992 raft_consensus.cc:729] T e9ed3afc39914c9bbb09a3a374df7670 P 8a9da5f92d7c4186877987e0ddd62caf [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 8a9da5f92d7c4186877987e0ddd62caf, State: Initialized, Role: FOLLOWER
I20260923 22:12:51.772781 8997 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 22:12:51.772680 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.135.254:35949 (local address 127.1.135.195:48581)
0923 22:12:51.772728 (+ 48us) negotiation.cc:107] Waiting for socket to connect
0923 22:12:51.772754 (+ 26us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.135.254:35949: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":32}
I20260923 22:12:51.772760 8992 consensus_queue.cc:260] T e9ed3afc39914c9bbb09a3a374df7670 P 8a9da5f92d7c4186877987e0ddd62caf [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4d0b914e7ab746e7b17789e7b32aca0a" member_type: VOTER last_known_addr { host: "127.1.135.195" port: 33563 } } peers { permanent_uuid: "e01531ae61ec4025942988b1a6dd62f0" member_type: VOTER last_known_addr { host: "127.1.135.193" port: 44597 } } peers { permanent_uuid: "8a9da5f92d7c4186877987e0ddd62caf" member_type: VOTER last_known_addr { host: "127.1.135.194" port: 41925 } }
I20260923 22:12:51.772943 8905 heartbeater.cc:503] Master 127.1.135.253:39445 was elected leader, sending a full tablet report...
I20260923 22:12:51.773077 8992 ts_tablet_manager.cc:1428] T e9ed3afc39914c9bbb09a3a374df7670 P 8a9da5f92d7c4186877987e0ddd62caf: Time spent starting tablet: real 0.001s user 0.000s sys 0.000s
I20260923 22:12:51.773353 8632 heartbeater.cc:503] Master 127.1.135.253:39445 was elected leader, sending a full tablet report...
I20260923 22:12:51.773734 9001 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 22:12:51.773298 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.135.254:35949 (local address 127.1.135.194:39617)
0923 22:12:51.773617 (+ 319us) negotiation.cc:107] Waiting for socket to connect
0923 22:12:51.773698 (+ 81us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.135.254:35949: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":294,"thread_start_us":66,"threads_started":1}
I20260923 22:12:51.774003 8999 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 22:12:51.772859 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.135.254:35949 (local address 127.1.135.193:57887)
0923 22:12:51.773945 (+ 1086us) negotiation.cc:107] Waiting for socket to connect
0923 22:12:51.773980 (+ 35us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.135.254:35949: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":1065,"thread_start_us":69,"threads_started":1}
I20260923 22:12:51.774207 8769 heartbeater.cc:503] Master 127.1.135.253:39445 was elected leader, sending a full tablet report...
I20260923 22:12:51.774420 9001 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 22:12:51.774229 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.135.254:35949 (local address 127.1.135.194:42725)
0923 22:12:51.774358 (+ 129us) negotiation.cc:107] Waiting for socket to connect
0923 22:12:51.774400 (+ 42us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.135.254:35949: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":118}
I20260923 22:12:51.774575 8999 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 22:12:51.774263 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.135.254:35949 (local address 127.1.135.193:43795)
0923 22:12:51.774527 (+ 264us) negotiation.cc:107] Waiting for socket to connect
0923 22:12:51.774554 (+ 27us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.135.254:35949: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":247}
I20260923 22:12:51.823985 8996 raft_consensus.cc:482] T e9ed3afc39914c9bbb09a3a374df7670 P 4d0b914e7ab746e7b17789e7b32aca0a [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260923 22:12:51.824119 8996 raft_consensus.cc:504] T e9ed3afc39914c9bbb09a3a374df7670 P 4d0b914e7ab746e7b17789e7b32aca0a [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4d0b914e7ab746e7b17789e7b32aca0a" member_type: VOTER last_known_addr { host: "127.1.135.195" port: 33563 } } peers { permanent_uuid: "e01531ae61ec4025942988b1a6dd62f0" member_type: VOTER last_known_addr { host: "127.1.135.193" port: 44597 } } peers { permanent_uuid: "8a9da5f92d7c4186877987e0ddd62caf" member_type: VOTER last_known_addr { host: "127.1.135.194" port: 41925 } }
I20260923 22:12:51.824493 8996 leader_election.cc:288] T e9ed3afc39914c9bbb09a3a374df7670 P 4d0b914e7ab746e7b17789e7b32aca0a [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers e01531ae61ec4025942988b1a6dd62f0 (127.1.135.193:44597), 8a9da5f92d7c4186877987e0ddd62caf (127.1.135.194:41925)
I20260923 22:12:51.827283 8997 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 22:12:51.824664 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.135.193:44597 (local address 127.1.135.195:46595)
0923 22:12:51.824742 (+ 78us) negotiation.cc:107] Waiting for socket to connect
0923 22:12:51.824755 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0923 22:12:51.824807 (+ 52us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0923 22:12:51.825011 (+ 204us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0923 22:12:51.825012 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0923 22:12:51.825013 (+ 1us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0923 22:12:51.825064 (+ 51us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 22:12:51.825065 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 22:12:51.825701 (+ 636us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 22:12:51.825702 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 22:12:51.826451 (+ 749us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 22:12:51.826453 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 22:12:51.827023 (+ 570us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 22:12:51.827024 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 22:12:51.827103 (+ 79us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 22:12:51.827106 (+ 3us) client_negotiation.cc:769] Sending connection context
0923 22:12:51.827168 (+ 62us) client_negotiation.cc:240] Negotiation successful
0923 22:12:51.827206 (+ 38us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":56}
I20260923 22:12:51.827284 8989 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 22:12:51.824721 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.135.195:46595 (local address 127.1.135.193:44597)
0923 22:12:51.824783 (+ 62us) server_negotiation.cc:206] Beginning negotiation
0923 22:12:51.824785 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0923 22:12:51.824798 (+ 13us) server_negotiation.cc:407] Connection header received
0923 22:12:51.824899 (+ 101us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 22:12:51.824900 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 22:12:51.824911 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 22:12:51.824943 (+ 32us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0923 22:12:51.825136 (+ 193us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 22:12:51.825618 (+ 482us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 22:12:51.826586 (+ 968us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 22:12:51.826937 (+ 351us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 22:12:51.827023 (+ 86us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 22:12:51.827034 (+ 11us) server_negotiation.cc:1035] Waiting for connection context
0923 22:12:51.827187 (+ 153us) server_negotiation.cc:299] Negotiation successful
0923 22:12:51.827223 (+ 36us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":36}
I20260923 22:12:51.827396 8988 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 22:12:51.824846 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.135.195:34771 (local address 127.1.135.194:41925)
0923 22:12:51.824887 (+ 41us) server_negotiation.cc:206] Beginning negotiation
0923 22:12:51.824889 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0923 22:12:51.825007 (+ 118us) server_negotiation.cc:407] Connection header received
0923 22:12:51.825089 (+ 82us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 22:12:51.825089 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 22:12:51.825099 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 22:12:51.825167 (+ 68us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0923 22:12:51.825325 (+ 158us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 22:12:51.825779 (+ 454us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 22:12:51.826701 (+ 922us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 22:12:51.827048 (+ 347us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 22:12:51.827132 (+ 84us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 22:12:51.827142 (+ 10us) server_negotiation.cc:1035] Waiting for connection context
0923 22:12:51.827300 (+ 158us) server_negotiation.cc:299] Negotiation successful
0923 22:12:51.827334 (+ 34us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":20}
I20260923 22:12:51.827423 9002 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 22:12:51.824771 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.135.194:41925 (local address 127.1.135.195:34771)
0923 22:12:51.824931 (+ 160us) negotiation.cc:107] Waiting for socket to connect
0923 22:12:51.824941 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0923 22:12:51.825024 (+ 83us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0923 22:12:51.825183 (+ 159us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0923 22:12:51.825183 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0923 22:12:51.825185 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0923 22:12:51.825246 (+ 61us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 22:12:51.825248 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 22:12:51.825870 (+ 622us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 22:12:51.825871 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 22:12:51.826614 (+ 743us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 22:12:51.826615 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 22:12:51.827156 (+ 541us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 22:12:51.827157 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 22:12:51.827218 (+ 61us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 22:12:51.827229 (+ 11us) client_negotiation.cc:769] Sending connection context
0923 22:12:51.827295 (+ 66us) client_negotiation.cc:240] Negotiation successful
0923 22:12:51.827330 (+ 35us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":134,"thread_start_us":111,"threads_started":1}
I20260923 22:12:51.827534 8584 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "e9ed3afc39914c9bbb09a3a374df7670" candidate_uuid: "4d0b914e7ab746e7b17789e7b32aca0a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e01531ae61ec4025942988b1a6dd62f0" is_pre_election: true
I20260923 22:12:51.827684 8584 raft_consensus.cc:2454] T e9ed3afc39914c9bbb09a3a374df7670 P e01531ae61ec4025942988b1a6dd62f0 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4d0b914e7ab746e7b17789e7b32aca0a in term 0.
I20260923 22:12:51.827919 8791 leader_election.cc:302] T e9ed3afc39914c9bbb09a3a374df7670 P 4d0b914e7ab746e7b17789e7b32aca0a [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4d0b914e7ab746e7b17789e7b32aca0a, e01531ae61ec4025942988b1a6dd62f0; no voters:
I20260923 22:12:51.827963 8721 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "e9ed3afc39914c9bbb09a3a374df7670" candidate_uuid: "4d0b914e7ab746e7b17789e7b32aca0a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8a9da5f92d7c4186877987e0ddd62caf" is_pre_election: true
I20260923 22:12:51.828047 8721 raft_consensus.cc:2454] T e9ed3afc39914c9bbb09a3a374df7670 P 8a9da5f92d7c4186877987e0ddd62caf [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4d0b914e7ab746e7b17789e7b32aca0a in term 0.
I20260923 22:12:51.828095 8996 raft_consensus.cc:2781] T e9ed3afc39914c9bbb09a3a374df7670 P 4d0b914e7ab746e7b17789e7b32aca0a [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260923 22:12:51.828136 8996 raft_consensus.cc:482] T e9ed3afc39914c9bbb09a3a374df7670 P 4d0b914e7ab746e7b17789e7b32aca0a [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260923 22:12:51.828156 8996 raft_consensus.cc:3037] T e9ed3afc39914c9bbb09a3a374df7670 P 4d0b914e7ab746e7b17789e7b32aca0a [term 0 FOLLOWER]: Advancing to term 1
I20260923 22:12:51.828843 8996 raft_consensus.cc:504] T e9ed3afc39914c9bbb09a3a374df7670 P 4d0b914e7ab746e7b17789e7b32aca0a [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4d0b914e7ab746e7b17789e7b32aca0a" member_type: VOTER last_known_addr { host: "127.1.135.195" port: 33563 } } peers { permanent_uuid: "e01531ae61ec4025942988b1a6dd62f0" member_type: VOTER last_known_addr { host: "127.1.135.193" port: 44597 } } peers { permanent_uuid: "8a9da5f92d7c4186877987e0ddd62caf" member_type: VOTER last_known_addr { host: "127.1.135.194" port: 41925 } }
I20260923 22:12:51.828967 8996 leader_election.cc:288] T e9ed3afc39914c9bbb09a3a374df7670 P 4d0b914e7ab746e7b17789e7b32aca0a [CANDIDATE]: Term 1 election: Requested vote from peers e01531ae61ec4025942988b1a6dd62f0 (127.1.135.193:44597), 8a9da5f92d7c4186877987e0ddd62caf (127.1.135.194:41925)
I20260923 22:12:51.829142 8584 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "e9ed3afc39914c9bbb09a3a374df7670" candidate_uuid: "4d0b914e7ab746e7b17789e7b32aca0a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e01531ae61ec4025942988b1a6dd62f0"
I20260923 22:12:51.829149 8721 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "e9ed3afc39914c9bbb09a3a374df7670" candidate_uuid: "4d0b914e7ab746e7b17789e7b32aca0a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8a9da5f92d7c4186877987e0ddd62caf"
I20260923 22:12:51.829211 8584 raft_consensus.cc:3037] T e9ed3afc39914c9bbb09a3a374df7670 P e01531ae61ec4025942988b1a6dd62f0 [term 0 FOLLOWER]: Advancing to term 1
I20260923 22:12:51.829231 8721 raft_consensus.cc:3037] T e9ed3afc39914c9bbb09a3a374df7670 P 8a9da5f92d7c4186877987e0ddd62caf [term 0 FOLLOWER]: Advancing to term 1
I20260923 22:12:51.829835 8584 raft_consensus.cc:2454] T e9ed3afc39914c9bbb09a3a374df7670 P e01531ae61ec4025942988b1a6dd62f0 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4d0b914e7ab746e7b17789e7b32aca0a in term 1.
I20260923 22:12:51.829856 8721 raft_consensus.cc:2454] T e9ed3afc39914c9bbb09a3a374df7670 P 8a9da5f92d7c4186877987e0ddd62caf [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4d0b914e7ab746e7b17789e7b32aca0a in term 1.
I20260923 22:12:51.829998 8791 leader_election.cc:302] T e9ed3afc39914c9bbb09a3a374df7670 P 4d0b914e7ab746e7b17789e7b32aca0a [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4d0b914e7ab746e7b17789e7b32aca0a, 8a9da5f92d7c4186877987e0ddd62caf; no voters:
I20260923 22:12:51.830109 8996 raft_consensus.cc:2781] T e9ed3afc39914c9bbb09a3a374df7670 P 4d0b914e7ab746e7b17789e7b32aca0a [term 1 FOLLOWER]: Leader election won for term 1
I20260923 22:12:51.830257 8996 raft_consensus.cc:686] T e9ed3afc39914c9bbb09a3a374df7670 P 4d0b914e7ab746e7b17789e7b32aca0a [term 1 LEADER]: Becoming Leader. State: Replica: 4d0b914e7ab746e7b17789e7b32aca0a, State: Running, Role: LEADER
I20260923 22:12:51.830323 8996 consensus_queue.cc:237] T e9ed3afc39914c9bbb09a3a374df7670 P 4d0b914e7ab746e7b17789e7b32aca0a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4d0b914e7ab746e7b17789e7b32aca0a" member_type: VOTER last_known_addr { host: "127.1.135.195" port: 33563 } } peers { permanent_uuid: "e01531ae61ec4025942988b1a6dd62f0" member_type: VOTER last_known_addr { host: "127.1.135.193" port: 44597 } } peers { permanent_uuid: "8a9da5f92d7c4186877987e0ddd62caf" member_type: VOTER last_known_addr { host: "127.1.135.194" port: 41925 } }
I20260923 22:12:51.830626 8997 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 22:12:51.830544 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.135.254:35949 (local address 127.1.135.195:57867)
0923 22:12:51.830575 (+ 31us) negotiation.cc:107] Waiting for socket to connect
0923 22:12:51.830601 (+ 26us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.135.254:35949: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":15}
I20260923 22:12:51.831072 8264 catalog_manager.cc:5495] T e9ed3afc39914c9bbb09a3a374df7670 P 4d0b914e7ab746e7b17789e7b32aca0a reported cstate change: term changed from 0 to 1, leader changed from <none> to 4d0b914e7ab746e7b17789e7b32aca0a (127.1.135.195). New cstate: current_term: 1 leader_uuid: "4d0b914e7ab746e7b17789e7b32aca0a" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4d0b914e7ab746e7b17789e7b32aca0a" member_type: VOTER last_known_addr { host: "127.1.135.195" port: 33563 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "e01531ae61ec4025942988b1a6dd62f0" member_type: VOTER last_known_addr { host: "127.1.135.193" port: 44597 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "8a9da5f92d7c4186877987e0ddd62caf" member_type: VOTER last_known_addr { host: "127.1.135.194" port: 41925 } health_report { overall_health: UNKNOWN } } }
I20260923 22:12:51.839603 8986 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 22:12:51.837702 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57680 (local address 127.1.135.195:33563)
0923 22:12:51.837757 (+ 55us) server_negotiation.cc:206] Beginning negotiation
0923 22:12:51.837758 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0923 22:12:51.837767 (+ 9us) server_negotiation.cc:407] Connection header received
0923 22:12:51.837820 (+ 53us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 22:12:51.837821 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 22:12:51.837835 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 22:12:51.837890 (+ 55us) server_negotiation.cc:226] Negotiated authn=TOKEN
0923 22:12:51.837971 (+ 81us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 22:12:51.838387 (+ 416us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 22:12:51.839045 (+ 658us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 22:12:51.839210 (+ 165us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 22:12:51.839258 (+ 48us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 22:12:51.839375 (+ 117us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0923 22:12:51.839463 (+ 88us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0923 22:12:51.839514 (+ 51us) server_negotiation.cc:1035] Waiting for connection context
0923 22:12:51.839548 (+ 34us) server_negotiation.cc:299] Negotiation successful
0923 22:12:51.839562 (+ 14us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":37}
I20260923 22:12:51.840490 8721 raft_consensus.cc:1261] T e9ed3afc39914c9bbb09a3a374df7670 P 8a9da5f92d7c4186877987e0ddd62caf [term 1 FOLLOWER]: Refusing update from remote peer 4d0b914e7ab746e7b17789e7b32aca0a: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260923 22:12:51.840643 8584 raft_consensus.cc:1261] T e9ed3afc39914c9bbb09a3a374df7670 P e01531ae61ec4025942988b1a6dd62f0 [term 1 FOLLOWER]: Refusing update from remote peer 4d0b914e7ab746e7b17789e7b32aca0a: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260923 22:12:51.840938 8996 consensus_queue.cc:1035] T e9ed3afc39914c9bbb09a3a374df7670 P 4d0b914e7ab746e7b17789e7b32aca0a [LEADER]: Connected to new peer: Peer: permanent_uuid: "e01531ae61ec4025942988b1a6dd62f0" member_type: VOTER last_known_addr { host: "127.1.135.193" port: 44597 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260923 22:12:51.841051 9003 consensus_queue.cc:1035] T e9ed3afc39914c9bbb09a3a374df7670 P 4d0b914e7ab746e7b17789e7b32aca0a [LEADER]: Connected to new peer: Peer: permanent_uuid: "8a9da5f92d7c4186877987e0ddd62caf" member_type: VOTER last_known_addr { host: "127.1.135.194" port: 41925 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260923 22:12:51.841821 9001 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 22:12:51.841730 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.135.254:35949 (local address 127.1.135.194:39285)
0923 22:12:51.841761 (+ 31us) negotiation.cc:107] Waiting for socket to connect
0923 22:12:51.841790 (+ 29us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.135.254:35949: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":17}
I20260923 22:12:51.842224 8999 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 22:12:51.842095 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.135.254:35949 (local address 127.1.135.193:59063)
0923 22:12:51.842166 (+ 71us) negotiation.cc:107] Waiting for socket to connect
0923 22:12:51.842193 (+ 27us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.135.254:35949: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":55}
I20260923 22:12:51.842710 9007 mvcc.cc:205] Tried to move back new op lower bound from 7332665638256984064 to 7332665638217449472. Current Snapshot: MvccSnapshot[applied={T|T < 7332665638256984064}]
I20260923 22:12:51.843008 8997 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 22:12:51.842870 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.135.254:35949 (local address 127.1.135.195:59551)
0923 22:12:51.842958 (+ 88us) negotiation.cc:107] Waiting for socket to connect
0923 22:12:51.842985 (+ 27us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.135.254:35949: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":73}
I20260923 22:12:51.843400 9001 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 22:12:51.843270 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.135.254:35949 (local address 127.1.135.194:52871)
0923 22:12:51.843350 (+ 80us) negotiation.cc:107] Waiting for socket to connect
0923 22:12:51.843375 (+ 25us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.135.254:35949: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":66}
I20260923 22:12:51.843442 9006 mvcc.cc:205] Tried to move back new op lower bound from 7332665638256984064 to 7332665638217449472. Current Snapshot: MvccSnapshot[applied={T|T < 7332665638256984064 or (T in {7332665638256984064})}]
I20260923 22:12:51.844889 8997 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 22:12:51.844802 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.135.254:35949 (local address 127.1.135.195:54447)
0923 22:12:51.844835 (+ 33us) negotiation.cc:107] Waiting for socket to connect
0923 22:12:51.844861 (+ 26us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.135.254:35949: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":16}
I20260923 22:12:51.844980 9008 mvcc.cc:205] Tried to move back new op lower bound from 7332665638256984064 to 7332665638217449472. Current Snapshot: MvccSnapshot[applied={T|T < 7332665638256984064}]
I20260923 22:12:51.845194 8999 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 22:12:51.845095 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.135.254:35949 (local address 127.1.135.193:40939)
0923 22:12:51.845130 (+ 35us) negotiation.cc:107] Waiting for socket to connect
0923 22:12:51.845161 (+ 31us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.135.254:35949: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":21}
I20260923 22:12:51.845682 8264 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:53534:
table { table_name: "test-table" } modify_external_catalogs: true
I20260923 22:12:51.845770 8264 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:53534:
table { table_name: "test-table" } modify_external_catalogs: true
I20260923 22:12:51.847007 8264 catalog_manager.cc:5782] T 00000000000000000000000000000000 P b32070aba06045cabad174bee345066a: Sending DeleteTablet for 3 replicas of tablet e9ed3afc39914c9bbb09a3a374df7670
I20260923 22:12:51.847294 8837 tablet_service.cc:1514] Processing DeleteTablet for tablet e9ed3afc39914c9bbb09a3a374df7670 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-23 22:12:51 UTC) from {username='oryx', principal='oryx/127.1.135.253@KRBTEST.COM'} at 127.0.0.1:57678
I20260923 22:12:51.847314 8564 tablet_service.cc:1514] Processing DeleteTablet for tablet e9ed3afc39914c9bbb09a3a374df7670 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-23 22:12:51 UTC) from {username='oryx', principal='oryx/127.1.135.253@KRBTEST.COM'} at 127.0.0.1:45734
I20260923 22:12:51.847491 9016 tablet_replica.cc:330] T e9ed3afc39914c9bbb09a3a374df7670 P 4d0b914e7ab746e7b17789e7b32aca0a: stopping tablet replica
I20260923 22:12:51.847541 9016 raft_consensus.cc:2229] T e9ed3afc39914c9bbb09a3a374df7670 P 4d0b914e7ab746e7b17789e7b32aca0a [term 1 LEADER]: Raft consensus shutting down.
I20260923 22:12:51.847563 8701 tablet_service.cc:1514] Processing DeleteTablet for tablet e9ed3afc39914c9bbb09a3a374df7670 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-23 22:12:51 UTC) from {username='oryx', principal='oryx/127.1.135.253@KRBTEST.COM'} at 127.0.0.1:34800
I20260923 22:12:51.847612 9016 raft_consensus.cc:2258] T e9ed3afc39914c9bbb09a3a374df7670 P 4d0b914e7ab746e7b17789e7b32aca0a [term 1 FOLLOWER]: Raft consensus is shut down!
I20260923 22:12:51.847719 9017 tablet_replica.cc:330] T e9ed3afc39914c9bbb09a3a374df7670 P e01531ae61ec4025942988b1a6dd62f0: stopping tablet replica
I20260923 22:12:51.847764 9017 raft_consensus.cc:2229] T e9ed3afc39914c9bbb09a3a374df7670 P e01531ae61ec4025942988b1a6dd62f0 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260923 22:12:51.847798 9017 raft_consensus.cc:2258] T e9ed3afc39914c9bbb09a3a374df7670 P e01531ae61ec4025942988b1a6dd62f0 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260923 22:12:51.847890 9018 tablet_replica.cc:330] T e9ed3afc39914c9bbb09a3a374df7670 P 8a9da5f92d7c4186877987e0ddd62caf: stopping tablet replica
I20260923 22:12:51.847906 9016 ts_tablet_manager.cc:1905] T e9ed3afc39914c9bbb09a3a374df7670 P 4d0b914e7ab746e7b17789e7b32aca0a: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260923 22:12:51.847930 9018 raft_consensus.cc:2229] T e9ed3afc39914c9bbb09a3a374df7670 P 8a9da5f92d7c4186877987e0ddd62caf [term 1 FOLLOWER]: Raft consensus shutting down.
I20260923 22:12:51.847965 9018 raft_consensus.cc:2258] T e9ed3afc39914c9bbb09a3a374df7670 P 8a9da5f92d7c4186877987e0ddd62caf [term 1 FOLLOWER]: Raft consensus is shut down!
I20260923 22:12:51.847963 1567 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskRiQe_a/build/release/bin/kudu with pid 8412
I20260923 22:12:51.848654 9018 ts_tablet_manager.cc:1905] T e9ed3afc39914c9bbb09a3a374df7670 P 8a9da5f92d7c4186877987e0ddd62caf: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260923 22:12:51.850039 9016 ts_tablet_manager.cc:1918] T e9ed3afc39914c9bbb09a3a374df7670 P 4d0b914e7ab746e7b17789e7b32aca0a: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260923 22:12:51.850090 9016 log.cc:1196] T e9ed3afc39914c9bbb09a3a374df7670 P 4d0b914e7ab746e7b17789e7b32aca0a: Deleting WAL directory at /tmp/dist-test-taskRiQe_a/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790201539513534-1567-0/minicluster-data/ts-2/wal/wals/e9ed3afc39914c9bbb09a3a374df7670
I20260923 22:12:51.850092 9018 ts_tablet_manager.cc:1918] T e9ed3afc39914c9bbb09a3a374df7670 P 8a9da5f92d7c4186877987e0ddd62caf: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260923 22:12:51.850133 9018 log.cc:1196] T e9ed3afc39914c9bbb09a3a374df7670 P 8a9da5f92d7c4186877987e0ddd62caf: Deleting WAL directory at /tmp/dist-test-taskRiQe_a/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790201539513534-1567-0/minicluster-data/ts-1/wal/wals/e9ed3afc39914c9bbb09a3a374df7670
I20260923 22:12:51.850323 9016 ts_tablet_manager.cc:1939] T e9ed3afc39914c9bbb09a3a374df7670 P 4d0b914e7ab746e7b17789e7b32aca0a: Deleting consensus metadata
I20260923 22:12:51.850380 9018 ts_tablet_manager.cc:1939] T e9ed3afc39914c9bbb09a3a374df7670 P 8a9da5f92d7c4186877987e0ddd62caf: Deleting consensus metadata
I20260923 22:12:51.850672 8250 catalog_manager.cc:4841] TS 8a9da5f92d7c4186877987e0ddd62caf (127.1.135.194:41925): tablet e9ed3afc39914c9bbb09a3a374df7670 (table test-table [id=d8fabe81fa1a42c28b8b0ad6258d7c11]) successfully deleted
I20260923 22:12:51.851745 8252 catalog_manager.cc:4841] TS 4d0b914e7ab746e7b17789e7b32aca0a (127.1.135.195:33563): tablet e9ed3afc39914c9bbb09a3a374df7670 (table test-table [id=d8fabe81fa1a42c28b8b0ad6258d7c11]) successfully deleted
I20260923 22:12:51.857435 8987 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 22:12:51.857328 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.135.193:44597 (local address 127.0.0.1:45746)
0923 22:12:51.857382 (+ 54us) negotiation.cc:107] Waiting for socket to connect
0923 22:12:51.857411 (+ 29us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.135.193:44597: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":37}
I20260923 22:12:51.857511 1567 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskRiQe_a/build/release/bin/kudu with pid 8638
W20260923 22:12:51.857578 8250 catalog_manager.cc:4584] TS e01531ae61ec4025942988b1a6dd62f0 (127.1.135.193:44597): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet e9ed3afc39914c9bbb09a3a374df7670: Network error: Client connection negotiation failed: client connection to 127.1.135.193:44597: connect: Connection refused (error 111)
I20260923 22:12:51.862383 1567 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskRiQe_a/build/release/bin/kudu with pid 8774
I20260923 22:12:51.867206 1567 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskRiQe_a/build/release/bin/kudu with pid 8131
I20260923 22:12:51.871675 1567 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskRiQe_a/build/release/bin/kudu with pid 8319
2026-09-23T22:12:51Z chronyd exiting
[ OK ] SecurityITest.TestNonDefaultPrincipalMultipleMaster (4576 ms)
[ RUN ] SecurityITest.TestNonDefaultPrincipal
Loading random data
Initializing database '/tmp/dist-test-taskRiQe_a/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790201539513534-1567-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Sep 23 22:12:51 dist-test-slave-s2fj krb5kdc[9021](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Sep 23 22:12:51 dist-test-slave-s2fj krb5kdc[9021](info): set up 2 sockets
Sep 23 22:12:51 dist-test-slave-s2fj krb5kdc[9021](info): commencing operation
krb5kdc: starting...
W20260923 22:12:52.900632 1567 mini_kdc.cc:121] Time spent starting KDC: real 1.008s user 0.004s sys 0.001s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskRiQe_a/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790201539513534-1567-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskRiQe_a/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790201539513534-1567-0/krb5kdc/test-user.keytab.
Sep 23 22:12:52 dist-test-slave-s2fj krb5kdc[9021](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790201572, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-09-23T22:12:52Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-09-23T22:12:52Z Disabled control of system clock
WARNING: no policy specified for oryx/127.1.135.254@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.1.135.254@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.1.135.254 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskRiQe_a/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790201539513534-1567-0/krb5kdc/oryx_127.1.135.254.keytab.
Entry for principal oryx/127.1.135.254 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskRiQe_a/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790201539513534-1567-0/krb5kdc/oryx_127.1.135.254.keytab.
I20260923 22:12:53.016979 1567 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskRiQe_a/build/release/bin/kudu
/tmp/dist-test-taskRiQe_a/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskRiQe_a/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790201539513534-1567-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskRiQe_a/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790201539513534-1567-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskRiQe_a/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790201539513534-1567-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskRiQe_a/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790201539513534-1567-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.1.135.254:35183
--webserver_interface=127.1.135.254
--webserver_port=0
--keytab_file=/tmp/dist-test-taskRiQe_a/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790201539513534-1567-0/krb5kdc/oryx_127.1.135.254.keytab
--principal=oryx/127.1.135.254
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.1.135.212:42511
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.1.135.254:35183
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--txn_manager_enabled=true with env {KRB5CCNAME=/tmp/dist-test-taskRiQe_a/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790201539513534-1567-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskRiQe_a/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790201539513534-1567-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskRiQe_a/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790201539513534-1567-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260923 22:12:53.096499 9034 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260923 22:12:53.096756 9034 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260923 22:12:53.096791 9034 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260923 22:12:53.097930 9034 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260923 22:12:53.097997 9034 flags.cc:425] Enabled experimental flag: --txn_manager_enabled=true
W20260923 22:12:53.098019 9034 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260923 22:12:53.098042 9034 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260923 22:12:53.098069 9034 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260923 22:12:53.098101 9034 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260923 22:12:53.099599 9034 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.1.135.212:42511
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskRiQe_a/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790201539513534-1567-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskRiQe_a/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790201539513534-1567-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.1.135.254:35183
--txn_manager_enabled=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskRiQe_a/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790201539513534-1567-0/krb5kdc/oryx_127.1.135.254.keytab
--principal=oryx/127.1.135.254
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.1.135.254:35183
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskRiQe_a/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790201539513534-1567-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.1.135.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.9034
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskRiQe_a/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790201539513534-1567-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 23 Sep 2026 20:53:37 UTC on ccycloud.quasar-rkviyx.root.comops.site
build id 4668083
I20260923 22:12:53.099880 9034 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260923 22:12:53.100143 9034 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260923 22:12:53.104802 9042 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260923 22:12:53.105043 9040 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260923 22:12:53.105075 9039 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260923 22:12:53.105299 9034 server_base.cc:1034] running on GCE node
I20260923 22:12:53.105577 9034 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260923 22:12:53.105866 9034 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260923 22:12:53.107035 9034 hybrid_clock.cc:648] HybridClock initialized: now 1790201573107017 us; error 32 us; skew 500 ppm
Sep 23 22:12:53 dist-test-slave-s2fj krb5kdc[9021](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790201573, etypes {rep=17 tkt=17 ses=17}, oryx/127.1.135.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260923 22:12:53.109501 9034 init.cc:377] Logged in from keytab as oryx/127.1.135.254@KRBTEST.COM (short username oryx)
I20260923 22:12:53.109932 9034 webserver.cc:460] Webserver started at http://127.1.135.254:38065/ using document root <none> and password file <none>
I20260923 22:12:53.110160 9034 fs_manager.cc:359] Metadata directory not provided
I20260923 22:12:53.110229 9034 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260923 22:12:53.110378 9034 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260923 22:12:53.111451 9034 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskRiQe_a/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790201539513534-1567-0/minicluster-data/master-0/data/instance:
uuid: "53d3bf4260ad47b69a41891f4fa3a6e5"
format_stamp: "Formatted at 2026-09-23 22:12:53 on dist-test-slave-s2fj"
server_key: "169bafdb47f5858a4241d3def4852dde"
server_key_iv: "d28b5d2c2d76c63c5bfebf027e7e6c8f"
server_key_version: "encryptionkey@0"
I20260923 22:12:53.111819 9034 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskRiQe_a/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790201539513534-1567-0/minicluster-data/master-0/wal/instance:
uuid: "53d3bf4260ad47b69a41891f4fa3a6e5"
format_stamp: "Formatted at 2026-09-23 22:12:53 on dist-test-slave-s2fj"
server_key: "169bafdb47f5858a4241d3def4852dde"
server_key_iv: "d28b5d2c2d76c63c5bfebf027e7e6c8f"
server_key_version: "encryptionkey@0"
I20260923 22:12:53.113197 9034 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.003s
I20260923 22:12:53.113886 9049 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260923 22:12:53.114061 9034 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.001s
I20260923 22:12:53.114158 9034 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskRiQe_a/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790201539513534-1567-0/minicluster-data/master-0/data,/tmp/dist-test-taskRiQe_a/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790201539513534-1567-0/minicluster-data/master-0/wal
uuid: "53d3bf4260ad47b69a41891f4fa3a6e5"
format_stamp: "Formatted at 2026-09-23 22:12:53 on dist-test-slave-s2fj"
server_key: "169bafdb47f5858a4241d3def4852dde"
server_key_iv: "d28b5d2c2d76c63c5bfebf027e7e6c8f"
server_key_version: "encryptionkey@0"
I20260923 22:12:53.114244 9034 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskRiQe_a/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790201539513534-1567-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskRiQe_a/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790201539513534-1567-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskRiQe_a/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790201539513534-1567-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260923 22:12:53.124354 9034 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260923 22:12:53.126132 9034 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260923 22:12:53.126295 9034 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260923 22:12:53.131019 9034 rpc_server.cc:307] RPC server started. Bound to: 127.1.135.254:35183
I20260923 22:12:53.131088 9111 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.1.135.254:35183 every 8 connection(s)
I20260923 22:12:53.131440 9034 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskRiQe_a/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790201539513534-1567-0/minicluster-data/master-0/data/info.pb
I20260923 22:12:53.131742 1567 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskRiQe_a/build/release/bin/kudu as pid 9034
I20260923 22:12:53.131847 1567 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskRiQe_a/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790201539513534-1567-0/minicluster-data/master-0/wal/instance
I20260923 22:12:53.132102 1567 external_mini_cluster.cc:1428] Setting key 3cb185f16ddfafa0686bf9f4deaf07f4
I20260923 22:12:53.132246 9112 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260923 22:12:53.135460 9112 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 53d3bf4260ad47b69a41891f4fa3a6e5: Bootstrap starting.
I20260923 22:12:53.136206 9112 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 53d3bf4260ad47b69a41891f4fa3a6e5: Neither blocks nor log segments found. Creating new log.
I20260923 22:12:53.136512 9112 log.cc:824] T 00000000000000000000000000000000 P 53d3bf4260ad47b69a41891f4fa3a6e5: Log is configured to *not* fsync() on all Append() calls
I20260923 22:12:53.137174 9112 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 53d3bf4260ad47b69a41891f4fa3a6e5: No bootstrap required, opened a new log
Sep 23 22:12:53 dist-test-slave-s2fj krb5kdc[9021](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790201572, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for oryx/127.1.135.254@KRBTEST.COM
I20260923 22:12:53.139305 9112 raft_consensus.cc:348] T 00000000000000000000000000000000 P 53d3bf4260ad47b69a41891f4fa3a6e5 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "53d3bf4260ad47b69a41891f4fa3a6e5" member_type: VOTER last_known_addr { host: "127.1.135.254" port: 35183 } }
I20260923 22:12:53.139460 9112 raft_consensus.cc:374] T 00000000000000000000000000000000 P 53d3bf4260ad47b69a41891f4fa3a6e5 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260923 22:12:53.139508 9112 raft_consensus.cc:729] T 00000000000000000000000000000000 P 53d3bf4260ad47b69a41891f4fa3a6e5 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 53d3bf4260ad47b69a41891f4fa3a6e5, State: Initialized, Role: FOLLOWER
I20260923 22:12:53.139678 9112 consensus_queue.cc:260] T 00000000000000000000000000000000 P 53d3bf4260ad47b69a41891f4fa3a6e5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "53d3bf4260ad47b69a41891f4fa3a6e5" member_type: VOTER last_known_addr { host: "127.1.135.254" port: 35183 } }
I20260923 22:12:53.139775 9112 raft_consensus.cc:388] T 00000000000000000000000000000000 P 53d3bf4260ad47b69a41891f4fa3a6e5 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260923 22:12:53.139812 9112 raft_consensus.cc:482] T 00000000000000000000000000000000 P 53d3bf4260ad47b69a41891f4fa3a6e5 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260923 22:12:53.139864 9112 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 53d3bf4260ad47b69a41891f4fa3a6e5 [term 0 FOLLOWER]: Advancing to term 1
I20260923 22:12:53.140542 9112 raft_consensus.cc:504] T 00000000000000000000000000000000 P 53d3bf4260ad47b69a41891f4fa3a6e5 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "53d3bf4260ad47b69a41891f4fa3a6e5" member_type: VOTER last_known_addr { host: "127.1.135.254" port: 35183 } }
I20260923 22:12:53.140678 9112 leader_election.cc:302] T 00000000000000000000000000000000 P 53d3bf4260ad47b69a41891f4fa3a6e5 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 53d3bf4260ad47b69a41891f4fa3a6e5; no voters:
I20260923 22:12:53.140875 9112 leader_election.cc:288] T 00000000000000000000000000000000 P 53d3bf4260ad47b69a41891f4fa3a6e5 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260923 22:12:53.140906 9117 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 53d3bf4260ad47b69a41891f4fa3a6e5 [term 1 FOLLOWER]: Leader election won for term 1
I20260923 22:12:53.141013 9117 raft_consensus.cc:686] T 00000000000000000000000000000000 P 53d3bf4260ad47b69a41891f4fa3a6e5 [term 1 LEADER]: Becoming Leader. State: Replica: 53d3bf4260ad47b69a41891f4fa3a6e5, State: Running, Role: LEADER
I20260923 22:12:53.141106 9117 consensus_queue.cc:237] T 00000000000000000000000000000000 P 53d3bf4260ad47b69a41891f4fa3a6e5 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "53d3bf4260ad47b69a41891f4fa3a6e5" member_type: VOTER last_known_addr { host: "127.1.135.254" port: 35183 } }
I20260923 22:12:53.141134 9112 sys_catalog.cc:564] T 00000000000000000000000000000000 P 53d3bf4260ad47b69a41891f4fa3a6e5 [sys.catalog]: configured and running, proceeding with master startup.
I20260923 22:12:53.141402 9117 sys_catalog.cc:455] T 00000000000000000000000000000000 P 53d3bf4260ad47b69a41891f4fa3a6e5 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "53d3bf4260ad47b69a41891f4fa3a6e5" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "53d3bf4260ad47b69a41891f4fa3a6e5" member_type: VOTER last_known_addr { host: "127.1.135.254" port: 35183 } } }
I20260923 22:12:53.141471 9117 sys_catalog.cc:458] T 00000000000000000000000000000000 P 53d3bf4260ad47b69a41891f4fa3a6e5 [sys.catalog]: This master's current role is: LEADER
I20260923 22:12:53.141592 9117 sys_catalog.cc:455] T 00000000000000000000000000000000 P 53d3bf4260ad47b69a41891f4fa3a6e5 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 53d3bf4260ad47b69a41891f4fa3a6e5. Latest consensus state: current_term: 1 leader_uuid: "53d3bf4260ad47b69a41891f4fa3a6e5" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "53d3bf4260ad47b69a41891f4fa3a6e5" member_type: VOTER last_known_addr { host: "127.1.135.254" port: 35183 } } }
I20260923 22:12:53.141655 9117 sys_catalog.cc:458] T 00000000000000000000000000000000 P 53d3bf4260ad47b69a41891f4fa3a6e5 [sys.catalog]: This master's current role is: LEADER
I20260923 22:12:53.142403 9122 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260923 22:12:53.143006 9122 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260923 22:12:53.143488 9115 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 22:12:53.132641 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48886 (local address 127.1.135.254:35183)
0923 22:12:53.132839 (+ 198us) server_negotiation.cc:206] Beginning negotiation
0923 22:12:53.132848 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0923 22:12:53.132880 (+ 32us) server_negotiation.cc:407] Connection header received
0923 22:12:53.133323 (+ 443us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 22:12:53.133330 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 22:12:53.133376 (+ 46us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 22:12:53.133488 (+ 112us) server_negotiation.cc:226] Negotiated authn=SASL
0923 22:12:53.134277 (+ 789us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 22:12:53.135138 (+ 861us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 22:12:53.136213 (+ 1075us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 22:12:53.136719 (+ 506us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 22:12:53.136850 (+ 131us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 22:12:53.139371 (+ 2521us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0923 22:12:53.139375 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0923 22:12:53.139375 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0923 22:12:53.139387 (+ 12us) server_negotiation.cc:922] calling sasl_server_start()
0923 22:12:53.141448 (+ 2061us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0923 22:12:53.141875 (+ 427us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0923 22:12:53.141878 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0923 22:12:53.141878 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0923 22:12:53.141979 (+ 101us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0923 22:12:53.142235 (+ 256us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0923 22:12:53.142236 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0923 22:12:53.142236 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0923 22:12:53.142560 (+ 324us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0923 22:12:53.142699 (+ 139us) server_negotiation.cc:1035] Waiting for connection context
0923 22:12:53.143166 (+ 467us) server_negotiation.cc:299] Negotiation successful
0923 22:12:53.143301 (+ 135us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":144,"thread_start_us":106,"threads_started":1}
I20260923 22:12:53.144801 9122 catalog_manager.cc:1347] Generated new cluster ID: ee0f7b8d8f074b66b9cf59ae5fd8a737
I20260923 22:12:53.144852 9122 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260923 22:12:53.158036 9122 catalog_manager.cc:1370] Generated new certificate authority record
I20260923 22:12:53.158573 9122 catalog_manager.cc:1504] Loading token signing keys...
I20260923 22:12:53.162701 9122 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 53d3bf4260ad47b69a41891f4fa3a6e5: Generated new TSK 0
I20260923 22:12:53.162878 9122 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for oryx/127.1.135.193@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.1.135.193@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.1.135.193 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskRiQe_a/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790201539513534-1567-0/krb5kdc/oryx_127.1.135.193.keytab.
Entry for principal oryx/127.1.135.193 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskRiQe_a/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790201539513534-1567-0/krb5kdc/oryx_127.1.135.193.keytab.
I20260923 22:12:53.196249 1567 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskRiQe_a/build/release/bin/kudu
/tmp/dist-test-taskRiQe_a/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskRiQe_a/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790201539513534-1567-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskRiQe_a/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790201539513534-1567-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskRiQe_a/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790201539513534-1567-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskRiQe_a/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790201539513534-1567-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.1.135.193:0
--local_ip_for_outbound_sockets=127.1.135.193
--webserver_interface=127.1.135.193
--webserver_port=0
--tserver_master_addrs=127.1.135.254:35183
--keytab_file=/tmp/dist-test-taskRiQe_a/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790201539513534-1567-0/krb5kdc/oryx_127.1.135.193.keytab
--principal=oryx/127.1.135.193
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.1.135.212:42511
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--enable_txn_system_client_init=true with env {KRB5CCNAME=/tmp/dist-test-taskRiQe_a/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790201539513534-1567-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskRiQe_a/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790201539513534-1567-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskRiQe_a/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790201539513534-1567-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260923 22:12:53.276428 9138 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260923 22:12:53.276633 9138 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260923 22:12:53.276659 9138 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260923 22:12:53.277752 9138 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260923 22:12:53.277783 9138 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260923 22:12:53.277804 9138 flags.cc:425] Enabled experimental flag: --enable_txn_system_client_init=true
W20260923 22:12:53.277822 9138 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.1.135.193
I20260923 22:12:53.279251 9138 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.1.135.212:42511
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskRiQe_a/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790201539513534-1567-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskRiQe_a/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790201539513534-1567-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskRiQe_a/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790201539513534-1567-0/krb5kdc/oryx_127.1.135.193.keytab
--principal=oryx/127.1.135.193
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.1.135.193:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskRiQe_a/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790201539513534-1567-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.1.135.193
--webserver_port=0
--enable_txn_system_client_init=true
--tserver_master_addrs=127.1.135.254:35183
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.9138
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.1.135.193
--log_dir=/tmp/dist-test-taskRiQe_a/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790201539513534-1567-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 23 Sep 2026 20:53:37 UTC on ccycloud.quasar-rkviyx.root.comops.site
build id 4668083
I20260923 22:12:53.279438 9138 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260923 22:12:53.279608 9138 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260923 22:12:53.284405 9146 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260923 22:12:53.284418 9144 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260923 22:12:53.284425 9143 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260923 22:12:53.284624 9138 server_base.cc:1034] running on GCE node
I20260923 22:12:53.284840 9138 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260923 22:12:53.285039 9138 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260923 22:12:53.286186 9138 hybrid_clock.cc:648] HybridClock initialized: now 1790201573286174 us; error 27 us; skew 500 ppm
Sep 23 22:12:53 dist-test-slave-s2fj krb5kdc[9021](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790201573, etypes {rep=17 tkt=17 ses=17}, oryx/127.1.135.193@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260923 22:12:53.288769 9138 init.cc:377] Logged in from keytab as oryx/127.1.135.193@KRBTEST.COM (short username oryx)
I20260923 22:12:53.289203 9138 webserver.cc:460] Webserver started at http://127.1.135.193:34405/ using document root <none> and password file <none>
I20260923 22:12:53.289433 9138 fs_manager.cc:359] Metadata directory not provided
I20260923 22:12:53.289476 9138 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260923 22:12:53.289642 9138 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260923 22:12:53.290705 9138 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskRiQe_a/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790201539513534-1567-0/minicluster-data/ts-0/data/instance:
uuid: "07273d7511d54b88b18936d1c8590b03"
format_stamp: "Formatted at 2026-09-23 22:12:53 on dist-test-slave-s2fj"
server_key: "b4e574fa098ac2f25911f5282c1aa788"
server_key_iv: "12bf88224ff0deb6219d1da6d85bae8c"
server_key_version: "encryptionkey@0"
I20260923 22:12:53.291059 9138 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskRiQe_a/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790201539513534-1567-0/minicluster-data/ts-0/wal/instance:
uuid: "07273d7511d54b88b18936d1c8590b03"
format_stamp: "Formatted at 2026-09-23 22:12:53 on dist-test-slave-s2fj"
server_key: "b4e574fa098ac2f25911f5282c1aa788"
server_key_iv: "12bf88224ff0deb6219d1da6d85bae8c"
server_key_version: "encryptionkey@0"
I20260923 22:12:53.292383 9138 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.003s
I20260923 22:12:53.293124 9153 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260923 22:12:53.293300 9138 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.001s
I20260923 22:12:53.293390 9138 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskRiQe_a/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790201539513534-1567-0/minicluster-data/ts-0/data,/tmp/dist-test-taskRiQe_a/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790201539513534-1567-0/minicluster-data/ts-0/wal
uuid: "07273d7511d54b88b18936d1c8590b03"
format_stamp: "Formatted at 2026-09-23 22:12:53 on dist-test-slave-s2fj"
server_key: "b4e574fa098ac2f25911f5282c1aa788"
server_key_iv: "12bf88224ff0deb6219d1da6d85bae8c"
server_key_version: "encryptionkey@0"
I20260923 22:12:53.293462 9138 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskRiQe_a/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790201539513534-1567-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskRiQe_a/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790201539513534-1567-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskRiQe_a/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790201539513534-1567-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260923 22:12:53.313392 9138 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260923 22:12:53.315801 9138 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260923 22:12:53.315965 9138 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260923 22:12:53.316520 9138 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260923 22:12:53.316576 9138 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260923 22:12:53.316603 9138 ts_tablet_manager.cc:610] Registered 0 tablets
I20260923 22:12:53.316720 9138 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260923 22:12:53.324214 9138 rpc_server.cc:307] RPC server started. Bound to: 127.1.135.193:33215
I20260923 22:12:53.324302 9268 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.1.135.193:33215 every 8 connection(s)
Sep 23 22:12:53 dist-test-slave-s2fj krb5kdc[9021](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790201573, etypes {rep=17 tkt=17 ses=17}, oryx/127.1.135.193@KRBTEST.COM for oryx/127.1.135.254@KRBTEST.COM
I20260923 22:12:53.324641 9138 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskRiQe_a/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790201539513534-1567-0/minicluster-data/ts-0/data/info.pb
I20260923 22:12:53.327548 9115 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 22:12:53.317115 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.135.193:52415 (local address 127.1.135.254:35183)
0923 22:12:53.317199 (+ 84us) server_negotiation.cc:206] Beginning negotiation
0923 22:12:53.317201 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0923 22:12:53.317474 (+ 273us) server_negotiation.cc:407] Connection header received
0923 22:12:53.318320 (+ 846us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 22:12:53.318321 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 22:12:53.318335 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 22:12:53.318412 (+ 77us) server_negotiation.cc:226] Negotiated authn=SASL
0923 22:12:53.319286 (+ 874us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 22:12:53.320082 (+ 796us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 22:12:53.321768 (+ 1686us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 22:12:53.322107 (+ 339us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 22:12:53.322210 (+ 103us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 22:12:53.325051 (+ 2841us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0923 22:12:53.325052 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0923 22:12:53.325053 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0923 22:12:53.325061 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0923 22:12:53.326484 (+ 1423us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0923 22:12:53.326831 (+ 347us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0923 22:12:53.326832 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0923 22:12:53.326832 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0923 22:12:53.326881 (+ 49us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0923 22:12:53.327032 (+ 151us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0923 22:12:53.327033 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0923 22:12:53.327033 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0923 22:12:53.327123 (+ 90us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0923 22:12:53.327167 (+ 44us) server_negotiation.cc:1035] Waiting for connection context
0923 22:12:53.327414 (+ 247us) server_negotiation.cc:299] Negotiation successful
0923 22:12:53.327480 (+ 66us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":61}
I20260923 22:12:53.327696 9162 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 22:12:53.317104 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.135.254:35183 (local address 127.1.135.193:52415)
0923 22:12:53.317293 (+ 189us) negotiation.cc:107] Waiting for socket to connect
0923 22:12:53.317362 (+ 69us) client_negotiation.cc:174] Beginning negotiation
0923 22:12:53.318181 (+ 819us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0923 22:12:53.318498 (+ 317us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0923 22:12:53.318502 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0923 22:12:53.318864 (+ 362us) client_negotiation.cc:189] Negotiated authn=SASL
0923 22:12:53.319165 (+ 301us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 22:12:53.319172 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 22:12:53.320195 (+ 1023us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 22:12:53.320196 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 22:12:53.321665 (+ 1469us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 22:12:53.321668 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 22:12:53.322223 (+ 555us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 22:12:53.322224 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 22:12:53.322384 (+ 160us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 22:12:53.322984 (+ 600us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0923 22:12:53.322996 (+ 12us) client_negotiation.cc:656] calling sasl_client_start()
0923 22:12:53.324925 (+ 1929us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0923 22:12:53.326558 (+ 1633us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0923 22:12:53.326562 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0923 22:12:53.326563 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0923 22:12:53.326765 (+ 202us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0923 22:12:53.326921 (+ 156us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0923 22:12:53.326921 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0923 22:12:53.326921 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0923 22:12:53.326998 (+ 77us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0923 22:12:53.327172 (+ 174us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0923 22:12:53.327176 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0923 22:12:53.327304 (+ 128us) client_negotiation.cc:769] Sending connection context
0923 22:12:53.327387 (+ 83us) client_negotiation.cc:240] Negotiation successful
0923 22:12:53.327503 (+ 116us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":152,"thread_start_us":75,"threads_started":1}
I20260923 22:12:53.328249 9269 heartbeater.cc:347] Connected to a master server at 127.1.135.254:35183
I20260923 22:12:53.328356 9269 heartbeater.cc:464] Registering TS with master...
I20260923 22:12:53.328531 9269 heartbeater.cc:511] Master 127.1.135.254:35183 requested a full tablet report, sending...
I20260923 22:12:53.329094 9065 ts_manager.cc:194] Registered new tserver with Master: 07273d7511d54b88b18936d1c8590b03 (127.1.135.193:33215)
I20260923 22:12:53.329968 9065 master_service.cc:496] Signed X509 certificate for tserver {username='oryx', principal='oryx/127.1.135.193@KRBTEST.COM'} at 127.1.135.193:52415
I20260923 22:12:53.331400 1567 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskRiQe_a/build/release/bin/kudu as pid 9138
I20260923 22:12:53.331526 1567 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskRiQe_a/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790201539513534-1567-0/minicluster-data/ts-0/wal/instance
I20260923 22:12:53.331758 1567 external_mini_cluster.cc:1428] Setting key 9ecf5ed023a0e8d8733bdf0206308da2
I20260923 22:12:53.335361 9115 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 22:12:53.329398 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.135.193:40993 (local address 127.1.135.254:35183)
0923 22:12:53.329439 (+ 41us) server_negotiation.cc:206] Beginning negotiation
0923 22:12:53.329441 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0923 22:12:53.329507 (+ 66us) server_negotiation.cc:407] Connection header received
0923 22:12:53.329562 (+ 55us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 22:12:53.329563 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 22:12:53.329572 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 22:12:53.329637 (+ 65us) server_negotiation.cc:226] Negotiated authn=SASL
0923 22:12:53.329900 (+ 263us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 22:12:53.330554 (+ 654us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 22:12:53.331340 (+ 786us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 22:12:53.331569 (+ 229us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 22:12:53.331628 (+ 59us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 22:12:53.332661 (+ 1033us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0923 22:12:53.332662 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0923 22:12:53.332662 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0923 22:12:53.332667 (+ 5us) server_negotiation.cc:922] calling sasl_server_start()
0923 22:12:53.334249 (+ 1582us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0923 22:12:53.334685 (+ 436us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0923 22:12:53.334686 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0923 22:12:53.334686 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0923 22:12:53.334735 (+ 49us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0923 22:12:53.334936 (+ 201us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0923 22:12:53.334936 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0923 22:12:53.334937 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0923 22:12:53.335026 (+ 89us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0923 22:12:53.335083 (+ 57us) server_negotiation.cc:1035] Waiting for connection context
0923 22:12:53.335241 (+ 158us) server_negotiation.cc:299] Negotiation successful
0923 22:12:53.335300 (+ 59us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":7,"server-negotiator.queue_time_us":22}
I20260923 22:12:53.335464 9275 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 22:12:53.329334 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.135.254:35183 (local address 127.1.135.193:40993)
0923 22:12:53.329447 (+ 113us) negotiation.cc:107] Waiting for socket to connect
0923 22:12:53.329458 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0923 22:12:53.329520 (+ 62us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0923 22:12:53.329638 (+ 118us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0923 22:12:53.329639 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0923 22:12:53.329789 (+ 150us) client_negotiation.cc:189] Negotiated authn=SASL
0923 22:12:53.329839 (+ 50us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 22:12:53.329840 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 22:12:53.330652 (+ 812us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 22:12:53.330653 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 22:12:53.331257 (+ 604us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 22:12:53.331260 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 22:12:53.331645 (+ 385us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 22:12:53.331646 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 22:12:53.331723 (+ 77us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 22:12:53.332207 (+ 484us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0923 22:12:53.332212 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0923 22:12:53.332589 (+ 377us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0923 22:12:53.334360 (+ 1771us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0923 22:12:53.334362 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0923 22:12:53.334362 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0923 22:12:53.334612 (+ 250us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0923 22:12:53.334804 (+ 192us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0923 22:12:53.334805 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0923 22:12:53.334806 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0923 22:12:53.334876 (+ 70us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0923 22:12:53.335094 (+ 218us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0923 22:12:53.335095 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0923 22:12:53.335146 (+ 51us) client_negotiation.cc:769] Sending connection context
0923 22:12:53.335219 (+ 73us) client_negotiation.cc:240] Negotiation successful
0923 22:12:53.335333 (+ 114us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":90,"thread_start_us":61,"threads_started":1}
WARNING: no policy specified for oryx/127.1.135.194@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.1.135.194@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.1.135.194 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskRiQe_a/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790201539513534-1567-0/krb5kdc/oryx_127.1.135.194.keytab.
Entry for principal oryx/127.1.135.194 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskRiQe_a/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790201539513534-1567-0/krb5kdc/oryx_127.1.135.194.keytab.
I20260923 22:12:53.358611 1567 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskRiQe_a/build/release/bin/kudu
/tmp/dist-test-taskRiQe_a/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskRiQe_a/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790201539513534-1567-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskRiQe_a/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790201539513534-1567-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskRiQe_a/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790201539513534-1567-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskRiQe_a/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790201539513534-1567-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.1.135.194:0
--local_ip_for_outbound_sockets=127.1.135.194
--webserver_interface=127.1.135.194
--webserver_port=0
--tserver_master_addrs=127.1.135.254:35183
--keytab_file=/tmp/dist-test-taskRiQe_a/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790201539513534-1567-0/krb5kdc/oryx_127.1.135.194.keytab
--principal=oryx/127.1.135.194
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.1.135.212:42511
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--enable_txn_system_client_init=true with env {KRB5CCNAME=/tmp/dist-test-taskRiQe_a/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790201539513534-1567-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskRiQe_a/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790201539513534-1567-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskRiQe_a/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790201539513534-1567-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260923 22:12:53.438788 9278 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260923 22:12:53.438998 9278 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260923 22:12:53.439023 9278 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260923 22:12:53.440119 9278 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260923 22:12:53.440152 9278 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260923 22:12:53.440172 9278 flags.cc:425] Enabled experimental flag: --enable_txn_system_client_init=true
W20260923 22:12:53.440189 9278 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.1.135.194
I20260923 22:12:53.441717 9278 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.1.135.212:42511
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskRiQe_a/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790201539513534-1567-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskRiQe_a/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790201539513534-1567-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskRiQe_a/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790201539513534-1567-0/krb5kdc/oryx_127.1.135.194.keytab
--principal=oryx/127.1.135.194
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.1.135.194:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskRiQe_a/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790201539513534-1567-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.1.135.194
--webserver_port=0
--enable_txn_system_client_init=true
--tserver_master_addrs=127.1.135.254:35183
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.9278
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.1.135.194
--log_dir=/tmp/dist-test-taskRiQe_a/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790201539513534-1567-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 23 Sep 2026 20:53:37 UTC on ccycloud.quasar-rkviyx.root.comops.site
build id 4668083
I20260923 22:12:53.442073 9278 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260923 22:12:53.442313 9278 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260923 22:12:53.447211 9284 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260923 22:12:53.447233 9283 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260923 22:12:53.447233 9286 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260923 22:12:53.447571 9278 server_base.cc:1034] running on GCE node
I20260923 22:12:53.447796 9278 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260923 22:12:53.448031 9278 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260923 22:12:53.449213 9278 hybrid_clock.cc:648] HybridClock initialized: now 1790201573449203 us; error 27 us; skew 500 ppm
Sep 23 22:12:53 dist-test-slave-s2fj krb5kdc[9021](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790201573, etypes {rep=17 tkt=17 ses=17}, oryx/127.1.135.194@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260923 22:12:53.451599 9278 init.cc:377] Logged in from keytab as oryx/127.1.135.194@KRBTEST.COM (short username oryx)
I20260923 22:12:53.452107 9278 webserver.cc:460] Webserver started at http://127.1.135.194:41429/ using document root <none> and password file <none>
I20260923 22:12:53.452358 9278 fs_manager.cc:359] Metadata directory not provided
I20260923 22:12:53.452423 9278 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260923 22:12:53.452565 9278 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260923 22:12:53.453580 9278 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskRiQe_a/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790201539513534-1567-0/minicluster-data/ts-1/data/instance:
uuid: "7a422c36ebff4c2384978dbc5c2b9afc"
format_stamp: "Formatted at 2026-09-23 22:12:53 on dist-test-slave-s2fj"
server_key: "1605a85b6e15cdec8abb7c6d8b7f885a"
server_key_iv: "17628a7ab9176e082553f9012dd68b24"
server_key_version: "encryptionkey@0"
I20260923 22:12:53.453943 9278 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskRiQe_a/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790201539513534-1567-0/minicluster-data/ts-1/wal/instance:
uuid: "7a422c36ebff4c2384978dbc5c2b9afc"
format_stamp: "Formatted at 2026-09-23 22:12:53 on dist-test-slave-s2fj"
server_key: "1605a85b6e15cdec8abb7c6d8b7f885a"
server_key_iv: "17628a7ab9176e082553f9012dd68b24"
server_key_version: "encryptionkey@0"
I20260923 22:12:53.455309 9278 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.001s
I20260923 22:12:53.455983 9293 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260923 22:12:53.456174 9278 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.000s
I20260923 22:12:53.456271 9278 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskRiQe_a/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790201539513534-1567-0/minicluster-data/ts-1/data,/tmp/dist-test-taskRiQe_a/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790201539513534-1567-0/minicluster-data/ts-1/wal
uuid: "7a422c36ebff4c2384978dbc5c2b9afc"
format_stamp: "Formatted at 2026-09-23 22:12:53 on dist-test-slave-s2fj"
server_key: "1605a85b6e15cdec8abb7c6d8b7f885a"
server_key_iv: "17628a7ab9176e082553f9012dd68b24"
server_key_version: "encryptionkey@0"
I20260923 22:12:53.456352 9278 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskRiQe_a/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790201539513534-1567-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskRiQe_a/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790201539513534-1567-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskRiQe_a/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790201539513534-1567-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260923 22:12:53.463665 9278 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260923 22:12:53.465479 9278 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260923 22:12:53.465643 9278 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260923 22:12:53.466163 9278 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260923 22:12:53.466220 9278 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260923 22:12:53.466260 9278 ts_tablet_manager.cc:610] Registered 0 tablets
I20260923 22:12:53.466284 9278 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
Sep 23 22:12:53 dist-test-slave-s2fj krb5kdc[9021](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790201573, etypes {rep=17 tkt=17 ses=17}, oryx/127.1.135.194@KRBTEST.COM for oryx/127.1.135.254@KRBTEST.COM
I20260923 22:12:53.473461 9278 rpc_server.cc:307] RPC server started. Bound to: 127.1.135.194:46503
I20260923 22:12:53.473809 9278 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskRiQe_a/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790201539513534-1567-0/minicluster-data/ts-1/data/info.pb
I20260923 22:12:53.474241 9408 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.1.135.194:46503 every 8 connection(s)
I20260923 22:12:53.475299 9115 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 22:12:53.466546 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.135.194:51447 (local address 127.1.135.254:35183)
0923 22:12:53.466615 (+ 69us) server_negotiation.cc:206] Beginning negotiation
0923 22:12:53.466618 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0923 22:12:53.466820 (+ 202us) server_negotiation.cc:407] Connection header received
0923 22:12:53.467270 (+ 450us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 22:12:53.467272 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 22:12:53.467287 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 22:12:53.467370 (+ 83us) server_negotiation.cc:226] Negotiated authn=SASL
0923 22:12:53.467981 (+ 611us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 22:12:53.468533 (+ 552us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 22:12:53.469533 (+ 1000us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 22:12:53.469747 (+ 214us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 22:12:53.469833 (+ 86us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 22:12:53.472463 (+ 2630us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0923 22:12:53.472465 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0923 22:12:53.472465 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0923 22:12:53.472478 (+ 13us) server_negotiation.cc:922] calling sasl_server_start()
0923 22:12:53.473967 (+ 1489us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0923 22:12:53.474401 (+ 434us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0923 22:12:53.474402 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0923 22:12:53.474402 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0923 22:12:53.474465 (+ 63us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0923 22:12:53.474766 (+ 301us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0923 22:12:53.474767 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0923 22:12:53.474767 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0923 22:12:53.474903 (+ 136us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0923 22:12:53.474956 (+ 53us) server_negotiation.cc:1035] Waiting for connection context
0923 22:12:53.475178 (+ 222us) server_negotiation.cc:299] Negotiation successful
0923 22:12:53.475238 (+ 60us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":39}
I20260923 22:12:53.475481 9302 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 22:12:53.466528 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.135.254:35183 (local address 127.1.135.194:51447)
0923 22:12:53.466694 (+ 166us) negotiation.cc:107] Waiting for socket to connect
0923 22:12:53.466732 (+ 38us) client_negotiation.cc:174] Beginning negotiation
0923 22:12:53.467181 (+ 449us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0923 22:12:53.467425 (+ 244us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0923 22:12:53.467428 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0923 22:12:53.467659 (+ 231us) client_negotiation.cc:189] Negotiated authn=SASL
0923 22:12:53.467888 (+ 229us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 22:12:53.467893 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 22:12:53.468634 (+ 741us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 22:12:53.468635 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 22:12:53.469430 (+ 795us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 22:12:53.469432 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 22:12:53.469842 (+ 410us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 22:12:53.469843 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 22:12:53.469957 (+ 114us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 22:12:53.470448 (+ 491us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0923 22:12:53.470462 (+ 14us) client_negotiation.cc:656] calling sasl_client_start()
0923 22:12:53.472336 (+ 1874us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0923 22:12:53.474053 (+ 1717us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0923 22:12:53.474060 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0923 22:12:53.474062 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0923 22:12:53.474314 (+ 252us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0923 22:12:53.474546 (+ 232us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0923 22:12:53.474548 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0923 22:12:53.474548 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0923 22:12:53.474675 (+ 127us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0923 22:12:53.474963 (+ 288us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0923 22:12:53.474967 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0923 22:12:53.475085 (+ 118us) client_negotiation.cc:769] Sending connection context
0923 22:12:53.475168 (+ 83us) client_negotiation.cc:240] Negotiation successful
0923 22:12:53.475285 (+ 117us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":129,"thread_start_us":74,"threads_started":1}
I20260923 22:12:53.475953 9409 heartbeater.cc:347] Connected to a master server at 127.1.135.254:35183
I20260923 22:12:53.476078 9409 heartbeater.cc:464] Registering TS with master...
I20260923 22:12:53.476250 9409 heartbeater.cc:511] Master 127.1.135.254:35183 requested a full tablet report, sending...
I20260923 22:12:53.476613 9066 ts_manager.cc:194] Registered new tserver with Master: 7a422c36ebff4c2384978dbc5c2b9afc (127.1.135.194:46503)
I20260923 22:12:53.477073 9066 master_service.cc:496] Signed X509 certificate for tserver {username='oryx', principal='oryx/127.1.135.194@KRBTEST.COM'} at 127.1.135.194:51447
I20260923 22:12:53.482434 9115 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 22:12:53.477037 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.135.194:43617 (local address 127.1.135.254:35183)
0923 22:12:53.477077 (+ 40us) server_negotiation.cc:206] Beginning negotiation
0923 22:12:53.477079 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0923 22:12:53.477173 (+ 94us) server_negotiation.cc:407] Connection header received
0923 22:12:53.477195 (+ 22us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 22:12:53.477196 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 22:12:53.477205 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 22:12:53.477255 (+ 50us) server_negotiation.cc:226] Negotiated authn=SASL
0923 22:12:53.477588 (+ 333us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 22:12:53.478053 (+ 465us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 22:12:53.478745 (+ 692us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 22:12:53.478942 (+ 197us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 22:12:53.479010 (+ 68us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 22:12:53.479876 (+ 866us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0923 22:12:53.479877 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0923 22:12:53.479878 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0923 22:12:53.479884 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0923 22:12:53.481318 (+ 1434us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0923 22:12:53.481649 (+ 331us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0923 22:12:53.481650 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0923 22:12:53.481650 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0923 22:12:53.481703 (+ 53us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0923 22:12:53.481901 (+ 198us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0923 22:12:53.481902 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0923 22:12:53.481902 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0923 22:12:53.482040 (+ 138us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0923 22:12:53.482122 (+ 82us) server_negotiation.cc:1035] Waiting for connection context
0923 22:12:53.482272 (+ 150us) server_negotiation.cc:299] Negotiation successful
0923 22:12:53.482344 (+ 72us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":21}
I20260923 22:12:53.482438 9415 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 22:12:53.476955 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.135.254:35183 (local address 127.1.135.194:43617)
0923 22:12:53.477076 (+ 121us) negotiation.cc:107] Waiting for socket to connect
0923 22:12:53.477088 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0923 22:12:53.477148 (+ 60us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0923 22:12:53.477277 (+ 129us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0923 22:12:53.477278 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0923 22:12:53.477456 (+ 178us) client_negotiation.cc:189] Negotiated authn=SASL
0923 22:12:53.477508 (+ 52us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 22:12:53.477509 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 22:12:53.478143 (+ 634us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 22:12:53.478144 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 22:12:53.478663 (+ 519us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 22:12:53.478664 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 22:12:53.479020 (+ 356us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 22:12:53.479021 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 22:12:53.479082 (+ 61us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 22:12:53.479497 (+ 415us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0923 22:12:53.479501 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0923 22:12:53.479773 (+ 272us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0923 22:12:53.481398 (+ 1625us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0923 22:12:53.481399 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0923 22:12:53.481400 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0923 22:12:53.481576 (+ 176us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0923 22:12:53.481771 (+ 195us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0923 22:12:53.481772 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0923 22:12:53.481772 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0923 22:12:53.481830 (+ 58us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0923 22:12:53.482114 (+ 284us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0923 22:12:53.482115 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0923 22:12:53.482159 (+ 44us) client_negotiation.cc:769] Sending connection context
0923 22:12:53.482248 (+ 89us) client_negotiation.cc:240] Negotiation successful
0923 22:12:53.482346 (+ 98us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":97,"thread_start_us":71,"threads_started":1}
I20260923 22:12:53.483491 1567 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskRiQe_a/build/release/bin/kudu as pid 9278
I20260923 22:12:53.483634 1567 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskRiQe_a/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790201539513534-1567-0/minicluster-data/ts-1/wal/instance
I20260923 22:12:53.483875 1567 external_mini_cluster.cc:1428] Setting key 3c2f8271443fe7c6a0915647a155a270
WARNING: no policy specified for oryx/127.1.135.195@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.1.135.195@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.1.135.195 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskRiQe_a/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790201539513534-1567-0/krb5kdc/oryx_127.1.135.195.keytab.
Entry for principal oryx/127.1.135.195 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskRiQe_a/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790201539513534-1567-0/krb5kdc/oryx_127.1.135.195.keytab.
I20260923 22:12:53.508435 1567 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskRiQe_a/build/release/bin/kudu
/tmp/dist-test-taskRiQe_a/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskRiQe_a/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790201539513534-1567-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskRiQe_a/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790201539513534-1567-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskRiQe_a/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790201539513534-1567-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskRiQe_a/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790201539513534-1567-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.1.135.195:0
--local_ip_for_outbound_sockets=127.1.135.195
--webserver_interface=127.1.135.195
--webserver_port=0
--tserver_master_addrs=127.1.135.254:35183
--keytab_file=/tmp/dist-test-taskRiQe_a/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790201539513534-1567-0/krb5kdc/oryx_127.1.135.195.keytab
--principal=oryx/127.1.135.195
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.1.135.212:42511
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--enable_txn_system_client_init=true with env {KRB5CCNAME=/tmp/dist-test-taskRiQe_a/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790201539513534-1567-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskRiQe_a/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790201539513534-1567-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskRiQe_a/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790201539513534-1567-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260923 22:12:53.589696 9418 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260923 22:12:53.589903 9418 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260923 22:12:53.589928 9418 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260923 22:12:53.591089 9418 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260923 22:12:53.591131 9418 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260923 22:12:53.591153 9418 flags.cc:425] Enabled experimental flag: --enable_txn_system_client_init=true
W20260923 22:12:53.591214 9418 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.1.135.195
I20260923 22:12:53.592775 9418 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.1.135.212:42511
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskRiQe_a/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790201539513534-1567-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskRiQe_a/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790201539513534-1567-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskRiQe_a/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790201539513534-1567-0/krb5kdc/oryx_127.1.135.195.keytab
--principal=oryx/127.1.135.195
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.1.135.195:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskRiQe_a/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790201539513534-1567-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.1.135.195
--webserver_port=0
--enable_txn_system_client_init=true
--tserver_master_addrs=127.1.135.254:35183
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.9418
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.1.135.195
--log_dir=/tmp/dist-test-taskRiQe_a/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790201539513534-1567-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 23 Sep 2026 20:53:37 UTC on ccycloud.quasar-rkviyx.root.comops.site
build id 4668083
I20260923 22:12:53.593022 9418 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260923 22:12:53.593267 9418 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260923 22:12:53.598287 9423 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260923 22:12:53.598294 9426 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260923 22:12:53.598297 9424 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260923 22:12:53.598592 9418 server_base.cc:1034] running on GCE node
I20260923 22:12:53.598771 9418 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260923 22:12:53.598959 9418 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260923 22:12:53.600128 9418 hybrid_clock.cc:648] HybridClock initialized: now 1790201573600113 us; error 40 us; skew 500 ppm
Sep 23 22:12:53 dist-test-slave-s2fj krb5kdc[9021](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790201573, etypes {rep=17 tkt=17 ses=17}, oryx/127.1.135.195@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260923 22:12:53.602502 9418 init.cc:377] Logged in from keytab as oryx/127.1.135.195@KRBTEST.COM (short username oryx)
I20260923 22:12:53.602931 9418 webserver.cc:460] Webserver started at http://127.1.135.195:36141/ using document root <none> and password file <none>
I20260923 22:12:53.603159 9418 fs_manager.cc:359] Metadata directory not provided
I20260923 22:12:53.603229 9418 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260923 22:12:53.603381 9418 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260923 22:12:53.604539 9418 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskRiQe_a/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790201539513534-1567-0/minicluster-data/ts-2/data/instance:
uuid: "f36bf50226164a81aa65fea3b9d4c07d"
format_stamp: "Formatted at 2026-09-23 22:12:53 on dist-test-slave-s2fj"
server_key: "2018891d99241809c5f0f004e8f8648a"
server_key_iv: "7d49e723e1febbc010d82b46b3f8a98d"
server_key_version: "encryptionkey@0"
I20260923 22:12:53.604908 9418 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskRiQe_a/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790201539513534-1567-0/minicluster-data/ts-2/wal/instance:
uuid: "f36bf50226164a81aa65fea3b9d4c07d"
format_stamp: "Formatted at 2026-09-23 22:12:53 on dist-test-slave-s2fj"
server_key: "2018891d99241809c5f0f004e8f8648a"
server_key_iv: "7d49e723e1febbc010d82b46b3f8a98d"
server_key_version: "encryptionkey@0"
I20260923 22:12:53.606240 9418 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.000s
I20260923 22:12:53.606933 9433 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260923 22:12:53.607105 9418 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260923 22:12:53.607208 9418 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskRiQe_a/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790201539513534-1567-0/minicluster-data/ts-2/data,/tmp/dist-test-taskRiQe_a/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790201539513534-1567-0/minicluster-data/ts-2/wal
uuid: "f36bf50226164a81aa65fea3b9d4c07d"
format_stamp: "Formatted at 2026-09-23 22:12:53 on dist-test-slave-s2fj"
server_key: "2018891d99241809c5f0f004e8f8648a"
server_key_iv: "7d49e723e1febbc010d82b46b3f8a98d"
server_key_version: "encryptionkey@0"
I20260923 22:12:53.607281 9418 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskRiQe_a/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790201539513534-1567-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskRiQe_a/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790201539513534-1567-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskRiQe_a/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790201539513534-1567-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260923 22:12:53.651914 9418 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260923 22:12:53.653882 9418 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260923 22:12:53.654052 9418 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260923 22:12:53.654573 9418 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260923 22:12:53.654640 9418 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260923 22:12:53.654678 9418 ts_tablet_manager.cc:610] Registered 0 tablets
I20260923 22:12:53.654699 9418 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260923 22:12:53.661628 9418 rpc_server.cc:307] RPC server started. Bound to: 127.1.135.195:40361
I20260923 22:12:53.661710 9548 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.1.135.195:40361 every 8 connection(s)
I20260923 22:12:53.661999 9418 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskRiQe_a/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790201539513534-1567-0/minicluster-data/ts-2/data/info.pb
Sep 23 22:12:53 dist-test-slave-s2fj krb5kdc[9021](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790201573, etypes {rep=17 tkt=17 ses=17}, oryx/127.1.135.195@KRBTEST.COM for oryx/127.1.135.254@KRBTEST.COM
I20260923 22:12:53.663645 1567 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskRiQe_a/build/release/bin/kudu as pid 9418
I20260923 22:12:53.663748 1567 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskRiQe_a/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790201539513534-1567-0/minicluster-data/ts-2/wal/instance
I20260923 22:12:53.663954 1567 external_mini_cluster.cc:1428] Setting key 0a32a337b30e3223efdada2ec2d24ea0
I20260923 22:12:53.665642 9115 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 22:12:53.655221 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.135.195:45945 (local address 127.1.135.254:35183)
0923 22:12:53.655284 (+ 63us) server_negotiation.cc:206] Beginning negotiation
0923 22:12:53.655289 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0923 22:12:53.655489 (+ 200us) server_negotiation.cc:407] Connection header received
0923 22:12:53.656104 (+ 615us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 22:12:53.656105 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 22:12:53.656118 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 22:12:53.656165 (+ 47us) server_negotiation.cc:226] Negotiated authn=SASL
0923 22:12:53.657052 (+ 887us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 22:12:53.657872 (+ 820us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 22:12:53.659675 (+ 1803us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 22:12:53.659899 (+ 224us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 22:12:53.659962 (+ 63us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 22:12:53.662764 (+ 2802us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0923 22:12:53.662765 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0923 22:12:53.662766 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0923 22:12:53.662773 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0923 22:12:53.664425 (+ 1652us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0923 22:12:53.664883 (+ 458us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0923 22:12:53.664883 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0923 22:12:53.664884 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0923 22:12:53.664967 (+ 83us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0923 22:12:53.665186 (+ 219us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0923 22:12:53.665187 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0923 22:12:53.665187 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0923 22:12:53.665286 (+ 99us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0923 22:12:53.665331 (+ 45us) server_negotiation.cc:1035] Waiting for connection context
0923 22:12:53.665523 (+ 192us) server_negotiation.cc:299] Negotiation successful
0923 22:12:53.665581 (+ 58us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":36}
I20260923 22:12:53.665900 9442 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 22:12:53.655230 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.135.254:35183 (local address 127.1.135.195:45945)
0923 22:12:53.655370 (+ 140us) negotiation.cc:107] Waiting for socket to connect
0923 22:12:53.655405 (+ 35us) client_negotiation.cc:174] Beginning negotiation
0923 22:12:53.655965 (+ 560us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0923 22:12:53.656261 (+ 296us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0923 22:12:53.656266 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0923 22:12:53.656675 (+ 409us) client_negotiation.cc:189] Negotiated authn=SASL
0923 22:12:53.656963 (+ 288us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 22:12:53.656968 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 22:12:53.658005 (+ 1037us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 22:12:53.658006 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 22:12:53.659560 (+ 1554us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 22:12:53.659563 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 22:12:53.659967 (+ 404us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 22:12:53.659969 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 22:12:53.660191 (+ 222us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 22:12:53.660782 (+ 591us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0923 22:12:53.660792 (+ 10us) client_negotiation.cc:656] calling sasl_client_start()
0923 22:12:53.662625 (+ 1833us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0923 22:12:53.664519 (+ 1894us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0923 22:12:53.664538 (+ 19us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0923 22:12:53.664556 (+ 18us) client_negotiation.cc:760] calling sasl_client_step()
0923 22:12:53.664798 (+ 242us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0923 22:12:53.665034 (+ 236us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0923 22:12:53.665034 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0923 22:12:53.665035 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0923 22:12:53.665139 (+ 104us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0923 22:12:53.665322 (+ 183us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0923 22:12:53.665325 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0923 22:12:53.665430 (+ 105us) client_negotiation.cc:769] Sending connection context
0923 22:12:53.665507 (+ 77us) client_negotiation.cc:240] Negotiation successful
0923 22:12:53.665632 (+ 125us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":104,"thread_start_us":81,"threads_started":1}
I20260923 22:12:53.666334 9549 heartbeater.cc:347] Connected to a master server at 127.1.135.254:35183
I20260923 22:12:53.666419 9549 heartbeater.cc:464] Registering TS with master...
I20260923 22:12:53.666599 9549 heartbeater.cc:511] Master 127.1.135.254:35183 requested a full tablet report, sending...
I20260923 22:12:53.667001 9065 ts_manager.cc:194] Registered new tserver with Master: f36bf50226164a81aa65fea3b9d4c07d (127.1.135.195:40361)
I20260923 22:12:53.667519 9065 master_service.cc:496] Signed X509 certificate for tserver {username='oryx', principal='oryx/127.1.135.195@KRBTEST.COM'} at 127.1.135.195:45945
I20260923 22:12:53.672571 9115 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 22:12:53.667387 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.135.195:45483 (local address 127.1.135.254:35183)
0923 22:12:53.667426 (+ 39us) server_negotiation.cc:206] Beginning negotiation
0923 22:12:53.667428 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0923 22:12:53.667496 (+ 68us) server_negotiation.cc:407] Connection header received
0923 22:12:53.667564 (+ 68us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 22:12:53.667565 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 22:12:53.667572 (+ 7us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 22:12:53.667646 (+ 74us) server_negotiation.cc:226] Negotiated authn=SASL
0923 22:12:53.668139 (+ 493us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 22:12:53.668602 (+ 463us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 22:12:53.669293 (+ 691us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 22:12:53.669502 (+ 209us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 22:12:53.669583 (+ 81us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 22:12:53.670359 (+ 776us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0923 22:12:53.670360 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0923 22:12:53.670361 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0923 22:12:53.670367 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0923 22:12:53.671445 (+ 1078us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0923 22:12:53.671791 (+ 346us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0923 22:12:53.671792 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0923 22:12:53.671793 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0923 22:12:53.671849 (+ 56us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0923 22:12:53.672048 (+ 199us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0923 22:12:53.672048 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0923 22:12:53.672049 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0923 22:12:53.672179 (+ 130us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0923 22:12:53.672256 (+ 77us) server_negotiation.cc:1035] Waiting for connection context
0923 22:12:53.672417 (+ 161us) server_negotiation.cc:299] Negotiation successful
0923 22:12:53.672484 (+ 67us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":21}
I20260923 22:12:53.672600 9555 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 22:12:53.667308 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.135.254:35183 (local address 127.1.135.195:45483)
0923 22:12:53.667434 (+ 126us) negotiation.cc:107] Waiting for socket to connect
0923 22:12:53.667445 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0923 22:12:53.667503 (+ 58us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0923 22:12:53.667757 (+ 254us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0923 22:12:53.667758 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0923 22:12:53.667965 (+ 207us) client_negotiation.cc:189] Negotiated authn=SASL
0923 22:12:53.668025 (+ 60us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 22:12:53.668026 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 22:12:53.668692 (+ 666us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 22:12:53.668693 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 22:12:53.669202 (+ 509us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 22:12:53.669204 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 22:12:53.669584 (+ 380us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 22:12:53.669585 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 22:12:53.669652 (+ 67us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 22:12:53.670001 (+ 349us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0923 22:12:53.670006 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0923 22:12:53.670261 (+ 255us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0923 22:12:53.671533 (+ 1272us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0923 22:12:53.671534 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0923 22:12:53.671535 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0923 22:12:53.671705 (+ 170us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0923 22:12:53.671916 (+ 211us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0923 22:12:53.671917 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0923 22:12:53.671917 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0923 22:12:53.671975 (+ 58us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0923 22:12:53.672249 (+ 274us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0923 22:12:53.672250 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0923 22:12:53.672296 (+ 46us) client_negotiation.cc:769] Sending connection context
0923 22:12:53.672379 (+ 83us) client_negotiation.cc:240] Negotiation successful
0923 22:12:53.672483 (+ 104us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":103,"thread_start_us":78,"threads_started":1}
I20260923 22:12:53.675689 1567 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
Sep 23 22:12:53 dist-test-slave-s2fj krb5kdc[9021](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790201573, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-user@KRBTEST.COM:
Sep 23 22:12:53 dist-test-slave-s2fj krb5kdc[9021](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, test-user@KRBTEST.COM for kudu/127.1.135.254@KRBTEST.COM, Server not found in Kerberos database
Sep 23 22:12:53 dist-test-slave-s2fj krb5kdc[9021](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, test-user@KRBTEST.COM for kudu/127.1.135.254@KRBTEST.COM, Server not found in Kerberos database
W20260923 22:12:53.696254 9115 negotiation.cc:337] Failed RPC negotiation. Trace:
0923 22:12:53.691089 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48894 (local address 127.1.135.254:35183)
0923 22:12:53.691154 (+ 65us) server_negotiation.cc:206] Beginning negotiation
0923 22:12:53.691156 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0923 22:12:53.691164 (+ 8us) server_negotiation.cc:407] Connection header received
0923 22:12:53.691188 (+ 24us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 22:12:53.691188 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 22:12:53.691197 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 22:12:53.691246 (+ 49us) server_negotiation.cc:226] Negotiated authn=SASL
0923 22:12:53.691778 (+ 532us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 22:12:53.692347 (+ 569us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 22:12:53.693053 (+ 706us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 22:12:53.693231 (+ 178us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 22:12:53.693282 (+ 51us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 22:12:53.696151 (+ 2869us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:48894: BlockingRecv error: recv got EOF from 127.0.0.1:48894 (error 108)
Metrics: {"server-negotiator.queue_time_us":37}
Sep 23 22:12:53 dist-test-slave-s2fj krb5kdc[9021](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790201573, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for oryx/127.1.135.254@KRBTEST.COM
I20260923 22:12:53.703673 9115 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 22:12:53.697047 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48896 (local address 127.1.135.254:35183)
0923 22:12:53.697103 (+ 56us) server_negotiation.cc:206] Beginning negotiation
0923 22:12:53.697106 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0923 22:12:53.697167 (+ 61us) server_negotiation.cc:407] Connection header received
0923 22:12:53.697241 (+ 74us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 22:12:53.697242 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 22:12:53.697251 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 22:12:53.697301 (+ 50us) server_negotiation.cc:226] Negotiated authn=SASL
0923 22:12:53.697756 (+ 455us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 22:12:53.698220 (+ 464us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 22:12:53.698908 (+ 688us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 22:12:53.699098 (+ 190us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 22:12:53.699182 (+ 84us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 22:12:53.701217 (+ 2035us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0923 22:12:53.701218 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0923 22:12:53.701219 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0923 22:12:53.701226 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0923 22:12:53.702553 (+ 1327us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0923 22:12:53.702909 (+ 356us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0923 22:12:53.702910 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0923 22:12:53.702911 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0923 22:12:53.702968 (+ 57us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0923 22:12:53.703172 (+ 204us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0923 22:12:53.703173 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0923 22:12:53.703173 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0923 22:12:53.703279 (+ 106us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0923 22:12:53.703380 (+ 101us) server_negotiation.cc:1035] Waiting for connection context
0923 22:12:53.703494 (+ 114us) server_negotiation.cc:299] Negotiation successful
0923 22:12:53.703573 (+ 79us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":33}
I20260923 22:12:53.705150 9065 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:48896:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260923 22:12:53.706634 9065 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260923 22:12:53.713023 9571 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 22:12:53.708749 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.135.193:33215 (local address 127.0.0.1:40636)
0923 22:12:53.709189 (+ 440us) negotiation.cc:107] Waiting for socket to connect
0923 22:12:53.709198 (+ 9us) client_negotiation.cc:174] Beginning negotiation
0923 22:12:53.709251 (+ 53us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0923 22:12:53.709426 (+ 175us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0923 22:12:53.709427 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0923 22:12:53.709429 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0923 22:12:53.709486 (+ 57us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 22:12:53.709487 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 22:12:53.710591 (+ 1104us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 22:12:53.710592 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 22:12:53.711977 (+ 1385us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 22:12:53.711980 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 22:12:53.712714 (+ 734us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 22:12:53.712715 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 22:12:53.712792 (+ 77us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 22:12:53.712807 (+ 15us) client_negotiation.cc:769] Sending connection context
0923 22:12:53.712881 (+ 74us) client_negotiation.cc:240] Negotiation successful
0923 22:12:53.712931 (+ 50us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":427,"mutex_wait_us":2,"thread_start_us":70,"threads_started":1}
I20260923 22:12:53.713037 9573 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 22:12:53.708864 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40636 (local address 127.1.135.193:33215)
0923 22:12:53.709055 (+ 191us) server_negotiation.cc:206] Beginning negotiation
0923 22:12:53.709061 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0923 22:12:53.709255 (+ 194us) server_negotiation.cc:407] Connection header received
0923 22:12:53.709309 (+ 54us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 22:12:53.709311 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 22:12:53.709326 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 22:12:53.709575 (+ 249us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0923 22:12:53.709636 (+ 61us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 22:12:53.710495 (+ 859us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 22:12:53.712107 (+ 1612us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 22:12:53.712609 (+ 502us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 22:12:53.712711 (+ 102us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 22:12:53.712791 (+ 80us) server_negotiation.cc:1035] Waiting for connection context
0923 22:12:53.712896 (+ 105us) server_negotiation.cc:299] Negotiation successful
0923 22:12:53.712936 (+ 40us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":147,"thread_start_us":79,"threads_started":1}
I20260923 22:12:53.713361 9576 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 22:12:53.709048 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34286 (local address 127.1.135.195:40361)
0923 22:12:53.709198 (+ 150us) server_negotiation.cc:206] Beginning negotiation
0923 22:12:53.709203 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0923 22:12:53.709443 (+ 240us) server_negotiation.cc:407] Connection header received
0923 22:12:53.709487 (+ 44us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 22:12:53.709489 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 22:12:53.709506 (+ 17us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 22:12:53.709579 (+ 73us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0923 22:12:53.709770 (+ 191us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 22:12:53.710815 (+ 1045us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 22:12:53.712096 (+ 1281us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 22:12:53.712761 (+ 665us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 22:12:53.712816 (+ 55us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 22:12:53.712883 (+ 67us) server_negotiation.cc:1035] Waiting for connection context
0923 22:12:53.713228 (+ 345us) server_negotiation.cc:299] Negotiation successful
0923 22:12:53.713270 (+ 42us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":120,"thread_start_us":89,"threads_started":1}
I20260923 22:12:53.713505 9575 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 22:12:53.708946 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.135.195:40361 (local address 127.0.0.1:34286)
0923 22:12:53.709322 (+ 376us) negotiation.cc:107] Waiting for socket to connect
0923 22:12:53.709329 (+ 7us) client_negotiation.cc:174] Beginning negotiation
0923 22:12:53.709386 (+ 57us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0923 22:12:53.709645 (+ 259us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0923 22:12:53.709646 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0923 22:12:53.709648 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0923 22:12:53.709697 (+ 49us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 22:12:53.709698 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 22:12:53.710875 (+ 1177us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 22:12:53.710876 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 22:12:53.711987 (+ 1111us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 22:12:53.711989 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 22:12:53.712911 (+ 922us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 22:12:53.712912 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 22:12:53.712970 (+ 58us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 22:12:53.712974 (+ 4us) client_negotiation.cc:769] Sending connection context
0923 22:12:53.713019 (+ 45us) client_negotiation.cc:240] Negotiation successful
0923 22:12:53.713060 (+ 41us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":356,"mutex_wait_us":35,"thread_start_us":65,"threads_started":1}
I20260923 22:12:53.713688 9572 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 22:12:53.708749 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.135.194:46503 (local address 127.0.0.1:32868)
0923 22:12:53.709038 (+ 289us) negotiation.cc:107] Waiting for socket to connect
0923 22:12:53.709081 (+ 43us) client_negotiation.cc:174] Beginning negotiation
0923 22:12:53.709149 (+ 68us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0923 22:12:53.709381 (+ 232us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0923 22:12:53.709385 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0923 22:12:53.709387 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0923 22:12:53.709459 (+ 72us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 22:12:53.709461 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 22:12:53.710916 (+ 1455us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 22:12:53.710917 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 22:12:53.711864 (+ 947us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 22:12:53.711866 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 22:12:53.712889 (+ 1023us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 22:12:53.712890 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 22:12:53.712951 (+ 61us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 22:12:53.712955 (+ 4us) client_negotiation.cc:769] Sending connection context
0923 22:12:53.713007 (+ 52us) client_negotiation.cc:240] Negotiation successful
0923 22:12:53.713047 (+ 40us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":258,"thread_start_us":59,"threads_started":1}
I20260923 22:12:53.713529 9574 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 22:12:53.708870 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:32868 (local address 127.1.135.194:46503)
0923 22:12:53.709136 (+ 266us) server_negotiation.cc:206] Beginning negotiation
0923 22:12:53.709139 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0923 22:12:53.709149 (+ 10us) server_negotiation.cc:407] Connection header received
0923 22:12:53.709256 (+ 107us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 22:12:53.709259 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 22:12:53.709278 (+ 19us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 22:12:53.709369 (+ 91us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0923 22:12:53.709555 (+ 186us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 22:12:53.710805 (+ 1250us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 22:12:53.711948 (+ 1143us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 22:12:53.712706 (+ 758us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 22:12:53.712779 (+ 73us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 22:12:53.712851 (+ 72us) server_negotiation.cc:1035] Waiting for connection context
0923 22:12:53.713301 (+ 450us) server_negotiation.cc:299] Negotiation successful
0923 22:12:53.713344 (+ 43us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":234,"thread_start_us":77,"threads_started":1}
I20260923 22:12:53.713802 9199 tablet_service.cc:1467] Processing CreateTablet for tablet 46d37003ec784a66bbd9328435345fc3 (DEFAULT_TABLE table=test-table [id=52213bf300ff47099f8c6b0ffb194129]), partition=RANGE (key) PARTITION UNBOUNDED
I20260923 22:12:53.714167 9199 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 46d37003ec784a66bbd9328435345fc3. 1 dirs total, 0 dirs full, 0 dirs failed
I20260923 22:12:53.714084 9483 tablet_service.cc:1467] Processing CreateTablet for tablet 46d37003ec784a66bbd9328435345fc3 (DEFAULT_TABLE table=test-table [id=52213bf300ff47099f8c6b0ffb194129]), partition=RANGE (key) PARTITION UNBOUNDED
I20260923 22:12:53.714380 9483 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 46d37003ec784a66bbd9328435345fc3. 1 dirs total, 0 dirs full, 0 dirs failed
I20260923 22:12:53.714327 9343 tablet_service.cc:1467] Processing CreateTablet for tablet 46d37003ec784a66bbd9328435345fc3 (DEFAULT_TABLE table=test-table [id=52213bf300ff47099f8c6b0ffb194129]), partition=RANGE (key) PARTITION UNBOUNDED
I20260923 22:12:53.714573 9343 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 46d37003ec784a66bbd9328435345fc3. 1 dirs total, 0 dirs full, 0 dirs failed
I20260923 22:12:53.717612 9577 tablet_bootstrap.cc:492] T 46d37003ec784a66bbd9328435345fc3 P 7a422c36ebff4c2384978dbc5c2b9afc: Bootstrap starting.
I20260923 22:12:53.718044 9579 tablet_bootstrap.cc:492] T 46d37003ec784a66bbd9328435345fc3 P 07273d7511d54b88b18936d1c8590b03: Bootstrap starting.
I20260923 22:12:53.718345 9577 tablet_bootstrap.cc:654] T 46d37003ec784a66bbd9328435345fc3 P 7a422c36ebff4c2384978dbc5c2b9afc: Neither blocks nor log segments found. Creating new log.
I20260923 22:12:53.718629 9579 tablet_bootstrap.cc:654] T 46d37003ec784a66bbd9328435345fc3 P 07273d7511d54b88b18936d1c8590b03: Neither blocks nor log segments found. Creating new log.
I20260923 22:12:53.718642 9577 log.cc:824] T 46d37003ec784a66bbd9328435345fc3 P 7a422c36ebff4c2384978dbc5c2b9afc: Log is configured to *not* fsync() on all Append() calls
I20260923 22:12:53.718928 9579 log.cc:824] T 46d37003ec784a66bbd9328435345fc3 P 07273d7511d54b88b18936d1c8590b03: Log is configured to *not* fsync() on all Append() calls
I20260923 22:12:53.719267 9578 tablet_bootstrap.cc:492] T 46d37003ec784a66bbd9328435345fc3 P f36bf50226164a81aa65fea3b9d4c07d: Bootstrap starting.
I20260923 22:12:53.719465 9577 tablet_bootstrap.cc:492] T 46d37003ec784a66bbd9328435345fc3 P 7a422c36ebff4c2384978dbc5c2b9afc: No bootstrap required, opened a new log
I20260923 22:12:53.719547 9577 ts_tablet_manager.cc:1397] T 46d37003ec784a66bbd9328435345fc3 P 7a422c36ebff4c2384978dbc5c2b9afc: Time spent bootstrapping tablet: real 0.002s user 0.002s sys 0.000s
I20260923 22:12:53.719672 9579 tablet_bootstrap.cc:492] T 46d37003ec784a66bbd9328435345fc3 P 07273d7511d54b88b18936d1c8590b03: No bootstrap required, opened a new log
I20260923 22:12:53.719758 9579 ts_tablet_manager.cc:1397] T 46d37003ec784a66bbd9328435345fc3 P 07273d7511d54b88b18936d1c8590b03: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20260923 22:12:53.719866 9578 tablet_bootstrap.cc:654] T 46d37003ec784a66bbd9328435345fc3 P f36bf50226164a81aa65fea3b9d4c07d: Neither blocks nor log segments found. Creating new log.
I20260923 22:12:53.720220 9578 log.cc:824] T 46d37003ec784a66bbd9328435345fc3 P f36bf50226164a81aa65fea3b9d4c07d: Log is configured to *not* fsync() on all Append() calls
I20260923 22:12:53.720992 9578 tablet_bootstrap.cc:492] T 46d37003ec784a66bbd9328435345fc3 P f36bf50226164a81aa65fea3b9d4c07d: No bootstrap required, opened a new log
I20260923 22:12:53.721062 9578 ts_tablet_manager.cc:1397] T 46d37003ec784a66bbd9328435345fc3 P f36bf50226164a81aa65fea3b9d4c07d: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20260923 22:12:53.721524 9579 raft_consensus.cc:348] T 46d37003ec784a66bbd9328435345fc3 P 07273d7511d54b88b18936d1c8590b03 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "07273d7511d54b88b18936d1c8590b03" member_type: VOTER last_known_addr { host: "127.1.135.193" port: 33215 } } peers { permanent_uuid: "f36bf50226164a81aa65fea3b9d4c07d" member_type: VOTER last_known_addr { host: "127.1.135.195" port: 40361 } } peers { permanent_uuid: "7a422c36ebff4c2384978dbc5c2b9afc" member_type: VOTER last_known_addr { host: "127.1.135.194" port: 46503 } }
I20260923 22:12:53.721675 9579 raft_consensus.cc:374] T 46d37003ec784a66bbd9328435345fc3 P 07273d7511d54b88b18936d1c8590b03 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260923 22:12:53.721717 9579 raft_consensus.cc:729] T 46d37003ec784a66bbd9328435345fc3 P 07273d7511d54b88b18936d1c8590b03 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 07273d7511d54b88b18936d1c8590b03, State: Initialized, Role: FOLLOWER
I20260923 22:12:53.721693 9577 raft_consensus.cc:348] T 46d37003ec784a66bbd9328435345fc3 P 7a422c36ebff4c2384978dbc5c2b9afc [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "07273d7511d54b88b18936d1c8590b03" member_type: VOTER last_known_addr { host: "127.1.135.193" port: 33215 } } peers { permanent_uuid: "f36bf50226164a81aa65fea3b9d4c07d" member_type: VOTER last_known_addr { host: "127.1.135.195" port: 40361 } } peers { permanent_uuid: "7a422c36ebff4c2384978dbc5c2b9afc" member_type: VOTER last_known_addr { host: "127.1.135.194" port: 46503 } }
I20260923 22:12:53.721860 9577 raft_consensus.cc:374] T 46d37003ec784a66bbd9328435345fc3 P 7a422c36ebff4c2384978dbc5c2b9afc [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260923 22:12:53.721903 9577 raft_consensus.cc:729] T 46d37003ec784a66bbd9328435345fc3 P 7a422c36ebff4c2384978dbc5c2b9afc [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 7a422c36ebff4c2384978dbc5c2b9afc, State: Initialized, Role: FOLLOWER
I20260923 22:12:53.721870 9579 consensus_queue.cc:260] T 46d37003ec784a66bbd9328435345fc3 P 07273d7511d54b88b18936d1c8590b03 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "07273d7511d54b88b18936d1c8590b03" member_type: VOTER last_known_addr { host: "127.1.135.193" port: 33215 } } peers { permanent_uuid: "f36bf50226164a81aa65fea3b9d4c07d" member_type: VOTER last_known_addr { host: "127.1.135.195" port: 40361 } } peers { permanent_uuid: "7a422c36ebff4c2384978dbc5c2b9afc" member_type: VOTER last_known_addr { host: "127.1.135.194" port: 46503 } }
I20260923 22:12:53.722048 9577 consensus_queue.cc:260] T 46d37003ec784a66bbd9328435345fc3 P 7a422c36ebff4c2384978dbc5c2b9afc [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "07273d7511d54b88b18936d1c8590b03" member_type: VOTER last_known_addr { host: "127.1.135.193" port: 33215 } } peers { permanent_uuid: "f36bf50226164a81aa65fea3b9d4c07d" member_type: VOTER last_known_addr { host: "127.1.135.195" port: 40361 } } peers { permanent_uuid: "7a422c36ebff4c2384978dbc5c2b9afc" member_type: VOTER last_known_addr { host: "127.1.135.194" port: 46503 } }
I20260923 22:12:53.722149 9579 ts_tablet_manager.cc:1428] T 46d37003ec784a66bbd9328435345fc3 P 07273d7511d54b88b18936d1c8590b03: Time spent starting tablet: real 0.002s user 0.003s sys 0.000s
I20260923 22:12:53.722194 9269 heartbeater.cc:503] Master 127.1.135.254:35183 was elected leader, sending a full tablet report...
I20260923 22:12:53.722299 9577 ts_tablet_manager.cc:1428] T 46d37003ec784a66bbd9328435345fc3 P 7a422c36ebff4c2384978dbc5c2b9afc: Time spent starting tablet: real 0.003s user 0.003s sys 0.000s
I20260923 22:12:53.722399 9409 heartbeater.cc:503] Master 127.1.135.254:35183 was elected leader, sending a full tablet report...
I20260923 22:12:53.723095 9578 raft_consensus.cc:348] T 46d37003ec784a66bbd9328435345fc3 P f36bf50226164a81aa65fea3b9d4c07d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "07273d7511d54b88b18936d1c8590b03" member_type: VOTER last_known_addr { host: "127.1.135.193" port: 33215 } } peers { permanent_uuid: "f36bf50226164a81aa65fea3b9d4c07d" member_type: VOTER last_known_addr { host: "127.1.135.195" port: 40361 } } peers { permanent_uuid: "7a422c36ebff4c2384978dbc5c2b9afc" member_type: VOTER last_known_addr { host: "127.1.135.194" port: 46503 } }
I20260923 22:12:53.723253 9578 raft_consensus.cc:374] T 46d37003ec784a66bbd9328435345fc3 P f36bf50226164a81aa65fea3b9d4c07d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260923 22:12:53.723305 9578 raft_consensus.cc:729] T 46d37003ec784a66bbd9328435345fc3 P f36bf50226164a81aa65fea3b9d4c07d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: f36bf50226164a81aa65fea3b9d4c07d, State: Initialized, Role: FOLLOWER
I20260923 22:12:53.723414 9578 consensus_queue.cc:260] T 46d37003ec784a66bbd9328435345fc3 P f36bf50226164a81aa65fea3b9d4c07d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "07273d7511d54b88b18936d1c8590b03" member_type: VOTER last_known_addr { host: "127.1.135.193" port: 33215 } } peers { permanent_uuid: "f36bf50226164a81aa65fea3b9d4c07d" member_type: VOTER last_known_addr { host: "127.1.135.195" port: 40361 } } peers { permanent_uuid: "7a422c36ebff4c2384978dbc5c2b9afc" member_type: VOTER last_known_addr { host: "127.1.135.194" port: 46503 } }
I20260923 22:12:53.723678 9578 ts_tablet_manager.cc:1428] T 46d37003ec784a66bbd9328435345fc3 P f36bf50226164a81aa65fea3b9d4c07d: Time spent starting tablet: real 0.003s user 0.003s sys 0.000s
I20260923 22:12:53.723712 9549 heartbeater.cc:503] Master 127.1.135.254:35183 was elected leader, sending a full tablet report...
W20260923 22:12:53.724319 9410 tablet.cc:2369] T 46d37003ec784a66bbd9328435345fc3 P 7a422c36ebff4c2384978dbc5c2b9afc: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260923 22:12:53.811934 9585 raft_consensus.cc:482] T 46d37003ec784a66bbd9328435345fc3 P f36bf50226164a81aa65fea3b9d4c07d [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260923 22:12:53.812116 9585 raft_consensus.cc:504] T 46d37003ec784a66bbd9328435345fc3 P f36bf50226164a81aa65fea3b9d4c07d [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "07273d7511d54b88b18936d1c8590b03" member_type: VOTER last_known_addr { host: "127.1.135.193" port: 33215 } } peers { permanent_uuid: "f36bf50226164a81aa65fea3b9d4c07d" member_type: VOTER last_known_addr { host: "127.1.135.195" port: 40361 } } peers { permanent_uuid: "7a422c36ebff4c2384978dbc5c2b9afc" member_type: VOTER last_known_addr { host: "127.1.135.194" port: 46503 } }
I20260923 22:12:53.812522 9585 leader_election.cc:288] T 46d37003ec784a66bbd9328435345fc3 P f36bf50226164a81aa65fea3b9d4c07d [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 07273d7511d54b88b18936d1c8590b03 (127.1.135.193:33215), 7a422c36ebff4c2384978dbc5c2b9afc (127.1.135.194:46503)
I20260923 22:12:53.816298 9573 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 22:12:53.812724 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.135.195:50461 (local address 127.1.135.193:33215)
0923 22:12:53.812769 (+ 45us) server_negotiation.cc:206] Beginning negotiation
0923 22:12:53.812770 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0923 22:12:53.812780 (+ 10us) server_negotiation.cc:407] Connection header received
0923 22:12:53.812873 (+ 93us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 22:12:53.812874 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 22:12:53.812884 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 22:12:53.812950 (+ 66us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0923 22:12:53.813147 (+ 197us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 22:12:53.813901 (+ 754us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 22:12:53.815349 (+ 1448us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 22:12:53.815899 (+ 550us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 22:12:53.815975 (+ 76us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 22:12:53.815983 (+ 8us) server_negotiation.cc:1035] Waiting for connection context
0923 22:12:53.816200 (+ 217us) server_negotiation.cc:299] Negotiation successful
0923 22:12:53.816236 (+ 36us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":24}
I20260923 22:12:53.816298 9574 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 22:12:53.812808 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.135.195:60051 (local address 127.1.135.194:46503)
0923 22:12:53.812856 (+ 48us) server_negotiation.cc:206] Beginning negotiation
0923 22:12:53.812859 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0923 22:12:53.812910 (+ 51us) server_negotiation.cc:407] Connection header received
0923 22:12:53.813029 (+ 119us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 22:12:53.813030 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 22:12:53.813037 (+ 7us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 22:12:53.813084 (+ 47us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0923 22:12:53.813229 (+ 145us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 22:12:53.813889 (+ 660us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 22:12:53.815347 (+ 1458us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 22:12:53.815899 (+ 552us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 22:12:53.815975 (+ 76us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 22:12:53.815983 (+ 8us) server_negotiation.cc:1035] Waiting for connection context
0923 22:12:53.816171 (+ 188us) server_negotiation.cc:299] Negotiation successful
0923 22:12:53.816226 (+ 55us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":24}
I20260923 22:12:53.816334 9442 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 22:12:53.812680 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.135.193:33215 (local address 127.1.135.195:50461)
0923 22:12:53.812737 (+ 57us) negotiation.cc:107] Waiting for socket to connect
0923 22:12:53.812748 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0923 22:12:53.812793 (+ 45us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0923 22:12:53.812966 (+ 173us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0923 22:12:53.812967 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0923 22:12:53.812969 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0923 22:12:53.813056 (+ 87us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 22:12:53.813057 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 22:12:53.813972 (+ 915us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 22:12:53.813972 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 22:12:53.815258 (+ 1286us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 22:12:53.815261 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 22:12:53.815985 (+ 724us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 22:12:53.815986 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 22:12:53.816115 (+ 129us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 22:12:53.816118 (+ 3us) client_negotiation.cc:769] Sending connection context
0923 22:12:53.816163 (+ 45us) client_negotiation.cc:240] Negotiation successful
0923 22:12:53.816227 (+ 64us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":38,"mutex_wait_us":15}
I20260923 22:12:53.816334 9586 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 22:12:53.812696 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.135.194:46503 (local address 127.1.135.195:60051)
0923 22:12:53.812852 (+ 156us) negotiation.cc:107] Waiting for socket to connect
0923 22:12:53.812861 (+ 9us) client_negotiation.cc:174] Beginning negotiation
0923 22:12:53.812944 (+ 83us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0923 22:12:53.813103 (+ 159us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0923 22:12:53.813104 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0923 22:12:53.813106 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0923 22:12:53.813163 (+ 57us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 22:12:53.813166 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 22:12:53.813972 (+ 806us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 22:12:53.813972 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 22:12:53.815259 (+ 1287us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 22:12:53.815261 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 22:12:53.815985 (+ 724us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 22:12:53.815986 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 22:12:53.816099 (+ 113us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 22:12:53.816103 (+ 4us) client_negotiation.cc:769] Sending connection context
0923 22:12:53.816163 (+ 60us) client_negotiation.cc:240] Negotiation successful
0923 22:12:53.816227 (+ 64us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":141,"mutex_wait_us":127,"thread_start_us":109,"threads_started":1}
I20260923 22:12:53.816716 9223 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "46d37003ec784a66bbd9328435345fc3" candidate_uuid: "f36bf50226164a81aa65fea3b9d4c07d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "07273d7511d54b88b18936d1c8590b03" is_pre_election: true
I20260923 22:12:53.816895 9363 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "46d37003ec784a66bbd9328435345fc3" candidate_uuid: "f36bf50226164a81aa65fea3b9d4c07d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7a422c36ebff4c2384978dbc5c2b9afc" is_pre_election: true
I20260923 22:12:53.816931 9223 raft_consensus.cc:2454] T 46d37003ec784a66bbd9328435345fc3 P 07273d7511d54b88b18936d1c8590b03 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f36bf50226164a81aa65fea3b9d4c07d in term 0.
I20260923 22:12:53.817024 9363 raft_consensus.cc:2454] T 46d37003ec784a66bbd9328435345fc3 P 7a422c36ebff4c2384978dbc5c2b9afc [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f36bf50226164a81aa65fea3b9d4c07d in term 0.
I20260923 22:12:53.817166 9435 leader_election.cc:302] T 46d37003ec784a66bbd9328435345fc3 P f36bf50226164a81aa65fea3b9d4c07d [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 07273d7511d54b88b18936d1c8590b03, f36bf50226164a81aa65fea3b9d4c07d; no voters:
I20260923 22:12:53.817313 9585 raft_consensus.cc:2781] T 46d37003ec784a66bbd9328435345fc3 P f36bf50226164a81aa65fea3b9d4c07d [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260923 22:12:53.817368 9585 raft_consensus.cc:482] T 46d37003ec784a66bbd9328435345fc3 P f36bf50226164a81aa65fea3b9d4c07d [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260923 22:12:53.817397 9585 raft_consensus.cc:3037] T 46d37003ec784a66bbd9328435345fc3 P f36bf50226164a81aa65fea3b9d4c07d [term 0 FOLLOWER]: Advancing to term 1
I20260923 22:12:53.818446 9585 raft_consensus.cc:504] T 46d37003ec784a66bbd9328435345fc3 P f36bf50226164a81aa65fea3b9d4c07d [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "07273d7511d54b88b18936d1c8590b03" member_type: VOTER last_known_addr { host: "127.1.135.193" port: 33215 } } peers { permanent_uuid: "f36bf50226164a81aa65fea3b9d4c07d" member_type: VOTER last_known_addr { host: "127.1.135.195" port: 40361 } } peers { permanent_uuid: "7a422c36ebff4c2384978dbc5c2b9afc" member_type: VOTER last_known_addr { host: "127.1.135.194" port: 46503 } }
I20260923 22:12:53.818596 9585 leader_election.cc:288] T 46d37003ec784a66bbd9328435345fc3 P f36bf50226164a81aa65fea3b9d4c07d [CANDIDATE]: Term 1 election: Requested vote from peers 07273d7511d54b88b18936d1c8590b03 (127.1.135.193:33215), 7a422c36ebff4c2384978dbc5c2b9afc (127.1.135.194:46503)
I20260923 22:12:53.818722 9223 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "46d37003ec784a66bbd9328435345fc3" candidate_uuid: "f36bf50226164a81aa65fea3b9d4c07d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "07273d7511d54b88b18936d1c8590b03"
I20260923 22:12:53.818770 9363 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "46d37003ec784a66bbd9328435345fc3" candidate_uuid: "f36bf50226164a81aa65fea3b9d4c07d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7a422c36ebff4c2384978dbc5c2b9afc"
I20260923 22:12:53.818806 9223 raft_consensus.cc:3037] T 46d37003ec784a66bbd9328435345fc3 P 07273d7511d54b88b18936d1c8590b03 [term 0 FOLLOWER]: Advancing to term 1
I20260923 22:12:53.818862 9363 raft_consensus.cc:3037] T 46d37003ec784a66bbd9328435345fc3 P 7a422c36ebff4c2384978dbc5c2b9afc [term 0 FOLLOWER]: Advancing to term 1
I20260923 22:12:53.819697 9223 raft_consensus.cc:2454] T 46d37003ec784a66bbd9328435345fc3 P 07273d7511d54b88b18936d1c8590b03 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f36bf50226164a81aa65fea3b9d4c07d in term 1.
I20260923 22:12:53.819700 9363 raft_consensus.cc:2454] T 46d37003ec784a66bbd9328435345fc3 P 7a422c36ebff4c2384978dbc5c2b9afc [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f36bf50226164a81aa65fea3b9d4c07d in term 1.
I20260923 22:12:53.819856 9435 leader_election.cc:302] T 46d37003ec784a66bbd9328435345fc3 P f36bf50226164a81aa65fea3b9d4c07d [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 07273d7511d54b88b18936d1c8590b03, f36bf50226164a81aa65fea3b9d4c07d; no voters:
I20260923 22:12:53.819947 9585 raft_consensus.cc:2781] T 46d37003ec784a66bbd9328435345fc3 P f36bf50226164a81aa65fea3b9d4c07d [term 1 FOLLOWER]: Leader election won for term 1
I20260923 22:12:53.820118 9585 raft_consensus.cc:686] T 46d37003ec784a66bbd9328435345fc3 P f36bf50226164a81aa65fea3b9d4c07d [term 1 LEADER]: Becoming Leader. State: Replica: f36bf50226164a81aa65fea3b9d4c07d, State: Running, Role: LEADER
I20260923 22:12:53.820221 9585 consensus_queue.cc:237] T 46d37003ec784a66bbd9328435345fc3 P f36bf50226164a81aa65fea3b9d4c07d [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "07273d7511d54b88b18936d1c8590b03" member_type: VOTER last_known_addr { host: "127.1.135.193" port: 33215 } } peers { permanent_uuid: "f36bf50226164a81aa65fea3b9d4c07d" member_type: VOTER last_known_addr { host: "127.1.135.195" port: 40361 } } peers { permanent_uuid: "7a422c36ebff4c2384978dbc5c2b9afc" member_type: VOTER last_known_addr { host: "127.1.135.194" port: 46503 } }
I20260923 22:12:53.821058 9066 catalog_manager.cc:5495] T 46d37003ec784a66bbd9328435345fc3 P f36bf50226164a81aa65fea3b9d4c07d reported cstate change: term changed from 0 to 1, leader changed from <none> to f36bf50226164a81aa65fea3b9d4c07d (127.1.135.195). New cstate: current_term: 1 leader_uuid: "f36bf50226164a81aa65fea3b9d4c07d" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "07273d7511d54b88b18936d1c8590b03" member_type: VOTER last_known_addr { host: "127.1.135.193" port: 33215 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "f36bf50226164a81aa65fea3b9d4c07d" member_type: VOTER last_known_addr { host: "127.1.135.195" port: 40361 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "7a422c36ebff4c2384978dbc5c2b9afc" member_type: VOTER last_known_addr { host: "127.1.135.194" port: 46503 } health_report { overall_health: UNKNOWN } } }
W20260923 22:12:53.825382 9270 tablet.cc:2369] T 46d37003ec784a66bbd9328435345fc3 P 07273d7511d54b88b18936d1c8590b03: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260923 22:12:53.826974 9576 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 22:12:53.824876 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34288 (local address 127.1.135.195:40361)
0923 22:12:53.824935 (+ 59us) server_negotiation.cc:206] Beginning negotiation
0923 22:12:53.824937 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0923 22:12:53.824944 (+ 7us) server_negotiation.cc:407] Connection header received
0923 22:12:53.824986 (+ 42us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 22:12:53.824987 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 22:12:53.825002 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 22:12:53.825060 (+ 58us) server_negotiation.cc:226] Negotiated authn=TOKEN
0923 22:12:53.825156 (+ 96us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 22:12:53.825675 (+ 519us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 22:12:53.826351 (+ 676us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 22:12:53.826532 (+ 181us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 22:12:53.826580 (+ 48us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 22:12:53.826696 (+ 116us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0923 22:12:53.826783 (+ 87us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0923 22:12:53.826840 (+ 57us) server_negotiation.cc:1035] Waiting for connection context
0923 22:12:53.826877 (+ 37us) server_negotiation.cc:299] Negotiation successful
0923 22:12:53.826892 (+ 15us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":40}
I20260923 22:12:53.828662 9223 raft_consensus.cc:1261] T 46d37003ec784a66bbd9328435345fc3 P 07273d7511d54b88b18936d1c8590b03 [term 1 FOLLOWER]: Refusing update from remote peer f36bf50226164a81aa65fea3b9d4c07d: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260923 22:12:53.828692 9363 raft_consensus.cc:1261] T 46d37003ec784a66bbd9328435345fc3 P 7a422c36ebff4c2384978dbc5c2b9afc [term 1 FOLLOWER]: Refusing update from remote peer f36bf50226164a81aa65fea3b9d4c07d: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260923 22:12:53.828959 9585 consensus_queue.cc:1035] T 46d37003ec784a66bbd9328435345fc3 P f36bf50226164a81aa65fea3b9d4c07d [LEADER]: Connected to new peer: Peer: permanent_uuid: "07273d7511d54b88b18936d1c8590b03" member_type: VOTER last_known_addr { host: "127.1.135.193" port: 33215 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260923 22:12:53.829080 9587 consensus_queue.cc:1035] T 46d37003ec784a66bbd9328435345fc3 P f36bf50226164a81aa65fea3b9d4c07d [LEADER]: Connected to new peer: Peer: permanent_uuid: "7a422c36ebff4c2384978dbc5c2b9afc" member_type: VOTER last_known_addr { host: "127.1.135.194" port: 46503 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260923 22:12:53.831410 9591 mvcc.cc:205] Tried to move back new op lower bound from 7332665646400004096 to 7332665646368223232. Current Snapshot: MvccSnapshot[applied={T|T < 7332665646400004096}]
I20260923 22:12:53.832212 9592 mvcc.cc:205] Tried to move back new op lower bound from 7332665646400004096 to 7332665646368223232. Current Snapshot: MvccSnapshot[applied={T|T < 7332665646400004096 or (T in {7332665646400004096})}]
I20260923 22:12:53.833360 9066 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:48896:
table { table_name: "test-table" } modify_external_catalogs: true
I20260923 22:12:53.833472 9066 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:48896:
table { table_name: "test-table" } modify_external_catalogs: true
I20260923 22:12:53.834215 9066 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 53d3bf4260ad47b69a41891f4fa3a6e5: Sending DeleteTablet for 3 replicas of tablet 46d37003ec784a66bbd9328435345fc3
I20260923 22:12:53.834545 9343 tablet_service.cc:1514] Processing DeleteTablet for tablet 46d37003ec784a66bbd9328435345fc3 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-23 22:12:53 UTC) from {username='oryx', principal='oryx/127.1.135.254@KRBTEST.COM'} at 127.0.0.1:32868
I20260923 22:12:53.834609 9199 tablet_service.cc:1514] Processing DeleteTablet for tablet 46d37003ec784a66bbd9328435345fc3 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-23 22:12:53 UTC) from {username='oryx', principal='oryx/127.1.135.254@KRBTEST.COM'} at 127.0.0.1:40636
I20260923 22:12:53.834755 9483 tablet_service.cc:1514] Processing DeleteTablet for tablet 46d37003ec784a66bbd9328435345fc3 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-23 22:12:53 UTC) from {username='oryx', principal='oryx/127.1.135.254@KRBTEST.COM'} at 127.0.0.1:34286
I20260923 22:12:53.834949 9602 tablet_replica.cc:330] T 46d37003ec784a66bbd9328435345fc3 P f36bf50226164a81aa65fea3b9d4c07d: stopping tablet replica
I20260923 22:12:53.834946 9066 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:48896:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260923 22:12:53.835098 9066 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260923 22:12:53.835060 9600 tablet_replica.cc:330] T 46d37003ec784a66bbd9328435345fc3 P 7a422c36ebff4c2384978dbc5c2b9afc: stopping tablet replica
I20260923 22:12:53.835196 9602 raft_consensus.cc:2229] T 46d37003ec784a66bbd9328435345fc3 P f36bf50226164a81aa65fea3b9d4c07d [term 1 LEADER]: Raft consensus shutting down.
I20260923 22:12:53.835323 9600 raft_consensus.cc:2229] T 46d37003ec784a66bbd9328435345fc3 P 7a422c36ebff4c2384978dbc5c2b9afc [term 1 FOLLOWER]: Raft consensus shutting down.
I20260923 22:12:53.835402 9602 raft_consensus.cc:2258] T 46d37003ec784a66bbd9328435345fc3 P f36bf50226164a81aa65fea3b9d4c07d [term 1 FOLLOWER]: Raft consensus is shut down!
I20260923 22:12:53.835505 9601 tablet_replica.cc:330] T 46d37003ec784a66bbd9328435345fc3 P 07273d7511d54b88b18936d1c8590b03: stopping tablet replica
I20260923 22:12:53.835609 9601 raft_consensus.cc:2229] T 46d37003ec784a66bbd9328435345fc3 P 07273d7511d54b88b18936d1c8590b03 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260923 22:12:53.835703 9601 raft_consensus.cc:2258] T 46d37003ec784a66bbd9328435345fc3 P 07273d7511d54b88b18936d1c8590b03 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260923 22:12:53.836144 9601 ts_tablet_manager.cc:1905] T 46d37003ec784a66bbd9328435345fc3 P 07273d7511d54b88b18936d1c8590b03: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260923 22:12:53.836665 9343 tablet_service.cc:1467] Processing CreateTablet for tablet c8cfb50a86ab4f0f8e3c93b2df2f736b (DEFAULT_TABLE table=test-table [id=8565656fe551407791ed4c1c252e19ae]), partition=RANGE (key) PARTITION UNBOUNDED
I20260923 22:12:53.836709 9199 tablet_service.cc:1467] Processing CreateTablet for tablet c8cfb50a86ab4f0f8e3c93b2df2f736b (DEFAULT_TABLE table=test-table [id=8565656fe551407791ed4c1c252e19ae]), partition=RANGE (key) PARTITION UNBOUNDED
I20260923 22:12:53.836710 9483 tablet_service.cc:1467] Processing CreateTablet for tablet c8cfb50a86ab4f0f8e3c93b2df2f736b (DEFAULT_TABLE table=test-table [id=8565656fe551407791ed4c1c252e19ae]), partition=RANGE (key) PARTITION UNBOUNDED
I20260923 22:12:53.836782 9343 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet c8cfb50a86ab4f0f8e3c93b2df2f736b. 1 dirs total, 0 dirs full, 0 dirs failed
I20260923 22:12:53.836807 9199 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet c8cfb50a86ab4f0f8e3c93b2df2f736b. 1 dirs total, 0 dirs full, 0 dirs failed
I20260923 22:12:53.836807 9483 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet c8cfb50a86ab4f0f8e3c93b2df2f736b. 1 dirs total, 0 dirs full, 0 dirs failed
I20260923 22:12:53.838084 9579 tablet_bootstrap.cc:492] T c8cfb50a86ab4f0f8e3c93b2df2f736b P 07273d7511d54b88b18936d1c8590b03: Bootstrap starting.
I20260923 22:12:53.838099 9578 tablet_bootstrap.cc:492] T c8cfb50a86ab4f0f8e3c93b2df2f736b P f36bf50226164a81aa65fea3b9d4c07d: Bootstrap starting.
I20260923 22:12:53.838460 9579 tablet_bootstrap.cc:654] T c8cfb50a86ab4f0f8e3c93b2df2f736b P 07273d7511d54b88b18936d1c8590b03: Neither blocks nor log segments found. Creating new log.
I20260923 22:12:53.838532 9578 tablet_bootstrap.cc:654] T c8cfb50a86ab4f0f8e3c93b2df2f736b P f36bf50226164a81aa65fea3b9d4c07d: Neither blocks nor log segments found. Creating new log.
I20260923 22:12:53.838838 9602 ts_tablet_manager.cc:1905] T 46d37003ec784a66bbd9328435345fc3 P f36bf50226164a81aa65fea3b9d4c07d: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260923 22:12:53.838863 9577 tablet_bootstrap.cc:492] T c8cfb50a86ab4f0f8e3c93b2df2f736b P 7a422c36ebff4c2384978dbc5c2b9afc: Bootstrap starting.
I20260923 22:12:53.839123 9579 tablet_bootstrap.cc:492] T c8cfb50a86ab4f0f8e3c93b2df2f736b P 07273d7511d54b88b18936d1c8590b03: No bootstrap required, opened a new log
I20260923 22:12:53.839176 9579 ts_tablet_manager.cc:1397] T c8cfb50a86ab4f0f8e3c93b2df2f736b P 07273d7511d54b88b18936d1c8590b03: Time spent bootstrapping tablet: real 0.001s user 0.001s sys 0.000s
I20260923 22:12:53.839419 9577 tablet_bootstrap.cc:654] T c8cfb50a86ab4f0f8e3c93b2df2f736b P 7a422c36ebff4c2384978dbc5c2b9afc: Neither blocks nor log segments found. Creating new log.
I20260923 22:12:53.839579 9579 raft_consensus.cc:348] T c8cfb50a86ab4f0f8e3c93b2df2f736b P 07273d7511d54b88b18936d1c8590b03 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "07273d7511d54b88b18936d1c8590b03" member_type: VOTER last_known_addr { host: "127.1.135.193" port: 33215 } } peers { permanent_uuid: "7a422c36ebff4c2384978dbc5c2b9afc" member_type: VOTER last_known_addr { host: "127.1.135.194" port: 46503 } } peers { permanent_uuid: "f36bf50226164a81aa65fea3b9d4c07d" member_type: VOTER last_known_addr { host: "127.1.135.195" port: 40361 } }
I20260923 22:12:53.839663 9579 raft_consensus.cc:374] T c8cfb50a86ab4f0f8e3c93b2df2f736b P 07273d7511d54b88b18936d1c8590b03 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260923 22:12:53.839692 9579 raft_consensus.cc:729] T c8cfb50a86ab4f0f8e3c93b2df2f736b P 07273d7511d54b88b18936d1c8590b03 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 07273d7511d54b88b18936d1c8590b03, State: Initialized, Role: FOLLOWER
I20260923 22:12:53.839753 9579 consensus_queue.cc:260] T c8cfb50a86ab4f0f8e3c93b2df2f736b P 07273d7511d54b88b18936d1c8590b03 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "07273d7511d54b88b18936d1c8590b03" member_type: VOTER last_known_addr { host: "127.1.135.193" port: 33215 } } peers { permanent_uuid: "7a422c36ebff4c2384978dbc5c2b9afc" member_type: VOTER last_known_addr { host: "127.1.135.194" port: 46503 } } peers { permanent_uuid: "f36bf50226164a81aa65fea3b9d4c07d" member_type: VOTER last_known_addr { host: "127.1.135.195" port: 40361 } }
I20260923 22:12:53.838405 9600 raft_consensus.cc:2258] T 46d37003ec784a66bbd9328435345fc3 P 7a422c36ebff4c2384978dbc5c2b9afc [term 1 FOLLOWER]: Raft consensus is shut down!
I20260923 22:12:53.839869 9579 ts_tablet_manager.cc:1428] T c8cfb50a86ab4f0f8e3c93b2df2f736b P 07273d7511d54b88b18936d1c8590b03: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20260923 22:12:53.840169 9601 ts_tablet_manager.cc:1918] T 46d37003ec784a66bbd9328435345fc3 P 07273d7511d54b88b18936d1c8590b03: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260923 22:12:53.840247 9601 log.cc:1196] T 46d37003ec784a66bbd9328435345fc3 P 07273d7511d54b88b18936d1c8590b03: Deleting WAL directory at /tmp/dist-test-taskRiQe_a/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790201539513534-1567-0/minicluster-data/ts-0/wal/wals/46d37003ec784a66bbd9328435345fc3
I20260923 22:12:53.840508 9601 ts_tablet_manager.cc:1939] T 46d37003ec784a66bbd9328435345fc3 P 07273d7511d54b88b18936d1c8590b03: Deleting consensus metadata
I20260923 22:12:53.840492 9600 ts_tablet_manager.cc:1905] T 46d37003ec784a66bbd9328435345fc3 P 7a422c36ebff4c2384978dbc5c2b9afc: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260923 22:12:53.840931 9051 catalog_manager.cc:4841] TS 07273d7511d54b88b18936d1c8590b03 (127.1.135.193:33215): tablet 46d37003ec784a66bbd9328435345fc3 (table test-table [id=52213bf300ff47099f8c6b0ffb194129]) successfully deleted
I20260923 22:12:53.841668 9602 ts_tablet_manager.cc:1918] T 46d37003ec784a66bbd9328435345fc3 P f36bf50226164a81aa65fea3b9d4c07d: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260923 22:12:53.841741 9602 log.cc:1196] T 46d37003ec784a66bbd9328435345fc3 P f36bf50226164a81aa65fea3b9d4c07d: Deleting WAL directory at /tmp/dist-test-taskRiQe_a/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790201539513534-1567-0/minicluster-data/ts-2/wal/wals/46d37003ec784a66bbd9328435345fc3
I20260923 22:12:53.841898 9600 ts_tablet_manager.cc:1918] T 46d37003ec784a66bbd9328435345fc3 P 7a422c36ebff4c2384978dbc5c2b9afc: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260923 22:12:53.841993 9600 log.cc:1196] T 46d37003ec784a66bbd9328435345fc3 P 7a422c36ebff4c2384978dbc5c2b9afc: Deleting WAL directory at /tmp/dist-test-taskRiQe_a/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790201539513534-1567-0/minicluster-data/ts-1/wal/wals/46d37003ec784a66bbd9328435345fc3
I20260923 22:12:53.842208 9602 ts_tablet_manager.cc:1939] T 46d37003ec784a66bbd9328435345fc3 P f36bf50226164a81aa65fea3b9d4c07d: Deleting consensus metadata
I20260923 22:12:53.842234 9600 ts_tablet_manager.cc:1939] T 46d37003ec784a66bbd9328435345fc3 P 7a422c36ebff4c2384978dbc5c2b9afc: Deleting consensus metadata
I20260923 22:12:53.842579 9052 catalog_manager.cc:4841] TS 7a422c36ebff4c2384978dbc5c2b9afc (127.1.135.194:46503): tablet 46d37003ec784a66bbd9328435345fc3 (table test-table [id=52213bf300ff47099f8c6b0ffb194129]) successfully deleted
I20260923 22:12:53.842773 9051 catalog_manager.cc:4841] TS f36bf50226164a81aa65fea3b9d4c07d (127.1.135.195:40361): tablet 46d37003ec784a66bbd9328435345fc3 (table test-table [id=52213bf300ff47099f8c6b0ffb194129]) successfully deleted
I20260923 22:12:53.843075 9577 tablet_bootstrap.cc:492] T c8cfb50a86ab4f0f8e3c93b2df2f736b P 7a422c36ebff4c2384978dbc5c2b9afc: No bootstrap required, opened a new log
I20260923 22:12:53.843137 9577 ts_tablet_manager.cc:1397] T c8cfb50a86ab4f0f8e3c93b2df2f736b P 7a422c36ebff4c2384978dbc5c2b9afc: Time spent bootstrapping tablet: real 0.004s user 0.001s sys 0.000s
I20260923 22:12:53.843281 9577 raft_consensus.cc:348] T c8cfb50a86ab4f0f8e3c93b2df2f736b P 7a422c36ebff4c2384978dbc5c2b9afc [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "07273d7511d54b88b18936d1c8590b03" member_type: VOTER last_known_addr { host: "127.1.135.193" port: 33215 } } peers { permanent_uuid: "7a422c36ebff4c2384978dbc5c2b9afc" member_type: VOTER last_known_addr { host: "127.1.135.194" port: 46503 } } peers { permanent_uuid: "f36bf50226164a81aa65fea3b9d4c07d" member_type: VOTER last_known_addr { host: "127.1.135.195" port: 40361 } }
I20260923 22:12:53.843350 9577 raft_consensus.cc:374] T c8cfb50a86ab4f0f8e3c93b2df2f736b P 7a422c36ebff4c2384978dbc5c2b9afc [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260923 22:12:53.843382 9577 raft_consensus.cc:729] T c8cfb50a86ab4f0f8e3c93b2df2f736b P 7a422c36ebff4c2384978dbc5c2b9afc [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 7a422c36ebff4c2384978dbc5c2b9afc, State: Initialized, Role: FOLLOWER
I20260923 22:12:53.843430 9577 consensus_queue.cc:260] T c8cfb50a86ab4f0f8e3c93b2df2f736b P 7a422c36ebff4c2384978dbc5c2b9afc [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "07273d7511d54b88b18936d1c8590b03" member_type: VOTER last_known_addr { host: "127.1.135.193" port: 33215 } } peers { permanent_uuid: "7a422c36ebff4c2384978dbc5c2b9afc" member_type: VOTER last_known_addr { host: "127.1.135.194" port: 46503 } } peers { permanent_uuid: "f36bf50226164a81aa65fea3b9d4c07d" member_type: VOTER last_known_addr { host: "127.1.135.195" port: 40361 } }
I20260923 22:12:53.843531 9577 ts_tablet_manager.cc:1428] T c8cfb50a86ab4f0f8e3c93b2df2f736b P 7a422c36ebff4c2384978dbc5c2b9afc: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20260923 22:12:53.844457 9578 tablet_bootstrap.cc:492] T c8cfb50a86ab4f0f8e3c93b2df2f736b P f36bf50226164a81aa65fea3b9d4c07d: No bootstrap required, opened a new log
I20260923 22:12:53.844501 9578 ts_tablet_manager.cc:1397] T c8cfb50a86ab4f0f8e3c93b2df2f736b P f36bf50226164a81aa65fea3b9d4c07d: Time spent bootstrapping tablet: real 0.006s user 0.001s sys 0.000s
I20260923 22:12:53.844638 9578 raft_consensus.cc:348] T c8cfb50a86ab4f0f8e3c93b2df2f736b P f36bf50226164a81aa65fea3b9d4c07d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "07273d7511d54b88b18936d1c8590b03" member_type: VOTER last_known_addr { host: "127.1.135.193" port: 33215 } } peers { permanent_uuid: "7a422c36ebff4c2384978dbc5c2b9afc" member_type: VOTER last_known_addr { host: "127.1.135.194" port: 46503 } } peers { permanent_uuid: "f36bf50226164a81aa65fea3b9d4c07d" member_type: VOTER last_known_addr { host: "127.1.135.195" port: 40361 } }
I20260923 22:12:53.844712 9578 raft_consensus.cc:374] T c8cfb50a86ab4f0f8e3c93b2df2f736b P f36bf50226164a81aa65fea3b9d4c07d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260923 22:12:53.844725 9578 raft_consensus.cc:729] T c8cfb50a86ab4f0f8e3c93b2df2f736b P f36bf50226164a81aa65fea3b9d4c07d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: f36bf50226164a81aa65fea3b9d4c07d, State: Initialized, Role: FOLLOWER
I20260923 22:12:53.844775 9578 consensus_queue.cc:260] T c8cfb50a86ab4f0f8e3c93b2df2f736b P f36bf50226164a81aa65fea3b9d4c07d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "07273d7511d54b88b18936d1c8590b03" member_type: VOTER last_known_addr { host: "127.1.135.193" port: 33215 } } peers { permanent_uuid: "7a422c36ebff4c2384978dbc5c2b9afc" member_type: VOTER last_known_addr { host: "127.1.135.194" port: 46503 } } peers { permanent_uuid: "f36bf50226164a81aa65fea3b9d4c07d" member_type: VOTER last_known_addr { host: "127.1.135.195" port: 40361 } }
I20260923 22:12:53.844866 9578 ts_tablet_manager.cc:1428] T c8cfb50a86ab4f0f8e3c93b2df2f736b P f36bf50226164a81aa65fea3b9d4c07d: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20260923 22:12:53.907003 9583 raft_consensus.cc:482] T c8cfb50a86ab4f0f8e3c93b2df2f736b P 07273d7511d54b88b18936d1c8590b03 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260923 22:12:53.907133 9583 raft_consensus.cc:504] T c8cfb50a86ab4f0f8e3c93b2df2f736b P 07273d7511d54b88b18936d1c8590b03 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "07273d7511d54b88b18936d1c8590b03" member_type: VOTER last_known_addr { host: "127.1.135.193" port: 33215 } } peers { permanent_uuid: "7a422c36ebff4c2384978dbc5c2b9afc" member_type: VOTER last_known_addr { host: "127.1.135.194" port: 46503 } } peers { permanent_uuid: "f36bf50226164a81aa65fea3b9d4c07d" member_type: VOTER last_known_addr { host: "127.1.135.195" port: 40361 } }
I20260923 22:12:53.907491 9583 leader_election.cc:288] T c8cfb50a86ab4f0f8e3c93b2df2f736b P 07273d7511d54b88b18936d1c8590b03 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 7a422c36ebff4c2384978dbc5c2b9afc (127.1.135.194:46503), f36bf50226164a81aa65fea3b9d4c07d (127.1.135.195:40361)
I20260923 22:12:53.911041 9574 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 22:12:53.907898 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.135.193:60087 (local address 127.1.135.194:46503)
0923 22:12:53.907937 (+ 39us) server_negotiation.cc:206] Beginning negotiation
0923 22:12:53.907939 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0923 22:12:53.907947 (+ 8us) server_negotiation.cc:407] Connection header received
0923 22:12:53.907975 (+ 28us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 22:12:53.907976 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 22:12:53.907986 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 22:12:53.908102 (+ 116us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0923 22:12:53.908287 (+ 185us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 22:12:53.909064 (+ 777us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 22:12:53.910190 (+ 1126us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 22:12:53.910645 (+ 455us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 22:12:53.910739 (+ 94us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 22:12:53.910749 (+ 10us) server_negotiation.cc:1035] Waiting for connection context
0923 22:12:53.910915 (+ 166us) server_negotiation.cc:299] Negotiation successful
0923 22:12:53.910970 (+ 55us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":21}
I20260923 22:12:53.911083 9606 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 22:12:53.907681 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.135.194:46503 (local address 127.1.135.193:60087)
0923 22:12:53.907806 (+ 125us) negotiation.cc:107] Waiting for socket to connect
0923 22:12:53.907818 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0923 22:12:53.907871 (+ 53us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0923 22:12:53.908120 (+ 249us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0923 22:12:53.908121 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0923 22:12:53.908122 (+ 1us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0923 22:12:53.908207 (+ 85us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 22:12:53.908208 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 22:12:53.909181 (+ 973us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 22:12:53.909181 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 22:12:53.910101 (+ 920us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 22:12:53.910103 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 22:12:53.910753 (+ 650us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 22:12:53.910755 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 22:12:53.910832 (+ 77us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 22:12:53.910837 (+ 5us) client_negotiation.cc:769] Sending connection context
0923 22:12:53.910901 (+ 64us) client_negotiation.cc:240] Negotiation successful
0923 22:12:53.910980 (+ 79us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":107,"mutex_wait_us":31,"thread_start_us":60,"threads_started":1}
I20260923 22:12:53.911512 9363 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "c8cfb50a86ab4f0f8e3c93b2df2f736b" candidate_uuid: "07273d7511d54b88b18936d1c8590b03" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7a422c36ebff4c2384978dbc5c2b9afc" is_pre_election: true
I20260923 22:12:53.911612 9363 raft_consensus.cc:2454] T c8cfb50a86ab4f0f8e3c93b2df2f736b P 7a422c36ebff4c2384978dbc5c2b9afc [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 07273d7511d54b88b18936d1c8590b03 in term 0.
I20260923 22:12:53.911708 9595 raft_consensus.cc:482] T c8cfb50a86ab4f0f8e3c93b2df2f736b P f36bf50226164a81aa65fea3b9d4c07d [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260923 22:12:53.911769 9595 raft_consensus.cc:504] T c8cfb50a86ab4f0f8e3c93b2df2f736b P f36bf50226164a81aa65fea3b9d4c07d [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "07273d7511d54b88b18936d1c8590b03" member_type: VOTER last_known_addr { host: "127.1.135.193" port: 33215 } } peers { permanent_uuid: "7a422c36ebff4c2384978dbc5c2b9afc" member_type: VOTER last_known_addr { host: "127.1.135.194" port: 46503 } } peers { permanent_uuid: "f36bf50226164a81aa65fea3b9d4c07d" member_type: VOTER last_known_addr { host: "127.1.135.195" port: 40361 } }
I20260923 22:12:53.911902 9607 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 22:12:53.907681 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.135.195:40361 (local address 127.1.135.193:35533)
0923 22:12:53.907873 (+ 192us) negotiation.cc:107] Waiting for socket to connect
0923 22:12:53.907882 (+ 9us) client_negotiation.cc:174] Beginning negotiation
0923 22:12:53.907952 (+ 70us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0923 22:12:53.908165 (+ 213us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0923 22:12:53.908166 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0923 22:12:53.908169 (+ 3us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0923 22:12:53.908249 (+ 80us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 22:12:53.908251 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 22:12:53.909165 (+ 914us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 22:12:53.909166 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 22:12:53.910466 (+ 1300us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 22:12:53.910469 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 22:12:53.911530 (+ 1061us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 22:12:53.911531 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 22:12:53.911601 (+ 70us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 22:12:53.911605 (+ 4us) client_negotiation.cc:769] Sending connection context
0923 22:12:53.911787 (+ 182us) client_negotiation.cc:240] Negotiation successful
0923 22:12:53.911826 (+ 39us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":182,"mutex_wait_us":50,"thread_start_us":59,"threads_started":1}
I20260923 22:12:53.911962 9595 leader_election.cc:288] T c8cfb50a86ab4f0f8e3c93b2df2f736b P f36bf50226164a81aa65fea3b9d4c07d [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 07273d7511d54b88b18936d1c8590b03 (127.1.135.193:33215), 7a422c36ebff4c2384978dbc5c2b9afc (127.1.135.194:46503)
I20260923 22:12:53.912144 9223 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "c8cfb50a86ab4f0f8e3c93b2df2f736b" candidate_uuid: "f36bf50226164a81aa65fea3b9d4c07d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "07273d7511d54b88b18936d1c8590b03" is_pre_election: true
I20260923 22:12:53.912180 9363 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "c8cfb50a86ab4f0f8e3c93b2df2f736b" candidate_uuid: "f36bf50226164a81aa65fea3b9d4c07d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7a422c36ebff4c2384978dbc5c2b9afc" is_pre_election: true
I20260923 22:12:53.912235 9223 raft_consensus.cc:2454] T c8cfb50a86ab4f0f8e3c93b2df2f736b P 07273d7511d54b88b18936d1c8590b03 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f36bf50226164a81aa65fea3b9d4c07d in term 0.
I20260923 22:12:53.912245 9363 raft_consensus.cc:2454] T c8cfb50a86ab4f0f8e3c93b2df2f736b P 7a422c36ebff4c2384978dbc5c2b9afc [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f36bf50226164a81aa65fea3b9d4c07d in term 0.
I20260923 22:12:53.912314 9576 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 22:12:53.907808 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.135.193:35533 (local address 127.1.135.195:40361)
0923 22:12:53.907851 (+ 43us) server_negotiation.cc:206] Beginning negotiation
0923 22:12:53.907853 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0923 22:12:53.907929 (+ 76us) server_negotiation.cc:407] Connection header received
0923 22:12:53.908029 (+ 100us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 22:12:53.908030 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 22:12:53.908041 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 22:12:53.908121 (+ 80us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0923 22:12:53.908371 (+ 250us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 22:12:53.909075 (+ 704us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 22:12:53.910563 (+ 1488us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 22:12:53.911126 (+ 563us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 22:12:53.911201 (+ 75us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 22:12:53.911210 (+ 9us) server_negotiation.cc:1035] Waiting for connection context
0923 22:12:53.911699 (+ 489us) server_negotiation.cc:299] Negotiation successful
0923 22:12:53.911738 (+ 39us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":22}
I20260923 22:12:53.912343 9156 leader_election.cc:302] T c8cfb50a86ab4f0f8e3c93b2df2f736b P 07273d7511d54b88b18936d1c8590b03 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 07273d7511d54b88b18936d1c8590b03, 7a422c36ebff4c2384978dbc5c2b9afc; no voters:
I20260923 22:12:53.912384 9436 leader_election.cc:302] T c8cfb50a86ab4f0f8e3c93b2df2f736b P f36bf50226164a81aa65fea3b9d4c07d [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 7a422c36ebff4c2384978dbc5c2b9afc, f36bf50226164a81aa65fea3b9d4c07d; no voters:
I20260923 22:12:53.912447 9583 raft_consensus.cc:2781] T c8cfb50a86ab4f0f8e3c93b2df2f736b P 07273d7511d54b88b18936d1c8590b03 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260923 22:12:53.912459 9595 raft_consensus.cc:2781] T c8cfb50a86ab4f0f8e3c93b2df2f736b P f36bf50226164a81aa65fea3b9d4c07d [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260923 22:12:53.912479 9583 raft_consensus.cc:482] T c8cfb50a86ab4f0f8e3c93b2df2f736b P 07273d7511d54b88b18936d1c8590b03 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260923 22:12:53.912485 9595 raft_consensus.cc:482] T c8cfb50a86ab4f0f8e3c93b2df2f736b P f36bf50226164a81aa65fea3b9d4c07d [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260923 22:12:53.912503 9583 raft_consensus.cc:3037] T c8cfb50a86ab4f0f8e3c93b2df2f736b P 07273d7511d54b88b18936d1c8590b03 [term 0 FOLLOWER]: Advancing to term 1
I20260923 22:12:53.912508 9595 raft_consensus.cc:3037] T c8cfb50a86ab4f0f8e3c93b2df2f736b P f36bf50226164a81aa65fea3b9d4c07d [term 0 FOLLOWER]: Advancing to term 1
I20260923 22:12:53.912511 9503 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "c8cfb50a86ab4f0f8e3c93b2df2f736b" candidate_uuid: "07273d7511d54b88b18936d1c8590b03" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f36bf50226164a81aa65fea3b9d4c07d" is_pre_election: true
W20260923 22:12:53.912511 9550 tablet.cc:2369] T c8cfb50a86ab4f0f8e3c93b2df2f736b P f36bf50226164a81aa65fea3b9d4c07d: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260923 22:12:53.913295 9583 raft_consensus.cc:504] T c8cfb50a86ab4f0f8e3c93b2df2f736b P 07273d7511d54b88b18936d1c8590b03 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "07273d7511d54b88b18936d1c8590b03" member_type: VOTER last_known_addr { host: "127.1.135.193" port: 33215 } } peers { permanent_uuid: "7a422c36ebff4c2384978dbc5c2b9afc" member_type: VOTER last_known_addr { host: "127.1.135.194" port: 46503 } } peers { permanent_uuid: "f36bf50226164a81aa65fea3b9d4c07d" member_type: VOTER last_known_addr { host: "127.1.135.195" port: 40361 } }
I20260923 22:12:53.913300 9595 raft_consensus.cc:504] T c8cfb50a86ab4f0f8e3c93b2df2f736b P f36bf50226164a81aa65fea3b9d4c07d [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "07273d7511d54b88b18936d1c8590b03" member_type: VOTER last_known_addr { host: "127.1.135.193" port: 33215 } } peers { permanent_uuid: "7a422c36ebff4c2384978dbc5c2b9afc" member_type: VOTER last_known_addr { host: "127.1.135.194" port: 46503 } } peers { permanent_uuid: "f36bf50226164a81aa65fea3b9d4c07d" member_type: VOTER last_known_addr { host: "127.1.135.195" port: 40361 } }
I20260923 22:12:53.913412 9583 leader_election.cc:288] T c8cfb50a86ab4f0f8e3c93b2df2f736b P 07273d7511d54b88b18936d1c8590b03 [CANDIDATE]: Term 1 election: Requested vote from peers 7a422c36ebff4c2384978dbc5c2b9afc (127.1.135.194:46503), f36bf50226164a81aa65fea3b9d4c07d (127.1.135.195:40361)
I20260923 22:12:53.913431 9595 leader_election.cc:288] T c8cfb50a86ab4f0f8e3c93b2df2f736b P f36bf50226164a81aa65fea3b9d4c07d [CANDIDATE]: Term 1 election: Requested vote from peers 07273d7511d54b88b18936d1c8590b03 (127.1.135.193:33215), 7a422c36ebff4c2384978dbc5c2b9afc (127.1.135.194:46503)
I20260923 22:12:53.913610 9363 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "c8cfb50a86ab4f0f8e3c93b2df2f736b" candidate_uuid: "07273d7511d54b88b18936d1c8590b03" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7a422c36ebff4c2384978dbc5c2b9afc"
I20260923 22:12:53.913647 9503 raft_consensus.cc:2379] T c8cfb50a86ab4f0f8e3c93b2df2f736b P f36bf50226164a81aa65fea3b9d4c07d [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 07273d7511d54b88b18936d1c8590b03 in current term 1: Already voted for candidate f36bf50226164a81aa65fea3b9d4c07d in this term.
I20260923 22:12:53.913631 9223 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "c8cfb50a86ab4f0f8e3c93b2df2f736b" candidate_uuid: "f36bf50226164a81aa65fea3b9d4c07d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "07273d7511d54b88b18936d1c8590b03"
I20260923 22:12:53.913609 9502 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "c8cfb50a86ab4f0f8e3c93b2df2f736b" candidate_uuid: "07273d7511d54b88b18936d1c8590b03" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f36bf50226164a81aa65fea3b9d4c07d"
I20260923 22:12:53.913671 9363 raft_consensus.cc:3037] T c8cfb50a86ab4f0f8e3c93b2df2f736b P 7a422c36ebff4c2384978dbc5c2b9afc [term 0 FOLLOWER]: Advancing to term 1
I20260923 22:12:53.913695 9223 raft_consensus.cc:2379] T c8cfb50a86ab4f0f8e3c93b2df2f736b P 07273d7511d54b88b18936d1c8590b03 [term 1 FOLLOWER]: Leader election vote request: Denying vote to candidate f36bf50226164a81aa65fea3b9d4c07d in current term 1: Already voted for candidate 07273d7511d54b88b18936d1c8590b03 in this term.
I20260923 22:12:53.914086 9362 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "c8cfb50a86ab4f0f8e3c93b2df2f736b" candidate_uuid: "f36bf50226164a81aa65fea3b9d4c07d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7a422c36ebff4c2384978dbc5c2b9afc"
I20260923 22:12:53.914302 9436 leader_election.cc:302] T c8cfb50a86ab4f0f8e3c93b2df2f736b P f36bf50226164a81aa65fea3b9d4c07d [CANDIDATE]: Term 1 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f36bf50226164a81aa65fea3b9d4c07d; no voters: 07273d7511d54b88b18936d1c8590b03, 7a422c36ebff4c2384978dbc5c2b9afc
I20260923 22:12:53.914412 9595 raft_consensus.cc:2726] T c8cfb50a86ab4f0f8e3c93b2df2f736b P f36bf50226164a81aa65fea3b9d4c07d [term 1 FOLLOWER]: Leader election lost for term 1. Reason: could not achieve majority
I20260923 22:12:53.914451 9363 raft_consensus.cc:2454] T c8cfb50a86ab4f0f8e3c93b2df2f736b P 7a422c36ebff4c2384978dbc5c2b9afc [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 07273d7511d54b88b18936d1c8590b03 in term 1.
I20260923 22:12:53.914604 9156 leader_election.cc:302] T c8cfb50a86ab4f0f8e3c93b2df2f736b P 07273d7511d54b88b18936d1c8590b03 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 07273d7511d54b88b18936d1c8590b03, 7a422c36ebff4c2384978dbc5c2b9afc; no voters: f36bf50226164a81aa65fea3b9d4c07d
I20260923 22:12:53.914731 9583 raft_consensus.cc:2781] T c8cfb50a86ab4f0f8e3c93b2df2f736b P 07273d7511d54b88b18936d1c8590b03 [term 1 FOLLOWER]: Leader election won for term 1
I20260923 22:12:53.914860 9583 raft_consensus.cc:686] T c8cfb50a86ab4f0f8e3c93b2df2f736b P 07273d7511d54b88b18936d1c8590b03 [term 1 LEADER]: Becoming Leader. State: Replica: 07273d7511d54b88b18936d1c8590b03, State: Running, Role: LEADER
I20260923 22:12:53.914928 9583 consensus_queue.cc:237] T c8cfb50a86ab4f0f8e3c93b2df2f736b P 07273d7511d54b88b18936d1c8590b03 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "07273d7511d54b88b18936d1c8590b03" member_type: VOTER last_known_addr { host: "127.1.135.193" port: 33215 } } peers { permanent_uuid: "7a422c36ebff4c2384978dbc5c2b9afc" member_type: VOTER last_known_addr { host: "127.1.135.194" port: 46503 } } peers { permanent_uuid: "f36bf50226164a81aa65fea3b9d4c07d" member_type: VOTER last_known_addr { host: "127.1.135.195" port: 40361 } }
I20260923 22:12:53.915623 9066 catalog_manager.cc:5495] T c8cfb50a86ab4f0f8e3c93b2df2f736b P 07273d7511d54b88b18936d1c8590b03 reported cstate change: term changed from 0 to 1, leader changed from <none> to 07273d7511d54b88b18936d1c8590b03 (127.1.135.193). New cstate: current_term: 1 leader_uuid: "07273d7511d54b88b18936d1c8590b03" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "07273d7511d54b88b18936d1c8590b03" member_type: VOTER last_known_addr { host: "127.1.135.193" port: 33215 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "7a422c36ebff4c2384978dbc5c2b9afc" member_type: VOTER last_known_addr { host: "127.1.135.194" port: 46503 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "f36bf50226164a81aa65fea3b9d4c07d" member_type: VOTER last_known_addr { host: "127.1.135.195" port: 40361 } health_report { overall_health: UNKNOWN } } }
Sep 23 22:12:53 dist-test-slave-s2fj krb5kdc[9021](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790201573, etypes {rep=17 tkt=17 ses=17}, oryx/127.1.135.254@KRBTEST.COM for oryx/127.1.135.254@KRBTEST.COM
I20260923 22:12:53.959302 9115 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 22:12:53.952951 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48900 (local address 127.1.135.254:35183)
0923 22:12:53.953004 (+ 53us) server_negotiation.cc:206] Beginning negotiation
0923 22:12:53.953006 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0923 22:12:53.953109 (+ 103us) server_negotiation.cc:407] Connection header received
0923 22:12:53.953173 (+ 64us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 22:12:53.953174 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 22:12:53.953183 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 22:12:53.953226 (+ 43us) server_negotiation.cc:226] Negotiated authn=SASL
0923 22:12:53.953673 (+ 447us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 22:12:53.954149 (+ 476us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 22:12:53.954821 (+ 672us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 22:12:53.954990 (+ 169us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 22:12:53.955037 (+ 47us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 22:12:53.957101 (+ 2064us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0923 22:12:53.957103 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0923 22:12:53.957103 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0923 22:12:53.957109 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0923 22:12:53.958274 (+ 1165us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0923 22:12:53.958706 (+ 432us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0923 22:12:53.958707 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0923 22:12:53.958707 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0923 22:12:53.958765 (+ 58us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0923 22:12:53.958919 (+ 154us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0923 22:12:53.958919 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0923 22:12:53.958920 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0923 22:12:53.959013 (+ 93us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0923 22:12:53.959060 (+ 47us) server_negotiation.cc:1035] Waiting for connection context
0923 22:12:53.959172 (+ 112us) server_negotiation.cc:299] Negotiation successful
0923 22:12:53.959236 (+ 64us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":24}
I20260923 22:12:53.959458 9615 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 22:12:53.952877 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.135.254:35183 (local address 127.0.0.1:48900)
0923 22:12:53.953024 (+ 147us) negotiation.cc:107] Waiting for socket to connect
0923 22:12:53.953043 (+ 19us) client_negotiation.cc:174] Beginning negotiation
0923 22:12:53.953107 (+ 64us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0923 22:12:53.953252 (+ 145us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0923 22:12:53.953253 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0923 22:12:53.953547 (+ 294us) client_negotiation.cc:189] Negotiated authn=SASL
0923 22:12:53.953598 (+ 51us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 22:12:53.953600 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 22:12:53.954216 (+ 616us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 22:12:53.954217 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 22:12:53.954748 (+ 531us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 22:12:53.954750 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 22:12:53.955051 (+ 301us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 22:12:53.955052 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 22:12:53.955090 (+ 38us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 22:12:53.955669 (+ 579us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0923 22:12:53.955673 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0923 22:12:53.957010 (+ 1337us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0923 22:12:53.958388 (+ 1378us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0923 22:12:53.958400 (+ 12us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0923 22:12:53.958402 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0923 22:12:53.958638 (+ 236us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0923 22:12:53.958825 (+ 187us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0923 22:12:53.958825 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0923 22:12:53.958825 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0923 22:12:53.958881 (+ 56us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0923 22:12:53.959061 (+ 180us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0923 22:12:53.959066 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0923 22:12:53.959106 (+ 40us) client_negotiation.cc:769] Sending connection context
0923 22:12:53.959156 (+ 50us) client_negotiation.cc:240] Negotiation successful
0923 22:12:53.959304 (+ 148us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":120,"mutex_wait_us":56,"thread_start_us":83,"threads_started":1}
I20260923 22:12:53.961462 9066 catalog_manager.cc:1908] Servicing CreateTable request from {username='oryx', principal='oryx/127.1.135.254@KRBTEST.COM'} at 127.0.0.1:48900:
name: "kudu_system.kudu_transactions"
schema {
columns {
name: "txn_id"
type: INT64
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "entry_type"
type: INT8
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "identifier"
type: STRING
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "metadata"
type: STRING
is_key: false
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
rows: "<redacted>""\006\001\000\000\000\000\000\000\000\000\007\001@B\017\000\000\000\000\000"
indirect_data: "<redacted>"""
}
partition_schema {
range_schema {
columns {
name: "txn_id"
}
}
}
table_type: TXN_STATUS_TABLE
W20260923 22:12:53.961673 9066 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table kudu_system.kudu_transactions in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260923 22:12:53.963017 9483 tablet_service.cc:1467] Processing CreateTablet for tablet 6e95070cd0d6412697bd35caba09c6ee (TXN_STATUS_TABLE table=kudu_system.kudu_transactions [id=3b2c7672728f427db2667c8383489c2d]), partition=RANGE (txn_id) PARTITION 0 <= VALUES < 1000000
I20260923 22:12:53.963023 9343 tablet_service.cc:1467] Processing CreateTablet for tablet 6e95070cd0d6412697bd35caba09c6ee (TXN_STATUS_TABLE table=kudu_system.kudu_transactions [id=3b2c7672728f427db2667c8383489c2d]), partition=RANGE (txn_id) PARTITION 0 <= VALUES < 1000000
I20260923 22:12:53.963174 9483 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 6e95070cd0d6412697bd35caba09c6ee. 1 dirs total, 0 dirs full, 0 dirs failed
I20260923 22:12:53.963178 9343 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 6e95070cd0d6412697bd35caba09c6ee. 1 dirs total, 0 dirs full, 0 dirs failed
I20260923 22:12:53.964319 9578 tablet_bootstrap.cc:492] T 6e95070cd0d6412697bd35caba09c6ee P f36bf50226164a81aa65fea3b9d4c07d: Bootstrap starting.
I20260923 22:12:53.964437 9199 tablet_service.cc:1467] Processing CreateTablet for tablet 6e95070cd0d6412697bd35caba09c6ee (TXN_STATUS_TABLE table=kudu_system.kudu_transactions [id=3b2c7672728f427db2667c8383489c2d]), partition=RANGE (txn_id) PARTITION 0 <= VALUES < 1000000
I20260923 22:12:53.964591 9199 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 6e95070cd0d6412697bd35caba09c6ee. 1 dirs total, 0 dirs full, 0 dirs failed
I20260923 22:12:53.964704 9578 tablet_bootstrap.cc:654] T 6e95070cd0d6412697bd35caba09c6ee P f36bf50226164a81aa65fea3b9d4c07d: Neither blocks nor log segments found. Creating new log.
I20260923 22:12:53.965204 9578 tablet_bootstrap.cc:492] T 6e95070cd0d6412697bd35caba09c6ee P f36bf50226164a81aa65fea3b9d4c07d: No bootstrap required, opened a new log
I20260923 22:12:53.965245 9578 ts_tablet_manager.cc:1397] T 6e95070cd0d6412697bd35caba09c6ee P f36bf50226164a81aa65fea3b9d4c07d: Time spent bootstrapping tablet: real 0.001s user 0.001s sys 0.000s
I20260923 22:12:53.965353 9578 raft_consensus.cc:348] T 6e95070cd0d6412697bd35caba09c6ee P f36bf50226164a81aa65fea3b9d4c07d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "07273d7511d54b88b18936d1c8590b03" member_type: VOTER last_known_addr { host: "127.1.135.193" port: 33215 } } peers { permanent_uuid: "7a422c36ebff4c2384978dbc5c2b9afc" member_type: VOTER last_known_addr { host: "127.1.135.194" port: 46503 } } peers { permanent_uuid: "f36bf50226164a81aa65fea3b9d4c07d" member_type: VOTER last_known_addr { host: "127.1.135.195" port: 40361 } }
I20260923 22:12:53.965425 9578 raft_consensus.cc:374] T 6e95070cd0d6412697bd35caba09c6ee P f36bf50226164a81aa65fea3b9d4c07d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260923 22:12:53.965440 9578 raft_consensus.cc:729] T 6e95070cd0d6412697bd35caba09c6ee P f36bf50226164a81aa65fea3b9d4c07d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: f36bf50226164a81aa65fea3b9d4c07d, State: Initialized, Role: FOLLOWER
I20260923 22:12:53.965492 9578 consensus_queue.cc:260] T 6e95070cd0d6412697bd35caba09c6ee P f36bf50226164a81aa65fea3b9d4c07d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "07273d7511d54b88b18936d1c8590b03" member_type: VOTER last_known_addr { host: "127.1.135.193" port: 33215 } } peers { permanent_uuid: "7a422c36ebff4c2384978dbc5c2b9afc" member_type: VOTER last_known_addr { host: "127.1.135.194" port: 46503 } } peers { permanent_uuid: "f36bf50226164a81aa65fea3b9d4c07d" member_type: VOTER last_known_addr { host: "127.1.135.195" port: 40361 } }
I20260923 22:12:53.965593 9578 ts_tablet_manager.cc:1428] T 6e95070cd0d6412697bd35caba09c6ee P f36bf50226164a81aa65fea3b9d4c07d: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20260923 22:12:53.965610 9595 tablet_replica.cc:439] T 6e95070cd0d6412697bd35caba09c6ee P f36bf50226164a81aa65fea3b9d4c07d: TxnStatusTablet state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "07273d7511d54b88b18936d1c8590b03" member_type: VOTER last_known_addr { host: "127.1.135.193" port: 33215 } } peers { permanent_uuid: "7a422c36ebff4c2384978dbc5c2b9afc" member_type: VOTER last_known_addr { host: "127.1.135.194" port: 46503 } } peers { permanent_uuid: "f36bf50226164a81aa65fea3b9d4c07d" member_type: VOTER last_known_addr { host: "127.1.135.195" port: 40361 } } }
I20260923 22:12:53.965687 9595 tablet_replica.cc:442] T 6e95070cd0d6412697bd35caba09c6ee P f36bf50226164a81aa65fea3b9d4c07d: This TxnStatusTablet replica's current role is: FOLLOWER
I20260923 22:12:53.966086 9579 tablet_bootstrap.cc:492] T 6e95070cd0d6412697bd35caba09c6ee P 07273d7511d54b88b18936d1c8590b03: Bootstrap starting.
I20260923 22:12:53.966125 9577 tablet_bootstrap.cc:492] T 6e95070cd0d6412697bd35caba09c6ee P 7a422c36ebff4c2384978dbc5c2b9afc: Bootstrap starting.
I20260923 22:12:53.966511 9579 tablet_bootstrap.cc:654] T 6e95070cd0d6412697bd35caba09c6ee P 07273d7511d54b88b18936d1c8590b03: Neither blocks nor log segments found. Creating new log.
I20260923 22:12:53.966600 9577 tablet_bootstrap.cc:654] T 6e95070cd0d6412697bd35caba09c6ee P 7a422c36ebff4c2384978dbc5c2b9afc: Neither blocks nor log segments found. Creating new log.
I20260923 22:12:53.967180 9579 tablet_bootstrap.cc:492] T 6e95070cd0d6412697bd35caba09c6ee P 07273d7511d54b88b18936d1c8590b03: No bootstrap required, opened a new log
I20260923 22:12:53.967237 9579 ts_tablet_manager.cc:1397] T 6e95070cd0d6412697bd35caba09c6ee P 07273d7511d54b88b18936d1c8590b03: Time spent bootstrapping tablet: real 0.001s user 0.001s sys 0.000s
I20260923 22:12:53.967377 9579 raft_consensus.cc:348] T 6e95070cd0d6412697bd35caba09c6ee P 07273d7511d54b88b18936d1c8590b03 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "07273d7511d54b88b18936d1c8590b03" member_type: VOTER last_known_addr { host: "127.1.135.193" port: 33215 } } peers { permanent_uuid: "7a422c36ebff4c2384978dbc5c2b9afc" member_type: VOTER last_known_addr { host: "127.1.135.194" port: 46503 } } peers { permanent_uuid: "f36bf50226164a81aa65fea3b9d4c07d" member_type: VOTER last_known_addr { host: "127.1.135.195" port: 40361 } }
I20260923 22:12:53.967454 9579 raft_consensus.cc:374] T 6e95070cd0d6412697bd35caba09c6ee P 07273d7511d54b88b18936d1c8590b03 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260923 22:12:53.967497 9579 raft_consensus.cc:729] T 6e95070cd0d6412697bd35caba09c6ee P 07273d7511d54b88b18936d1c8590b03 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 07273d7511d54b88b18936d1c8590b03, State: Initialized, Role: FOLLOWER
I20260923 22:12:53.967553 9579 consensus_queue.cc:260] T 6e95070cd0d6412697bd35caba09c6ee P 07273d7511d54b88b18936d1c8590b03 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "07273d7511d54b88b18936d1c8590b03" member_type: VOTER last_known_addr { host: "127.1.135.193" port: 33215 } } peers { permanent_uuid: "7a422c36ebff4c2384978dbc5c2b9afc" member_type: VOTER last_known_addr { host: "127.1.135.194" port: 46503 } } peers { permanent_uuid: "f36bf50226164a81aa65fea3b9d4c07d" member_type: VOTER last_known_addr { host: "127.1.135.195" port: 40361 } }
I20260923 22:12:53.967653 9579 ts_tablet_manager.cc:1428] T 6e95070cd0d6412697bd35caba09c6ee P 07273d7511d54b88b18936d1c8590b03: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20260923 22:12:53.967708 9583 tablet_replica.cc:439] T 6e95070cd0d6412697bd35caba09c6ee P 07273d7511d54b88b18936d1c8590b03: TxnStatusTablet state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "07273d7511d54b88b18936d1c8590b03" member_type: VOTER last_known_addr { host: "127.1.135.193" port: 33215 } } peers { permanent_uuid: "7a422c36ebff4c2384978dbc5c2b9afc" member_type: VOTER last_known_addr { host: "127.1.135.194" port: 46503 } } peers { permanent_uuid: "f36bf50226164a81aa65fea3b9d4c07d" member_type: VOTER last_known_addr { host: "127.1.135.195" port: 40361 } } }
I20260923 22:12:53.967785 9583 tablet_replica.cc:442] T 6e95070cd0d6412697bd35caba09c6ee P 07273d7511d54b88b18936d1c8590b03: This TxnStatusTablet replica's current role is: FOLLOWER
I20260923 22:12:53.967881 9577 tablet_bootstrap.cc:492] T 6e95070cd0d6412697bd35caba09c6ee P 7a422c36ebff4c2384978dbc5c2b9afc: No bootstrap required, opened a new log
I20260923 22:12:53.967923 9577 ts_tablet_manager.cc:1397] T 6e95070cd0d6412697bd35caba09c6ee P 7a422c36ebff4c2384978dbc5c2b9afc: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20260923 22:12:53.968091 9577 raft_consensus.cc:348] T 6e95070cd0d6412697bd35caba09c6ee P 7a422c36ebff4c2384978dbc5c2b9afc [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "07273d7511d54b88b18936d1c8590b03" member_type: VOTER last_known_addr { host: "127.1.135.193" port: 33215 } } peers { permanent_uuid: "7a422c36ebff4c2384978dbc5c2b9afc" member_type: VOTER last_known_addr { host: "127.1.135.194" port: 46503 } } peers { permanent_uuid: "f36bf50226164a81aa65fea3b9d4c07d" member_type: VOTER last_known_addr { host: "127.1.135.195" port: 40361 } }
I20260923 22:12:53.968184 9577 raft_consensus.cc:374] T 6e95070cd0d6412697bd35caba09c6ee P 7a422c36ebff4c2384978dbc5c2b9afc [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260923 22:12:53.968205 9577 raft_consensus.cc:729] T 6e95070cd0d6412697bd35caba09c6ee P 7a422c36ebff4c2384978dbc5c2b9afc [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 7a422c36ebff4c2384978dbc5c2b9afc, State: Initialized, Role: FOLLOWER
I20260923 22:12:53.968245 9577 consensus_queue.cc:260] T 6e95070cd0d6412697bd35caba09c6ee P 7a422c36ebff4c2384978dbc5c2b9afc [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "07273d7511d54b88b18936d1c8590b03" member_type: VOTER last_known_addr { host: "127.1.135.193" port: 33215 } } peers { permanent_uuid: "7a422c36ebff4c2384978dbc5c2b9afc" member_type: VOTER last_known_addr { host: "127.1.135.194" port: 46503 } } peers { permanent_uuid: "f36bf50226164a81aa65fea3b9d4c07d" member_type: VOTER last_known_addr { host: "127.1.135.195" port: 40361 } }
I20260923 22:12:53.968348 9577 ts_tablet_manager.cc:1428] T 6e95070cd0d6412697bd35caba09c6ee P 7a422c36ebff4c2384978dbc5c2b9afc: Time spent starting tablet: real 0.000s user 0.001s sys 0.000s
I20260923 22:12:53.968514 9584 tablet_replica.cc:439] T 6e95070cd0d6412697bd35caba09c6ee P 7a422c36ebff4c2384978dbc5c2b9afc: TxnStatusTablet state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "07273d7511d54b88b18936d1c8590b03" member_type: VOTER last_known_addr { host: "127.1.135.193" port: 33215 } } peers { permanent_uuid: "7a422c36ebff4c2384978dbc5c2b9afc" member_type: VOTER last_known_addr { host: "127.1.135.194" port: 46503 } } peers { permanent_uuid: "f36bf50226164a81aa65fea3b9d4c07d" member_type: VOTER last_known_addr { host: "127.1.135.195" port: 40361 } } }
I20260923 22:12:53.968595 9584 tablet_replica.cc:442] T 6e95070cd0d6412697bd35caba09c6ee P 7a422c36ebff4c2384978dbc5c2b9afc: This TxnStatusTablet replica's current role is: FOLLOWER
I20260923 22:12:54.057981 9583 raft_consensus.cc:482] T 6e95070cd0d6412697bd35caba09c6ee P 07273d7511d54b88b18936d1c8590b03 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260923 22:12:54.058099 9583 raft_consensus.cc:504] T 6e95070cd0d6412697bd35caba09c6ee P 07273d7511d54b88b18936d1c8590b03 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "07273d7511d54b88b18936d1c8590b03" member_type: VOTER last_known_addr { host: "127.1.135.193" port: 33215 } } peers { permanent_uuid: "7a422c36ebff4c2384978dbc5c2b9afc" member_type: VOTER last_known_addr { host: "127.1.135.194" port: 46503 } } peers { permanent_uuid: "f36bf50226164a81aa65fea3b9d4c07d" member_type: VOTER last_known_addr { host: "127.1.135.195" port: 40361 } }
I20260923 22:12:54.058284 9583 leader_election.cc:288] T 6e95070cd0d6412697bd35caba09c6ee P 07273d7511d54b88b18936d1c8590b03 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 7a422c36ebff4c2384978dbc5c2b9afc (127.1.135.194:46503), f36bf50226164a81aa65fea3b9d4c07d (127.1.135.195:40361)
I20260923 22:12:54.058455 9363 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "6e95070cd0d6412697bd35caba09c6ee" candidate_uuid: "07273d7511d54b88b18936d1c8590b03" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7a422c36ebff4c2384978dbc5c2b9afc" is_pre_election: true
I20260923 22:12:54.058524 9502 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "6e95070cd0d6412697bd35caba09c6ee" candidate_uuid: "07273d7511d54b88b18936d1c8590b03" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f36bf50226164a81aa65fea3b9d4c07d" is_pre_election: true
I20260923 22:12:54.058584 9363 raft_consensus.cc:2454] T 6e95070cd0d6412697bd35caba09c6ee P 7a422c36ebff4c2384978dbc5c2b9afc [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 07273d7511d54b88b18936d1c8590b03 in term 0.
I20260923 22:12:54.058638 9502 raft_consensus.cc:2454] T 6e95070cd0d6412697bd35caba09c6ee P f36bf50226164a81aa65fea3b9d4c07d [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 07273d7511d54b88b18936d1c8590b03 in term 0.
I20260923 22:12:54.058790 9156 leader_election.cc:302] T 6e95070cd0d6412697bd35caba09c6ee P 07273d7511d54b88b18936d1c8590b03 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 07273d7511d54b88b18936d1c8590b03, 7a422c36ebff4c2384978dbc5c2b9afc; no voters:
I20260923 22:12:54.058902 9583 raft_consensus.cc:2781] T 6e95070cd0d6412697bd35caba09c6ee P 07273d7511d54b88b18936d1c8590b03 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260923 22:12:54.058955 9583 raft_consensus.cc:482] T 6e95070cd0d6412697bd35caba09c6ee P 07273d7511d54b88b18936d1c8590b03 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260923 22:12:54.058977 9583 raft_consensus.cc:3037] T 6e95070cd0d6412697bd35caba09c6ee P 07273d7511d54b88b18936d1c8590b03 [term 0 FOLLOWER]: Advancing to term 1
I20260923 22:12:54.059644 9583 raft_consensus.cc:504] T 6e95070cd0d6412697bd35caba09c6ee P 07273d7511d54b88b18936d1c8590b03 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "07273d7511d54b88b18936d1c8590b03" member_type: VOTER last_known_addr { host: "127.1.135.193" port: 33215 } } peers { permanent_uuid: "7a422c36ebff4c2384978dbc5c2b9afc" member_type: VOTER last_known_addr { host: "127.1.135.194" port: 46503 } } peers { permanent_uuid: "f36bf50226164a81aa65fea3b9d4c07d" member_type: VOTER last_known_addr { host: "127.1.135.195" port: 40361 } }
I20260923 22:12:54.059799 9583 leader_election.cc:288] T 6e95070cd0d6412697bd35caba09c6ee P 07273d7511d54b88b18936d1c8590b03 [CANDIDATE]: Term 1 election: Requested vote from peers 7a422c36ebff4c2384978dbc5c2b9afc (127.1.135.194:46503), f36bf50226164a81aa65fea3b9d4c07d (127.1.135.195:40361)
I20260923 22:12:54.059955 9502 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "6e95070cd0d6412697bd35caba09c6ee" candidate_uuid: "07273d7511d54b88b18936d1c8590b03" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f36bf50226164a81aa65fea3b9d4c07d"
I20260923 22:12:54.059927 9363 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "6e95070cd0d6412697bd35caba09c6ee" candidate_uuid: "07273d7511d54b88b18936d1c8590b03" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7a422c36ebff4c2384978dbc5c2b9afc"
I20260923 22:12:54.060036 9363 raft_consensus.cc:3037] T 6e95070cd0d6412697bd35caba09c6ee P 7a422c36ebff4c2384978dbc5c2b9afc [term 0 FOLLOWER]: Advancing to term 1
I20260923 22:12:54.060040 9502 raft_consensus.cc:3037] T 6e95070cd0d6412697bd35caba09c6ee P f36bf50226164a81aa65fea3b9d4c07d [term 0 FOLLOWER]: Advancing to term 1
I20260923 22:12:54.060705 9502 raft_consensus.cc:2454] T 6e95070cd0d6412697bd35caba09c6ee P f36bf50226164a81aa65fea3b9d4c07d [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 07273d7511d54b88b18936d1c8590b03 in term 1.
I20260923 22:12:54.060711 9363 raft_consensus.cc:2454] T 6e95070cd0d6412697bd35caba09c6ee P 7a422c36ebff4c2384978dbc5c2b9afc [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 07273d7511d54b88b18936d1c8590b03 in term 1.
I20260923 22:12:54.060878 9155 leader_election.cc:302] T 6e95070cd0d6412697bd35caba09c6ee P 07273d7511d54b88b18936d1c8590b03 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 07273d7511d54b88b18936d1c8590b03, f36bf50226164a81aa65fea3b9d4c07d; no voters:
I20260923 22:12:54.060987 9583 raft_consensus.cc:2781] T 6e95070cd0d6412697bd35caba09c6ee P 07273d7511d54b88b18936d1c8590b03 [term 1 FOLLOWER]: Leader election won for term 1
I20260923 22:12:54.061070 9583 raft_consensus.cc:686] T 6e95070cd0d6412697bd35caba09c6ee P 07273d7511d54b88b18936d1c8590b03 [term 1 LEADER]: Becoming Leader. State: Replica: 07273d7511d54b88b18936d1c8590b03, State: Running, Role: LEADER
I20260923 22:12:54.061122 9583 consensus_queue.cc:237] T 6e95070cd0d6412697bd35caba09c6ee P 07273d7511d54b88b18936d1c8590b03 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "07273d7511d54b88b18936d1c8590b03" member_type: VOTER last_known_addr { host: "127.1.135.193" port: 33215 } } peers { permanent_uuid: "7a422c36ebff4c2384978dbc5c2b9afc" member_type: VOTER last_known_addr { host: "127.1.135.194" port: 46503 } } peers { permanent_uuid: "f36bf50226164a81aa65fea3b9d4c07d" member_type: VOTER last_known_addr { host: "127.1.135.195" port: 40361 } }
I20260923 22:12:54.061443 9608 tablet_replica.cc:439] T 6e95070cd0d6412697bd35caba09c6ee P 07273d7511d54b88b18936d1c8590b03: TxnStatusTablet state changed. Reason: New leader 07273d7511d54b88b18936d1c8590b03. Latest consensus state: current_term: 1 leader_uuid: "07273d7511d54b88b18936d1c8590b03" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "07273d7511d54b88b18936d1c8590b03" member_type: VOTER last_known_addr { host: "127.1.135.193" port: 33215 } } peers { permanent_uuid: "7a422c36ebff4c2384978dbc5c2b9afc" member_type: VOTER last_known_addr { host: "127.1.135.194" port: 46503 } } peers { permanent_uuid: "f36bf50226164a81aa65fea3b9d4c07d" member_type: VOTER last_known_addr { host: "127.1.135.195" port: 40361 } } }
I20260923 22:12:54.061540 9608 tablet_replica.cc:442] T 6e95070cd0d6412697bd35caba09c6ee P 07273d7511d54b88b18936d1c8590b03: This TxnStatusTablet replica's current role is: LEADER
I20260923 22:12:54.061760 9066 catalog_manager.cc:5495] T 6e95070cd0d6412697bd35caba09c6ee P 07273d7511d54b88b18936d1c8590b03 reported cstate change: term changed from 0 to 1, leader changed from <none> to 07273d7511d54b88b18936d1c8590b03 (127.1.135.193). New cstate: current_term: 1 leader_uuid: "07273d7511d54b88b18936d1c8590b03" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "07273d7511d54b88b18936d1c8590b03" member_type: VOTER last_known_addr { host: "127.1.135.193" port: 33215 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "7a422c36ebff4c2384978dbc5c2b9afc" member_type: VOTER last_known_addr { host: "127.1.135.194" port: 46503 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "f36bf50226164a81aa65fea3b9d4c07d" member_type: VOTER last_known_addr { host: "127.1.135.195" port: 40361 } health_report { overall_health: UNKNOWN } } }
I20260923 22:12:54.062069 9620 txn_status_manager.cc:873] Waiting until node catch up with all replicated operations in previous term...
I20260923 22:12:54.062206 9620 txn_status_manager.cc:929] Loading transaction status metadata into memory...
I20260923 22:12:54.083482 9573 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 22:12:54.080904 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40640 (local address 127.1.135.193:33215)
0923 22:12:54.080959 (+ 55us) server_negotiation.cc:206] Beginning negotiation
0923 22:12:54.080962 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0923 22:12:54.080971 (+ 9us) server_negotiation.cc:407] Connection header received
0923 22:12:54.081062 (+ 91us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 22:12:54.081063 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 22:12:54.081088 (+ 25us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 22:12:54.081161 (+ 73us) server_negotiation.cc:226] Negotiated authn=TOKEN
0923 22:12:54.081321 (+ 160us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 22:12:54.081899 (+ 578us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 22:12:54.082661 (+ 762us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 22:12:54.082856 (+ 195us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 22:12:54.082941 (+ 85us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 22:12:54.083135 (+ 194us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0923 22:12:54.083242 (+ 107us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0923 22:12:54.083326 (+ 84us) server_negotiation.cc:1035] Waiting for connection context
0923 22:12:54.083367 (+ 41us) server_negotiation.cc:299] Negotiation successful
0923 22:12:54.083387 (+ 20us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":32}
I20260923 22:12:54.083483 9615 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 22:12:54.080844 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.135.193:33215 (local address 127.0.0.1:40640)
0923 22:12:54.080910 (+ 66us) negotiation.cc:107] Waiting for socket to connect
0923 22:12:54.080922 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0923 22:12:54.080975 (+ 53us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0923 22:12:54.081167 (+ 192us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0923 22:12:54.081168 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0923 22:12:54.081169 (+ 1us) client_negotiation.cc:189] Negotiated authn=TOKEN
0923 22:12:54.081237 (+ 68us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 22:12:54.081240 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 22:12:54.081985 (+ 745us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 22:12:54.081986 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 22:12:54.082564 (+ 578us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 22:12:54.082565 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 22:12:54.082942 (+ 377us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 22:12:54.082943 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 22:12:54.083016 (+ 73us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 22:12:54.083033 (+ 17us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0923 22:12:54.083309 (+ 276us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0923 22:12:54.083318 (+ 9us) client_negotiation.cc:769] Sending connection context
0923 22:12:54.083363 (+ 45us) client_negotiation.cc:240] Negotiation successful
0923 22:12:54.083409 (+ 46us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":26}
I20260923 22:12:54.084717 9363 raft_consensus.cc:1261] T 6e95070cd0d6412697bd35caba09c6ee P 7a422c36ebff4c2384978dbc5c2b9afc [term 1 FOLLOWER]: Refusing update from remote peer 07273d7511d54b88b18936d1c8590b03: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260923 22:12:54.084820 9502 raft_consensus.cc:1261] T 6e95070cd0d6412697bd35caba09c6ee P f36bf50226164a81aa65fea3b9d4c07d [term 1 FOLLOWER]: Refusing update from remote peer 07273d7511d54b88b18936d1c8590b03: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260923 22:12:54.084940 9583 consensus_queue.cc:1035] T 6e95070cd0d6412697bd35caba09c6ee P 07273d7511d54b88b18936d1c8590b03 [LEADER]: Connected to new peer: Peer: permanent_uuid: "7a422c36ebff4c2384978dbc5c2b9afc" member_type: VOTER last_known_addr { host: "127.1.135.194" port: 46503 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260923 22:12:54.085083 9608 consensus_queue.cc:1035] T 6e95070cd0d6412697bd35caba09c6ee P 07273d7511d54b88b18936d1c8590b03 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f36bf50226164a81aa65fea3b9d4c07d" member_type: VOTER last_known_addr { host: "127.1.135.195" port: 40361 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260923 22:12:54.085777 9595 tablet_replica.cc:439] T 6e95070cd0d6412697bd35caba09c6ee P f36bf50226164a81aa65fea3b9d4c07d: TxnStatusTablet state changed. Reason: New leader 07273d7511d54b88b18936d1c8590b03. Latest consensus state: current_term: 1 leader_uuid: "07273d7511d54b88b18936d1c8590b03" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "07273d7511d54b88b18936d1c8590b03" member_type: VOTER last_known_addr { host: "127.1.135.193" port: 33215 } } peers { permanent_uuid: "7a422c36ebff4c2384978dbc5c2b9afc" member_type: VOTER last_known_addr { host: "127.1.135.194" port: 46503 } } peers { permanent_uuid: "f36bf50226164a81aa65fea3b9d4c07d" member_type: VOTER last_known_addr { host: "127.1.135.195" port: 40361 } } }
I20260923 22:12:54.085870 9595 tablet_replica.cc:442] T 6e95070cd0d6412697bd35caba09c6ee P f36bf50226164a81aa65fea3b9d4c07d: This TxnStatusTablet replica's current role is: FOLLOWER
I20260923 22:12:54.086071 9584 tablet_replica.cc:439] T 6e95070cd0d6412697bd35caba09c6ee P 7a422c36ebff4c2384978dbc5c2b9afc: TxnStatusTablet state changed. Reason: New leader 07273d7511d54b88b18936d1c8590b03. Latest consensus state: current_term: 1 leader_uuid: "07273d7511d54b88b18936d1c8590b03" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "07273d7511d54b88b18936d1c8590b03" member_type: VOTER last_known_addr { host: "127.1.135.193" port: 33215 } } peers { permanent_uuid: "7a422c36ebff4c2384978dbc5c2b9afc" member_type: VOTER last_known_addr { host: "127.1.135.194" port: 46503 } } peers { permanent_uuid: "f36bf50226164a81aa65fea3b9d4c07d" member_type: VOTER last_known_addr { host: "127.1.135.195" port: 40361 } } }
I20260923 22:12:54.086165 9584 tablet_replica.cc:442] T 6e95070cd0d6412697bd35caba09c6ee P 7a422c36ebff4c2384978dbc5c2b9afc: This TxnStatusTablet replica's current role is: FOLLOWER
I20260923 22:12:54.086791 9608 tablet_replica.cc:439] T 6e95070cd0d6412697bd35caba09c6ee P 07273d7511d54b88b18936d1c8590b03: TxnStatusTablet state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "07273d7511d54b88b18936d1c8590b03" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "07273d7511d54b88b18936d1c8590b03" member_type: VOTER last_known_addr { host: "127.1.135.193" port: 33215 } } peers { permanent_uuid: "7a422c36ebff4c2384978dbc5c2b9afc" member_type: VOTER last_known_addr { host: "127.1.135.194" port: 46503 } } peers { permanent_uuid: "f36bf50226164a81aa65fea3b9d4c07d" member_type: VOTER last_known_addr { host: "127.1.135.195" port: 40361 } } }
I20260923 22:12:54.086871 9608 tablet_replica.cc:442] T 6e95070cd0d6412697bd35caba09c6ee P 07273d7511d54b88b18936d1c8590b03: This TxnStatusTablet replica's current role is: LEADER
I20260923 22:12:54.086967 9608 tablet_replica.cc:439] T 6e95070cd0d6412697bd35caba09c6ee P 07273d7511d54b88b18936d1c8590b03: TxnStatusTablet state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "07273d7511d54b88b18936d1c8590b03" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "07273d7511d54b88b18936d1c8590b03" member_type: VOTER last_known_addr { host: "127.1.135.193" port: 33215 } } peers { permanent_uuid: "7a422c36ebff4c2384978dbc5c2b9afc" member_type: VOTER last_known_addr { host: "127.1.135.194" port: 46503 } } peers { permanent_uuid: "f36bf50226164a81aa65fea3b9d4c07d" member_type: VOTER last_known_addr { host: "127.1.135.195" port: 40361 } } }
I20260923 22:12:54.087008 9608 tablet_replica.cc:442] T 6e95070cd0d6412697bd35caba09c6ee P 07273d7511d54b88b18936d1c8590b03: This TxnStatusTablet replica's current role is: LEADER
I20260923 22:12:54.087339 9595 tablet_replica.cc:439] T 6e95070cd0d6412697bd35caba09c6ee P f36bf50226164a81aa65fea3b9d4c07d: TxnStatusTablet state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "07273d7511d54b88b18936d1c8590b03" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "07273d7511d54b88b18936d1c8590b03" member_type: VOTER last_known_addr { host: "127.1.135.193" port: 33215 } } peers { permanent_uuid: "7a422c36ebff4c2384978dbc5c2b9afc" member_type: VOTER last_known_addr { host: "127.1.135.194" port: 46503 } } peers { permanent_uuid: "f36bf50226164a81aa65fea3b9d4c07d" member_type: VOTER last_known_addr { host: "127.1.135.195" port: 40361 } } }
I20260923 22:12:54.087419 9595 tablet_replica.cc:442] T 6e95070cd0d6412697bd35caba09c6ee P f36bf50226164a81aa65fea3b9d4c07d: This TxnStatusTablet replica's current role is: FOLLOWER
I20260923 22:12:54.087735 9584 tablet_replica.cc:439] T 6e95070cd0d6412697bd35caba09c6ee P 7a422c36ebff4c2384978dbc5c2b9afc: TxnStatusTablet state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "07273d7511d54b88b18936d1c8590b03" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "07273d7511d54b88b18936d1c8590b03" member_type: VOTER last_known_addr { host: "127.1.135.193" port: 33215 } } peers { permanent_uuid: "7a422c36ebff4c2384978dbc5c2b9afc" member_type: VOTER last_known_addr { host: "127.1.135.194" port: 46503 } } peers { permanent_uuid: "f36bf50226164a81aa65fea3b9d4c07d" member_type: VOTER last_known_addr { host: "127.1.135.195" port: 40361 } } }
I20260923 22:12:54.087811 9584 tablet_replica.cc:442] T 6e95070cd0d6412697bd35caba09c6ee P 7a422c36ebff4c2384978dbc5c2b9afc: This TxnStatusTablet replica's current role is: FOLLOWER
I20260923 22:12:54.090939 9573 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 22:12:54.088805 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40654 (local address 127.1.135.193:33215)
0923 22:12:54.088842 (+ 37us) server_negotiation.cc:206] Beginning negotiation
0923 22:12:54.088844 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0923 22:12:54.088853 (+ 9us) server_negotiation.cc:407] Connection header received
0923 22:12:54.088905 (+ 52us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 22:12:54.088906 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 22:12:54.088917 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 22:12:54.088971 (+ 54us) server_negotiation.cc:226] Negotiated authn=TOKEN
0923 22:12:54.089071 (+ 100us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 22:12:54.089518 (+ 447us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 22:12:54.090272 (+ 754us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 22:12:54.090471 (+ 199us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 22:12:54.090550 (+ 79us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 22:12:54.090687 (+ 137us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0923 22:12:54.090719 (+ 32us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0923 22:12:54.090787 (+ 68us) server_negotiation.cc:1035] Waiting for connection context
0923 22:12:54.090847 (+ 60us) server_negotiation.cc:299] Negotiation successful
0923 22:12:54.090864 (+ 17us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":19}
I20260923 22:12:54.095695 9573 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 22:12:54.093398 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.135.193:59549 (local address 127.1.135.193:33215)
0923 22:12:54.093431 (+ 33us) server_negotiation.cc:206] Beginning negotiation
0923 22:12:54.093433 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0923 22:12:54.093558 (+ 125us) server_negotiation.cc:407] Connection header received
0923 22:12:54.093629 (+ 71us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 22:12:54.093629 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 22:12:54.093638 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 22:12:54.093719 (+ 81us) server_negotiation.cc:226] Negotiated authn=TOKEN
0923 22:12:54.093832 (+ 113us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 22:12:54.094255 (+ 423us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 22:12:54.095019 (+ 764us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 22:12:54.095187 (+ 168us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 22:12:54.095258 (+ 71us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 22:12:54.095441 (+ 183us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0923 22:12:54.095473 (+ 32us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0923 22:12:54.095545 (+ 72us) server_negotiation.cc:1035] Waiting for connection context
0923 22:12:54.095603 (+ 58us) server_negotiation.cc:299] Negotiation successful
0923 22:12:54.095617 (+ 14us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":14}
I20260923 22:12:54.095923 9627 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 22:12:54.093351 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.135.193:33215 (local address 127.1.135.193:59549)
0923 22:12:54.093482 (+ 131us) negotiation.cc:107] Waiting for socket to connect
0923 22:12:54.093491 (+ 9us) client_negotiation.cc:174] Beginning negotiation
0923 22:12:54.093574 (+ 83us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0923 22:12:54.093719 (+ 145us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0923 22:12:54.093720 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0923 22:12:54.093722 (+ 2us) client_negotiation.cc:189] Negotiated authn=TOKEN
0923 22:12:54.093772 (+ 50us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 22:12:54.093773 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 22:12:54.094334 (+ 561us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 22:12:54.094335 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 22:12:54.094921 (+ 586us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 22:12:54.094922 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 22:12:54.095273 (+ 351us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 22:12:54.095274 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 22:12:54.095333 (+ 59us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 22:12:54.095352 (+ 19us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0923 22:12:54.095545 (+ 193us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0923 22:12:54.095547 (+ 2us) client_negotiation.cc:769] Sending connection context
0923 22:12:54.095598 (+ 51us) client_negotiation.cc:240] Negotiation successful
0923 22:12:54.095637 (+ 39us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":102,"thread_start_us":78,"threads_started":1}
I20260923 22:12:54.098099 9502 raft_consensus.cc:1261] T c8cfb50a86ab4f0f8e3c93b2df2f736b P f36bf50226164a81aa65fea3b9d4c07d [term 1 FOLLOWER]: Refusing update from remote peer 07273d7511d54b88b18936d1c8590b03: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260923 22:12:54.098121 9363 raft_consensus.cc:1261] T c8cfb50a86ab4f0f8e3c93b2df2f736b P 7a422c36ebff4c2384978dbc5c2b9afc [term 1 FOLLOWER]: Refusing update from remote peer 07273d7511d54b88b18936d1c8590b03: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260923 22:12:54.098284 9583 consensus_queue.cc:1035] T c8cfb50a86ab4f0f8e3c93b2df2f736b P 07273d7511d54b88b18936d1c8590b03 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f36bf50226164a81aa65fea3b9d4c07d" member_type: VOTER last_known_addr { host: "127.1.135.195" port: 40361 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260923 22:12:54.098368 9608 consensus_queue.cc:1035] T c8cfb50a86ab4f0f8e3c93b2df2f736b P 07273d7511d54b88b18936d1c8590b03 [LEADER]: Connected to new peer: Peer: permanent_uuid: "7a422c36ebff4c2384978dbc5c2b9afc" member_type: VOTER last_known_addr { host: "127.1.135.194" port: 46503 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260923 22:12:54.111083 9063 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:48896:
table { table_name: "test-table" } modify_external_catalogs: true
I20260923 22:12:54.111196 9063 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:48896:
table { table_name: "test-table" } modify_external_catalogs: true
I20260923 22:12:54.111845 9063 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 53d3bf4260ad47b69a41891f4fa3a6e5: Sending DeleteTablet for 3 replicas of tablet c8cfb50a86ab4f0f8e3c93b2df2f736b
I20260923 22:12:54.112124 9483 tablet_service.cc:1514] Processing DeleteTablet for tablet c8cfb50a86ab4f0f8e3c93b2df2f736b with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-23 22:12:54 UTC) from {username='oryx', principal='oryx/127.1.135.254@KRBTEST.COM'} at 127.0.0.1:34286
I20260923 22:12:54.112138 9343 tablet_service.cc:1514] Processing DeleteTablet for tablet c8cfb50a86ab4f0f8e3c93b2df2f736b with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-23 22:12:54 UTC) from {username='oryx', principal='oryx/127.1.135.254@KRBTEST.COM'} at 127.0.0.1:32868
I20260923 22:12:54.112234 9602 tablet_replica.cc:330] T c8cfb50a86ab4f0f8e3c93b2df2f736b P f36bf50226164a81aa65fea3b9d4c07d: stopping tablet replica
I20260923 22:12:54.112241 9600 tablet_replica.cc:330] T c8cfb50a86ab4f0f8e3c93b2df2f736b P 7a422c36ebff4c2384978dbc5c2b9afc: stopping tablet replica
I20260923 22:12:54.112293 9600 raft_consensus.cc:2229] T c8cfb50a86ab4f0f8e3c93b2df2f736b P 7a422c36ebff4c2384978dbc5c2b9afc [term 1 FOLLOWER]: Raft consensus shutting down.
I20260923 22:12:54.112293 9602 raft_consensus.cc:2229] T c8cfb50a86ab4f0f8e3c93b2df2f736b P f36bf50226164a81aa65fea3b9d4c07d [term 1 FOLLOWER]: Raft consensus shutting down.
I20260923 22:12:54.112326 9602 raft_consensus.cc:2258] T c8cfb50a86ab4f0f8e3c93b2df2f736b P f36bf50226164a81aa65fea3b9d4c07d [term 1 FOLLOWER]: Raft consensus is shut down!
I20260923 22:12:54.112326 9600 raft_consensus.cc:2258] T c8cfb50a86ab4f0f8e3c93b2df2f736b P 7a422c36ebff4c2384978dbc5c2b9afc [term 1 FOLLOWER]: Raft consensus is shut down!
I20260923 22:12:54.112591 9602 ts_tablet_manager.cc:1905] T c8cfb50a86ab4f0f8e3c93b2df2f736b P f36bf50226164a81aa65fea3b9d4c07d: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260923 22:12:54.113945 9203 tablet_service.cc:1514] Processing DeleteTablet for tablet c8cfb50a86ab4f0f8e3c93b2df2f736b with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-23 22:12:54 UTC) from {username='oryx', principal='oryx/127.1.135.254@KRBTEST.COM'} at 127.0.0.1:40636
I20260923 22:12:54.114044 9601 tablet_replica.cc:330] T c8cfb50a86ab4f0f8e3c93b2df2f736b P 07273d7511d54b88b18936d1c8590b03: stopping tablet replica
I20260923 22:12:54.114086 9601 raft_consensus.cc:2229] T c8cfb50a86ab4f0f8e3c93b2df2f736b P 07273d7511d54b88b18936d1c8590b03 [term 1 LEADER]: Raft consensus shutting down.
I20260923 22:12:54.114197 9601 raft_consensus.cc:2258] T c8cfb50a86ab4f0f8e3c93b2df2f736b P 07273d7511d54b88b18936d1c8590b03 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260923 22:12:54.114249 9602 ts_tablet_manager.cc:1918] T c8cfb50a86ab4f0f8e3c93b2df2f736b P f36bf50226164a81aa65fea3b9d4c07d: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260923 22:12:54.114296 9602 log.cc:1196] T c8cfb50a86ab4f0f8e3c93b2df2f736b P f36bf50226164a81aa65fea3b9d4c07d: Deleting WAL directory at /tmp/dist-test-taskRiQe_a/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790201539513534-1567-0/minicluster-data/ts-2/wal/wals/c8cfb50a86ab4f0f8e3c93b2df2f736b
I20260923 22:12:54.114523 9602 ts_tablet_manager.cc:1939] T c8cfb50a86ab4f0f8e3c93b2df2f736b P f36bf50226164a81aa65fea3b9d4c07d: Deleting consensus metadata
I20260923 22:12:54.114548 9600 ts_tablet_manager.cc:1905] T c8cfb50a86ab4f0f8e3c93b2df2f736b P 7a422c36ebff4c2384978dbc5c2b9afc: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260923 22:12:54.115790 9051 catalog_manager.cc:4841] TS f36bf50226164a81aa65fea3b9d4c07d (127.1.135.195:40361): tablet c8cfb50a86ab4f0f8e3c93b2df2f736b (table test-table [id=8565656fe551407791ed4c1c252e19ae]) successfully deleted
I20260923 22:12:54.116020 9601 ts_tablet_manager.cc:1905] T c8cfb50a86ab4f0f8e3c93b2df2f736b P 07273d7511d54b88b18936d1c8590b03: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260923 22:12:54.117373 9601 ts_tablet_manager.cc:1918] T c8cfb50a86ab4f0f8e3c93b2df2f736b P 07273d7511d54b88b18936d1c8590b03: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260923 22:12:54.117429 9601 log.cc:1196] T c8cfb50a86ab4f0f8e3c93b2df2f736b P 07273d7511d54b88b18936d1c8590b03: Deleting WAL directory at /tmp/dist-test-taskRiQe_a/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790201539513534-1567-0/minicluster-data/ts-0/wal/wals/c8cfb50a86ab4f0f8e3c93b2df2f736b
I20260923 22:12:54.117643 9601 ts_tablet_manager.cc:1939] T c8cfb50a86ab4f0f8e3c93b2df2f736b P 07273d7511d54b88b18936d1c8590b03: Deleting consensus metadata
I20260923 22:12:54.117952 9051 catalog_manager.cc:4841] TS 07273d7511d54b88b18936d1c8590b03 (127.1.135.193:33215): tablet c8cfb50a86ab4f0f8e3c93b2df2f736b (table test-table [id=8565656fe551407791ed4c1c252e19ae]) successfully deleted
I20260923 22:12:54.118468 9600 ts_tablet_manager.cc:1918] T c8cfb50a86ab4f0f8e3c93b2df2f736b P 7a422c36ebff4c2384978dbc5c2b9afc: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260923 22:12:54.118528 9600 log.cc:1196] T c8cfb50a86ab4f0f8e3c93b2df2f736b P 7a422c36ebff4c2384978dbc5c2b9afc: Deleting WAL directory at /tmp/dist-test-taskRiQe_a/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790201539513534-1567-0/minicluster-data/ts-1/wal/wals/c8cfb50a86ab4f0f8e3c93b2df2f736b
I20260923 22:12:54.118729 9600 ts_tablet_manager.cc:1939] T c8cfb50a86ab4f0f8e3c93b2df2f736b P 7a422c36ebff4c2384978dbc5c2b9afc: Deleting consensus metadata
I20260923 22:12:54.119029 9052 catalog_manager.cc:4841] TS 7a422c36ebff4c2384978dbc5c2b9afc (127.1.135.194:46503): tablet c8cfb50a86ab4f0f8e3c93b2df2f736b (table test-table [id=8565656fe551407791ed4c1c252e19ae]) successfully deleted
Sep 23 22:12:54 dist-test-slave-s2fj krb5kdc[9021](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790201574, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
Sep 23 22:12:54 dist-test-slave-s2fj krb5kdc[9021](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, test-admin@KRBTEST.COM for kudu/127.1.135.254@KRBTEST.COM, Server not found in Kerberos database
Sep 23 22:12:54 dist-test-slave-s2fj krb5kdc[9021](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, test-admin@KRBTEST.COM for kudu/127.1.135.254@KRBTEST.COM, Server not found in Kerberos database
W20260923 22:12:54.140187 9115 negotiation.cc:337] Failed RPC negotiation. Trace:
0923 22:12:54.134229 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48910 (local address 127.1.135.254:35183)
0923 22:12:54.134300 (+ 71us) server_negotiation.cc:206] Beginning negotiation
0923 22:12:54.134303 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0923 22:12:54.134430 (+ 127us) server_negotiation.cc:407] Connection header received
0923 22:12:54.134495 (+ 65us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 22:12:54.134496 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 22:12:54.134507 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 22:12:54.134555 (+ 48us) server_negotiation.cc:226] Negotiated authn=SASL
0923 22:12:54.135213 (+ 658us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 22:12:54.135685 (+ 472us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 22:12:54.136769 (+ 1084us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 22:12:54.136953 (+ 184us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 22:12:54.137014 (+ 61us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 22:12:54.140125 (+ 3111us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:48910: BlockingRecv error: recv got EOF from 127.0.0.1:48910 (error 108)
Metrics: {"server-negotiator.queue_time_us":41}
Sep 23 22:12:54 dist-test-slave-s2fj krb5kdc[9021](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790201574, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for oryx/127.1.135.254@KRBTEST.COM
I20260923 22:12:54.148257 9115 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 22:12:54.141003 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48922 (local address 127.1.135.254:35183)
0923 22:12:54.141044 (+ 41us) server_negotiation.cc:206] Beginning negotiation
0923 22:12:54.141046 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0923 22:12:54.141139 (+ 93us) server_negotiation.cc:407] Connection header received
0923 22:12:54.141177 (+ 38us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 22:12:54.141178 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 22:12:54.141189 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 22:12:54.141257 (+ 68us) server_negotiation.cc:226] Negotiated authn=SASL
0923 22:12:54.141773 (+ 516us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 22:12:54.142541 (+ 768us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 22:12:54.143199 (+ 658us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 22:12:54.143473 (+ 274us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 22:12:54.143546 (+ 73us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 22:12:54.145658 (+ 2112us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0923 22:12:54.145660 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0923 22:12:54.145660 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0923 22:12:54.145667 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0923 22:12:54.147175 (+ 1508us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0923 22:12:54.147491 (+ 316us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0923 22:12:54.147492 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0923 22:12:54.147492 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0923 22:12:54.147557 (+ 65us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0923 22:12:54.147737 (+ 180us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0923 22:12:54.147738 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0923 22:12:54.147739 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0923 22:12:54.147867 (+ 128us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0923 22:12:54.147963 (+ 96us) server_negotiation.cc:1035] Waiting for connection context
0923 22:12:54.148085 (+ 122us) server_negotiation.cc:299] Negotiation successful
0923 22:12:54.148169 (+ 84us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":16}
I20260923 22:12:54.149715 9063 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:48922:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260923 22:12:54.149890 9063 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260923 22:12:54.151320 9203 tablet_service.cc:1467] Processing CreateTablet for tablet 89768fbac223427bbdeb22824fb8248c (DEFAULT_TABLE table=test-table [id=44b462cee4034ee79bb2a13aef3f1c79]), partition=RANGE (key) PARTITION UNBOUNDED
I20260923 22:12:54.151403 9483 tablet_service.cc:1467] Processing CreateTablet for tablet 89768fbac223427bbdeb22824fb8248c (DEFAULT_TABLE table=test-table [id=44b462cee4034ee79bb2a13aef3f1c79]), partition=RANGE (key) PARTITION UNBOUNDED
I20260923 22:12:54.151418 9343 tablet_service.cc:1467] Processing CreateTablet for tablet 89768fbac223427bbdeb22824fb8248c (DEFAULT_TABLE table=test-table [id=44b462cee4034ee79bb2a13aef3f1c79]), partition=RANGE (key) PARTITION UNBOUNDED
I20260923 22:12:54.151456 9203 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 89768fbac223427bbdeb22824fb8248c. 1 dirs total, 0 dirs full, 0 dirs failed
I20260923 22:12:54.151517 9483 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 89768fbac223427bbdeb22824fb8248c. 1 dirs total, 0 dirs full, 0 dirs failed
I20260923 22:12:54.151549 9343 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 89768fbac223427bbdeb22824fb8248c. 1 dirs total, 0 dirs full, 0 dirs failed
I20260923 22:12:54.152849 9578 tablet_bootstrap.cc:492] T 89768fbac223427bbdeb22824fb8248c P f36bf50226164a81aa65fea3b9d4c07d: Bootstrap starting.
I20260923 22:12:54.152895 9579 tablet_bootstrap.cc:492] T 89768fbac223427bbdeb22824fb8248c P 07273d7511d54b88b18936d1c8590b03: Bootstrap starting.
I20260923 22:12:54.153200 9578 tablet_bootstrap.cc:654] T 89768fbac223427bbdeb22824fb8248c P f36bf50226164a81aa65fea3b9d4c07d: Neither blocks nor log segments found. Creating new log.
I20260923 22:12:54.153285 9579 tablet_bootstrap.cc:654] T 89768fbac223427bbdeb22824fb8248c P 07273d7511d54b88b18936d1c8590b03: Neither blocks nor log segments found. Creating new log.
I20260923 22:12:54.153465 9577 tablet_bootstrap.cc:492] T 89768fbac223427bbdeb22824fb8248c P 7a422c36ebff4c2384978dbc5c2b9afc: Bootstrap starting.
I20260923 22:12:54.153867 9577 tablet_bootstrap.cc:654] T 89768fbac223427bbdeb22824fb8248c P 7a422c36ebff4c2384978dbc5c2b9afc: Neither blocks nor log segments found. Creating new log.
I20260923 22:12:54.154528 9578 tablet_bootstrap.cc:492] T 89768fbac223427bbdeb22824fb8248c P f36bf50226164a81aa65fea3b9d4c07d: No bootstrap required, opened a new log
I20260923 22:12:54.154594 9578 ts_tablet_manager.cc:1397] T 89768fbac223427bbdeb22824fb8248c P f36bf50226164a81aa65fea3b9d4c07d: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20260923 22:12:54.154716 9578 raft_consensus.cc:348] T 89768fbac223427bbdeb22824fb8248c P f36bf50226164a81aa65fea3b9d4c07d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "07273d7511d54b88b18936d1c8590b03" member_type: VOTER last_known_addr { host: "127.1.135.193" port: 33215 } } peers { permanent_uuid: "7a422c36ebff4c2384978dbc5c2b9afc" member_type: VOTER last_known_addr { host: "127.1.135.194" port: 46503 } } peers { permanent_uuid: "f36bf50226164a81aa65fea3b9d4c07d" member_type: VOTER last_known_addr { host: "127.1.135.195" port: 40361 } }
I20260923 22:12:54.154788 9578 raft_consensus.cc:374] T 89768fbac223427bbdeb22824fb8248c P f36bf50226164a81aa65fea3b9d4c07d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260923 22:12:54.154820 9578 raft_consensus.cc:729] T 89768fbac223427bbdeb22824fb8248c P f36bf50226164a81aa65fea3b9d4c07d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: f36bf50226164a81aa65fea3b9d4c07d, State: Initialized, Role: FOLLOWER
I20260923 22:12:54.154872 9578 consensus_queue.cc:260] T 89768fbac223427bbdeb22824fb8248c P f36bf50226164a81aa65fea3b9d4c07d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "07273d7511d54b88b18936d1c8590b03" member_type: VOTER last_known_addr { host: "127.1.135.193" port: 33215 } } peers { permanent_uuid: "7a422c36ebff4c2384978dbc5c2b9afc" member_type: VOTER last_known_addr { host: "127.1.135.194" port: 46503 } } peers { permanent_uuid: "f36bf50226164a81aa65fea3b9d4c07d" member_type: VOTER last_known_addr { host: "127.1.135.195" port: 40361 } }
I20260923 22:12:54.154963 9578 ts_tablet_manager.cc:1428] T 89768fbac223427bbdeb22824fb8248c P f36bf50226164a81aa65fea3b9d4c07d: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20260923 22:12:54.155157 9577 tablet_bootstrap.cc:492] T 89768fbac223427bbdeb22824fb8248c P 7a422c36ebff4c2384978dbc5c2b9afc: No bootstrap required, opened a new log
I20260923 22:12:54.155211 9577 ts_tablet_manager.cc:1397] T 89768fbac223427bbdeb22824fb8248c P 7a422c36ebff4c2384978dbc5c2b9afc: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20260923 22:12:54.155370 9577 raft_consensus.cc:348] T 89768fbac223427bbdeb22824fb8248c P 7a422c36ebff4c2384978dbc5c2b9afc [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "07273d7511d54b88b18936d1c8590b03" member_type: VOTER last_known_addr { host: "127.1.135.193" port: 33215 } } peers { permanent_uuid: "7a422c36ebff4c2384978dbc5c2b9afc" member_type: VOTER last_known_addr { host: "127.1.135.194" port: 46503 } } peers { permanent_uuid: "f36bf50226164a81aa65fea3b9d4c07d" member_type: VOTER last_known_addr { host: "127.1.135.195" port: 40361 } }
I20260923 22:12:54.155462 9577 raft_consensus.cc:374] T 89768fbac223427bbdeb22824fb8248c P 7a422c36ebff4c2384978dbc5c2b9afc [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260923 22:12:54.155490 9577 raft_consensus.cc:729] T 89768fbac223427bbdeb22824fb8248c P 7a422c36ebff4c2384978dbc5c2b9afc [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 7a422c36ebff4c2384978dbc5c2b9afc, State: Initialized, Role: FOLLOWER
I20260923 22:12:54.155546 9577 consensus_queue.cc:260] T 89768fbac223427bbdeb22824fb8248c P 7a422c36ebff4c2384978dbc5c2b9afc [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "07273d7511d54b88b18936d1c8590b03" member_type: VOTER last_known_addr { host: "127.1.135.193" port: 33215 } } peers { permanent_uuid: "7a422c36ebff4c2384978dbc5c2b9afc" member_type: VOTER last_known_addr { host: "127.1.135.194" port: 46503 } } peers { permanent_uuid: "f36bf50226164a81aa65fea3b9d4c07d" member_type: VOTER last_known_addr { host: "127.1.135.195" port: 40361 } }
I20260923 22:12:54.155632 9577 ts_tablet_manager.cc:1428] T 89768fbac223427bbdeb22824fb8248c P 7a422c36ebff4c2384978dbc5c2b9afc: Time spent starting tablet: real 0.000s user 0.001s sys 0.000s
I20260923 22:12:54.156265 9579 tablet_bootstrap.cc:492] T 89768fbac223427bbdeb22824fb8248c P 07273d7511d54b88b18936d1c8590b03: No bootstrap required, opened a new log
I20260923 22:12:54.156319 9579 ts_tablet_manager.cc:1397] T 89768fbac223427bbdeb22824fb8248c P 07273d7511d54b88b18936d1c8590b03: Time spent bootstrapping tablet: real 0.003s user 0.001s sys 0.000s
I20260923 22:12:54.156486 9579 raft_consensus.cc:348] T 89768fbac223427bbdeb22824fb8248c P 07273d7511d54b88b18936d1c8590b03 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "07273d7511d54b88b18936d1c8590b03" member_type: VOTER last_known_addr { host: "127.1.135.193" port: 33215 } } peers { permanent_uuid: "7a422c36ebff4c2384978dbc5c2b9afc" member_type: VOTER last_known_addr { host: "127.1.135.194" port: 46503 } } peers { permanent_uuid: "f36bf50226164a81aa65fea3b9d4c07d" member_type: VOTER last_known_addr { host: "127.1.135.195" port: 40361 } }
I20260923 22:12:54.156576 9579 raft_consensus.cc:374] T 89768fbac223427bbdeb22824fb8248c P 07273d7511d54b88b18936d1c8590b03 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260923 22:12:54.156603 9579 raft_consensus.cc:729] T 89768fbac223427bbdeb22824fb8248c P 07273d7511d54b88b18936d1c8590b03 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 07273d7511d54b88b18936d1c8590b03, State: Initialized, Role: FOLLOWER
I20260923 22:12:54.156642 9579 consensus_queue.cc:260] T 89768fbac223427bbdeb22824fb8248c P 07273d7511d54b88b18936d1c8590b03 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "07273d7511d54b88b18936d1c8590b03" member_type: VOTER last_known_addr { host: "127.1.135.193" port: 33215 } } peers { permanent_uuid: "7a422c36ebff4c2384978dbc5c2b9afc" member_type: VOTER last_known_addr { host: "127.1.135.194" port: 46503 } } peers { permanent_uuid: "f36bf50226164a81aa65fea3b9d4c07d" member_type: VOTER last_known_addr { host: "127.1.135.195" port: 40361 } }
I20260923 22:12:54.156740 9579 ts_tablet_manager.cc:1428] T 89768fbac223427bbdeb22824fb8248c P 07273d7511d54b88b18936d1c8590b03: Time spent starting tablet: real 0.000s user 0.001s sys 0.000s
I20260923 22:12:54.360603 9584 raft_consensus.cc:482] T 89768fbac223427bbdeb22824fb8248c P 7a422c36ebff4c2384978dbc5c2b9afc [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260923 22:12:54.360704 9584 raft_consensus.cc:504] T 89768fbac223427bbdeb22824fb8248c P 7a422c36ebff4c2384978dbc5c2b9afc [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "07273d7511d54b88b18936d1c8590b03" member_type: VOTER last_known_addr { host: "127.1.135.193" port: 33215 } } peers { permanent_uuid: "7a422c36ebff4c2384978dbc5c2b9afc" member_type: VOTER last_known_addr { host: "127.1.135.194" port: 46503 } } peers { permanent_uuid: "f36bf50226164a81aa65fea3b9d4c07d" member_type: VOTER last_known_addr { host: "127.1.135.195" port: 40361 } }
I20260923 22:12:54.361086 9584 leader_election.cc:288] T 89768fbac223427bbdeb22824fb8248c P 7a422c36ebff4c2384978dbc5c2b9afc [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 07273d7511d54b88b18936d1c8590b03 (127.1.135.193:33215), f36bf50226164a81aa65fea3b9d4c07d (127.1.135.195:40361)
I20260923 22:12:54.364166 9576 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 22:12:54.361594 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.135.194:36987 (local address 127.1.135.195:40361)
0923 22:12:54.361655 (+ 61us) server_negotiation.cc:206] Beginning negotiation
0923 22:12:54.361657 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0923 22:12:54.361684 (+ 27us) server_negotiation.cc:407] Connection header received
0923 22:12:54.361747 (+ 63us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 22:12:54.361748 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 22:12:54.361757 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 22:12:54.361827 (+ 70us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0923 22:12:54.361978 (+ 151us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 22:12:54.362453 (+ 475us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 22:12:54.363423 (+ 970us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 22:12:54.363785 (+ 362us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 22:12:54.363870 (+ 85us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 22:12:54.363882 (+ 12us) server_negotiation.cc:1035] Waiting for connection context
0923 22:12:54.364043 (+ 161us) server_negotiation.cc:299] Negotiation successful
0923 22:12:54.364105 (+ 62us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":38}
I20260923 22:12:54.364181 9648 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 22:12:54.361494 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.135.195:40361 (local address 127.1.135.194:36987)
0923 22:12:54.361586 (+ 92us) negotiation.cc:107] Waiting for socket to connect
0923 22:12:54.361605 (+ 19us) client_negotiation.cc:174] Beginning negotiation
0923 22:12:54.361677 (+ 72us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0923 22:12:54.361831 (+ 154us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0923 22:12:54.361832 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0923 22:12:54.361834 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0923 22:12:54.361901 (+ 67us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 22:12:54.361903 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 22:12:54.362544 (+ 641us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 22:12:54.362545 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 22:12:54.363334 (+ 789us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 22:12:54.363335 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 22:12:54.363887 (+ 552us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 22:12:54.363888 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 22:12:54.363965 (+ 77us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 22:12:54.363969 (+ 4us) client_negotiation.cc:769] Sending connection context
0923 22:12:54.364035 (+ 66us) client_negotiation.cc:240] Negotiation successful
0923 22:12:54.364089 (+ 54us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":79,"thread_start_us":42,"threads_started":1}
I20260923 22:12:54.364255 9573 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 22:12:54.361364 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.135.194:58653 (local address 127.1.135.193:33215)
0923 22:12:54.361441 (+ 77us) server_negotiation.cc:206] Beginning negotiation
0923 22:12:54.361443 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0923 22:12:54.361514 (+ 71us) server_negotiation.cc:407] Connection header received
0923 22:12:54.361660 (+ 146us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 22:12:54.361662 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 22:12:54.361679 (+ 17us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 22:12:54.361741 (+ 62us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0923 22:12:54.362176 (+ 435us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 22:12:54.362628 (+ 452us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 22:12:54.363492 (+ 864us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 22:12:54.363846 (+ 354us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 22:12:54.363945 (+ 99us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 22:12:54.363953 (+ 8us) server_negotiation.cc:1035] Waiting for connection context
0923 22:12:54.364149 (+ 196us) server_negotiation.cc:299] Negotiation successful
0923 22:12:54.364186 (+ 37us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":46}
I20260923 22:12:54.364538 9647 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 22:12:54.361301 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.135.193:33215 (local address 127.1.135.194:58653)
0923 22:12:54.361428 (+ 127us) negotiation.cc:107] Waiting for socket to connect
0923 22:12:54.361440 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0923 22:12:54.361526 (+ 86us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0923 22:12:54.361761 (+ 235us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0923 22:12:54.361762 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0923 22:12:54.361764 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0923 22:12:54.362085 (+ 321us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 22:12:54.362086 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 22:12:54.362708 (+ 622us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 22:12:54.362709 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 22:12:54.363401 (+ 692us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 22:12:54.363403 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 22:12:54.363946 (+ 543us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 22:12:54.363947 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 22:12:54.364032 (+ 85us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 22:12:54.364035 (+ 3us) client_negotiation.cc:769] Sending connection context
0923 22:12:54.364139 (+ 104us) client_negotiation.cc:240] Negotiation successful
0923 22:12:54.364178 (+ 39us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":107,"thread_start_us":83,"threads_started":1}
I20260923 22:12:54.364629 9502 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "89768fbac223427bbdeb22824fb8248c" candidate_uuid: "7a422c36ebff4c2384978dbc5c2b9afc" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f36bf50226164a81aa65fea3b9d4c07d" is_pre_election: true
I20260923 22:12:54.364714 9502 raft_consensus.cc:2454] T 89768fbac223427bbdeb22824fb8248c P f36bf50226164a81aa65fea3b9d4c07d [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7a422c36ebff4c2384978dbc5c2b9afc in term 0.
I20260923 22:12:54.364817 9223 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "89768fbac223427bbdeb22824fb8248c" candidate_uuid: "7a422c36ebff4c2384978dbc5c2b9afc" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "07273d7511d54b88b18936d1c8590b03" is_pre_election: true
I20260923 22:12:54.364897 9223 raft_consensus.cc:2454] T 89768fbac223427bbdeb22824fb8248c P 07273d7511d54b88b18936d1c8590b03 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7a422c36ebff4c2384978dbc5c2b9afc in term 0.
I20260923 22:12:54.364872 9295 leader_election.cc:302] T 89768fbac223427bbdeb22824fb8248c P 7a422c36ebff4c2384978dbc5c2b9afc [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 7a422c36ebff4c2384978dbc5c2b9afc, f36bf50226164a81aa65fea3b9d4c07d; no voters:
I20260923 22:12:54.364997 9584 raft_consensus.cc:2781] T 89768fbac223427bbdeb22824fb8248c P 7a422c36ebff4c2384978dbc5c2b9afc [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260923 22:12:54.365046 9584 raft_consensus.cc:482] T 89768fbac223427bbdeb22824fb8248c P 7a422c36ebff4c2384978dbc5c2b9afc [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260923 22:12:54.365067 9584 raft_consensus.cc:3037] T 89768fbac223427bbdeb22824fb8248c P 7a422c36ebff4c2384978dbc5c2b9afc [term 0 FOLLOWER]: Advancing to term 1
I20260923 22:12:54.365718 9584 raft_consensus.cc:504] T 89768fbac223427bbdeb22824fb8248c P 7a422c36ebff4c2384978dbc5c2b9afc [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "07273d7511d54b88b18936d1c8590b03" member_type: VOTER last_known_addr { host: "127.1.135.193" port: 33215 } } peers { permanent_uuid: "7a422c36ebff4c2384978dbc5c2b9afc" member_type: VOTER last_known_addr { host: "127.1.135.194" port: 46503 } } peers { permanent_uuid: "f36bf50226164a81aa65fea3b9d4c07d" member_type: VOTER last_known_addr { host: "127.1.135.195" port: 40361 } }
I20260923 22:12:54.365847 9584 leader_election.cc:288] T 89768fbac223427bbdeb22824fb8248c P 7a422c36ebff4c2384978dbc5c2b9afc [CANDIDATE]: Term 1 election: Requested vote from peers 07273d7511d54b88b18936d1c8590b03 (127.1.135.193:33215), f36bf50226164a81aa65fea3b9d4c07d (127.1.135.195:40361)
I20260923 22:12:54.366004 9223 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "89768fbac223427bbdeb22824fb8248c" candidate_uuid: "7a422c36ebff4c2384978dbc5c2b9afc" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "07273d7511d54b88b18936d1c8590b03"
I20260923 22:12:54.366080 9223 raft_consensus.cc:3037] T 89768fbac223427bbdeb22824fb8248c P 07273d7511d54b88b18936d1c8590b03 [term 0 FOLLOWER]: Advancing to term 1
I20260923 22:12:54.366014 9502 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "89768fbac223427bbdeb22824fb8248c" candidate_uuid: "7a422c36ebff4c2384978dbc5c2b9afc" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f36bf50226164a81aa65fea3b9d4c07d"
I20260923 22:12:54.366112 9502 raft_consensus.cc:3037] T 89768fbac223427bbdeb22824fb8248c P f36bf50226164a81aa65fea3b9d4c07d [term 0 FOLLOWER]: Advancing to term 1
I20260923 22:12:54.366739 9502 raft_consensus.cc:2454] T 89768fbac223427bbdeb22824fb8248c P f36bf50226164a81aa65fea3b9d4c07d [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7a422c36ebff4c2384978dbc5c2b9afc in term 1.
I20260923 22:12:54.366878 9223 raft_consensus.cc:2454] T 89768fbac223427bbdeb22824fb8248c P 07273d7511d54b88b18936d1c8590b03 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7a422c36ebff4c2384978dbc5c2b9afc in term 1.
I20260923 22:12:54.366914 9295 leader_election.cc:302] T 89768fbac223427bbdeb22824fb8248c P 7a422c36ebff4c2384978dbc5c2b9afc [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 7a422c36ebff4c2384978dbc5c2b9afc, f36bf50226164a81aa65fea3b9d4c07d; no voters:
I20260923 22:12:54.367074 9584 raft_consensus.cc:2781] T 89768fbac223427bbdeb22824fb8248c P 7a422c36ebff4c2384978dbc5c2b9afc [term 1 FOLLOWER]: Leader election won for term 1
I20260923 22:12:54.367182 9584 raft_consensus.cc:686] T 89768fbac223427bbdeb22824fb8248c P 7a422c36ebff4c2384978dbc5c2b9afc [term 1 LEADER]: Becoming Leader. State: Replica: 7a422c36ebff4c2384978dbc5c2b9afc, State: Running, Role: LEADER
I20260923 22:12:54.367233 9584 consensus_queue.cc:237] T 89768fbac223427bbdeb22824fb8248c P 7a422c36ebff4c2384978dbc5c2b9afc [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "07273d7511d54b88b18936d1c8590b03" member_type: VOTER last_known_addr { host: "127.1.135.193" port: 33215 } } peers { permanent_uuid: "7a422c36ebff4c2384978dbc5c2b9afc" member_type: VOTER last_known_addr { host: "127.1.135.194" port: 46503 } } peers { permanent_uuid: "f36bf50226164a81aa65fea3b9d4c07d" member_type: VOTER last_known_addr { host: "127.1.135.195" port: 40361 } }
I20260923 22:12:54.367949 9063 catalog_manager.cc:5495] T 89768fbac223427bbdeb22824fb8248c P 7a422c36ebff4c2384978dbc5c2b9afc reported cstate change: term changed from 0 to 1, leader changed from <none> to 7a422c36ebff4c2384978dbc5c2b9afc (127.1.135.194). New cstate: current_term: 1 leader_uuid: "7a422c36ebff4c2384978dbc5c2b9afc" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "07273d7511d54b88b18936d1c8590b03" member_type: VOTER last_known_addr { host: "127.1.135.193" port: 33215 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "7a422c36ebff4c2384978dbc5c2b9afc" member_type: VOTER last_known_addr { host: "127.1.135.194" port: 46503 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "f36bf50226164a81aa65fea3b9d4c07d" member_type: VOTER last_known_addr { host: "127.1.135.195" port: 40361 } health_report { overall_health: UNKNOWN } } }
I20260923 22:12:54.382647 9574 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 22:12:54.380174 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41492 (local address 127.1.135.194:46503)
0923 22:12:54.380233 (+ 59us) server_negotiation.cc:206] Beginning negotiation
0923 22:12:54.380235 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0923 22:12:54.380262 (+ 27us) server_negotiation.cc:407] Connection header received
0923 22:12:54.380370 (+ 108us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 22:12:54.380371 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 22:12:54.380394 (+ 23us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 22:12:54.380494 (+ 100us) server_negotiation.cc:226] Negotiated authn=TOKEN
0923 22:12:54.380633 (+ 139us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 22:12:54.381100 (+ 467us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 22:12:54.381847 (+ 747us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 22:12:54.382056 (+ 209us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 22:12:54.382139 (+ 83us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 22:12:54.382309 (+ 170us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0923 22:12:54.382423 (+ 114us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0923 22:12:54.382500 (+ 77us) server_negotiation.cc:1035] Waiting for connection context
0923 22:12:54.382548 (+ 48us) server_negotiation.cc:299] Negotiation successful
0923 22:12:54.382566 (+ 18us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":37}
I20260923 22:12:54.383594 9223 raft_consensus.cc:1261] T 89768fbac223427bbdeb22824fb8248c P 07273d7511d54b88b18936d1c8590b03 [term 1 FOLLOWER]: Refusing update from remote peer 7a422c36ebff4c2384978dbc5c2b9afc: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260923 22:12:54.383620 9502 raft_consensus.cc:1261] T 89768fbac223427bbdeb22824fb8248c P f36bf50226164a81aa65fea3b9d4c07d [term 1 FOLLOWER]: Refusing update from remote peer 7a422c36ebff4c2384978dbc5c2b9afc: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260923 22:12:54.383847 9584 consensus_queue.cc:1035] T 89768fbac223427bbdeb22824fb8248c P 7a422c36ebff4c2384978dbc5c2b9afc [LEADER]: Connected to new peer: Peer: permanent_uuid: "f36bf50226164a81aa65fea3b9d4c07d" member_type: VOTER last_known_addr { host: "127.1.135.195" port: 40361 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260923 22:12:54.384034 9649 consensus_queue.cc:1035] T 89768fbac223427bbdeb22824fb8248c P 7a422c36ebff4c2384978dbc5c2b9afc [LEADER]: Connected to new peer: Peer: permanent_uuid: "07273d7511d54b88b18936d1c8590b03" member_type: VOTER last_known_addr { host: "127.1.135.193" port: 33215 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260923 22:12:54.387061 9063 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:48922:
table { table_name: "test-table" } modify_external_catalogs: true
I20260923 22:12:54.387176 9063 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:48922:
table { table_name: "test-table" } modify_external_catalogs: true
I20260923 22:12:54.387795 9063 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 53d3bf4260ad47b69a41891f4fa3a6e5: Sending DeleteTablet for 3 replicas of tablet 89768fbac223427bbdeb22824fb8248c
I20260923 22:12:54.388007 9343 tablet_service.cc:1514] Processing DeleteTablet for tablet 89768fbac223427bbdeb22824fb8248c with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-23 22:12:54 UTC) from {username='oryx', principal='oryx/127.1.135.254@KRBTEST.COM'} at 127.0.0.1:32868
I20260923 22:12:54.388051 9203 tablet_service.cc:1514] Processing DeleteTablet for tablet 89768fbac223427bbdeb22824fb8248c with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-23 22:12:54 UTC) from {username='oryx', principal='oryx/127.1.135.254@KRBTEST.COM'} at 127.0.0.1:40636
I20260923 22:12:54.388172 9600 tablet_replica.cc:330] T 89768fbac223427bbdeb22824fb8248c P 7a422c36ebff4c2384978dbc5c2b9afc: stopping tablet replica
I20260923 22:12:54.388228 9600 raft_consensus.cc:2229] T 89768fbac223427bbdeb22824fb8248c P 7a422c36ebff4c2384978dbc5c2b9afc [term 1 LEADER]: Raft consensus shutting down.
I20260923 22:12:54.388307 9600 raft_consensus.cc:2258] T 89768fbac223427bbdeb22824fb8248c P 7a422c36ebff4c2384978dbc5c2b9afc [term 1 FOLLOWER]: Raft consensus is shut down!
I20260923 22:12:54.388276 9063 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:48922:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20260923 22:12:54.388383 9601 tablet_replica.cc:330] T 89768fbac223427bbdeb22824fb8248c P 07273d7511d54b88b18936d1c8590b03: stopping tablet replica
W20260923 22:12:54.388406 9063 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260923 22:12:54.388437 9601 raft_consensus.cc:2229] T 89768fbac223427bbdeb22824fb8248c P 07273d7511d54b88b18936d1c8590b03 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260923 22:12:54.388478 9601 raft_consensus.cc:2258] T 89768fbac223427bbdeb22824fb8248c P 07273d7511d54b88b18936d1c8590b03 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260923 22:12:54.388561 9483 tablet_service.cc:1514] Processing DeleteTablet for tablet 89768fbac223427bbdeb22824fb8248c with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-23 22:12:54 UTC) from {username='oryx', principal='oryx/127.1.135.254@KRBTEST.COM'} at 127.0.0.1:34286
I20260923 22:12:54.388643 9602 tablet_replica.cc:330] T 89768fbac223427bbdeb22824fb8248c P f36bf50226164a81aa65fea3b9d4c07d: stopping tablet replica
I20260923 22:12:54.388687 9602 raft_consensus.cc:2229] T 89768fbac223427bbdeb22824fb8248c P f36bf50226164a81aa65fea3b9d4c07d [term 1 FOLLOWER]: Raft consensus shutting down.
I20260923 22:12:54.388727 9602 raft_consensus.cc:2258] T 89768fbac223427bbdeb22824fb8248c P f36bf50226164a81aa65fea3b9d4c07d [term 1 FOLLOWER]: Raft consensus is shut down!
I20260923 22:12:54.388782 9601 ts_tablet_manager.cc:1905] T 89768fbac223427bbdeb22824fb8248c P 07273d7511d54b88b18936d1c8590b03: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260923 22:12:54.389309 9602 ts_tablet_manager.cc:1905] T 89768fbac223427bbdeb22824fb8248c P f36bf50226164a81aa65fea3b9d4c07d: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260923 22:12:54.390375 9601 ts_tablet_manager.cc:1918] T 89768fbac223427bbdeb22824fb8248c P 07273d7511d54b88b18936d1c8590b03: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260923 22:12:54.390439 9601 log.cc:1196] T 89768fbac223427bbdeb22824fb8248c P 07273d7511d54b88b18936d1c8590b03: Deleting WAL directory at /tmp/dist-test-taskRiQe_a/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790201539513534-1567-0/minicluster-data/ts-0/wal/wals/89768fbac223427bbdeb22824fb8248c
I20260923 22:12:54.390648 9601 ts_tablet_manager.cc:1939] T 89768fbac223427bbdeb22824fb8248c P 07273d7511d54b88b18936d1c8590b03: Deleting consensus metadata
I20260923 22:12:54.390889 9602 ts_tablet_manager.cc:1918] T 89768fbac223427bbdeb22824fb8248c P f36bf50226164a81aa65fea3b9d4c07d: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260923 22:12:54.390950 9602 log.cc:1196] T 89768fbac223427bbdeb22824fb8248c P f36bf50226164a81aa65fea3b9d4c07d: Deleting WAL directory at /tmp/dist-test-taskRiQe_a/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790201539513534-1567-0/minicluster-data/ts-2/wal/wals/89768fbac223427bbdeb22824fb8248c
I20260923 22:12:54.390970 9051 catalog_manager.cc:4841] TS 07273d7511d54b88b18936d1c8590b03 (127.1.135.193:33215): tablet 89768fbac223427bbdeb22824fb8248c (table test-table [id=44b462cee4034ee79bb2a13aef3f1c79]) successfully deleted
I20260923 22:12:54.391152 9602 ts_tablet_manager.cc:1939] T 89768fbac223427bbdeb22824fb8248c P f36bf50226164a81aa65fea3b9d4c07d: Deleting consensus metadata
I20260923 22:12:54.391413 9600 ts_tablet_manager.cc:1905] T 89768fbac223427bbdeb22824fb8248c P 7a422c36ebff4c2384978dbc5c2b9afc: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260923 22:12:54.391570 9051 catalog_manager.cc:4841] TS f36bf50226164a81aa65fea3b9d4c07d (127.1.135.195:40361): tablet 89768fbac223427bbdeb22824fb8248c (table test-table [id=44b462cee4034ee79bb2a13aef3f1c79]) successfully deleted
I20260923 22:12:54.392848 9600 ts_tablet_manager.cc:1918] T 89768fbac223427bbdeb22824fb8248c P 7a422c36ebff4c2384978dbc5c2b9afc: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260923 22:12:54.392905 9600 log.cc:1196] T 89768fbac223427bbdeb22824fb8248c P 7a422c36ebff4c2384978dbc5c2b9afc: Deleting WAL directory at /tmp/dist-test-taskRiQe_a/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790201539513534-1567-0/minicluster-data/ts-1/wal/wals/89768fbac223427bbdeb22824fb8248c
I20260923 22:12:54.392935 9343 tablet_service.cc:1467] Processing CreateTablet for tablet 5e1cdb94d0b4481bb14d9cccb195eb50 (DEFAULT_TABLE table=test-table [id=24dee7017ecf4a468b1213ea485b4fcf]), partition=RANGE (key) PARTITION UNBOUNDED
I20260923 22:12:54.393030 9343 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 5e1cdb94d0b4481bb14d9cccb195eb50. 1 dirs total, 0 dirs full, 0 dirs failed
I20260923 22:12:54.393110 9600 ts_tablet_manager.cc:1939] T 89768fbac223427bbdeb22824fb8248c P 7a422c36ebff4c2384978dbc5c2b9afc: Deleting consensus metadata
I20260923 22:12:54.393277 9483 tablet_service.cc:1467] Processing CreateTablet for tablet 5e1cdb94d0b4481bb14d9cccb195eb50 (DEFAULT_TABLE table=test-table [id=24dee7017ecf4a468b1213ea485b4fcf]), partition=RANGE (key) PARTITION UNBOUNDED
I20260923 22:12:54.393357 9483 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 5e1cdb94d0b4481bb14d9cccb195eb50. 1 dirs total, 0 dirs full, 0 dirs failed
I20260923 22:12:54.394212 9577 tablet_bootstrap.cc:492] T 5e1cdb94d0b4481bb14d9cccb195eb50 P 7a422c36ebff4c2384978dbc5c2b9afc: Bootstrap starting.
I20260923 22:12:54.394222 9203 tablet_service.cc:1467] Processing CreateTablet for tablet 5e1cdb94d0b4481bb14d9cccb195eb50 (DEFAULT_TABLE table=test-table [id=24dee7017ecf4a468b1213ea485b4fcf]), partition=RANGE (key) PARTITION UNBOUNDED
I20260923 22:12:54.394320 9203 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 5e1cdb94d0b4481bb14d9cccb195eb50. 1 dirs total, 0 dirs full, 0 dirs failed
I20260923 22:12:54.394515 9578 tablet_bootstrap.cc:492] T 5e1cdb94d0b4481bb14d9cccb195eb50 P f36bf50226164a81aa65fea3b9d4c07d: Bootstrap starting.
I20260923 22:12:54.394613 9577 tablet_bootstrap.cc:654] T 5e1cdb94d0b4481bb14d9cccb195eb50 P 7a422c36ebff4c2384978dbc5c2b9afc: Neither blocks nor log segments found. Creating new log.
I20260923 22:12:54.394810 9578 tablet_bootstrap.cc:654] T 5e1cdb94d0b4481bb14d9cccb195eb50 P f36bf50226164a81aa65fea3b9d4c07d: Neither blocks nor log segments found. Creating new log.
I20260923 22:12:54.395354 9578 tablet_bootstrap.cc:492] T 5e1cdb94d0b4481bb14d9cccb195eb50 P f36bf50226164a81aa65fea3b9d4c07d: No bootstrap required, opened a new log
I20260923 22:12:54.395396 9578 ts_tablet_manager.cc:1397] T 5e1cdb94d0b4481bb14d9cccb195eb50 P f36bf50226164a81aa65fea3b9d4c07d: Time spent bootstrapping tablet: real 0.001s user 0.001s sys 0.000s
I20260923 22:12:54.395488 9578 raft_consensus.cc:348] T 5e1cdb94d0b4481bb14d9cccb195eb50 P f36bf50226164a81aa65fea3b9d4c07d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "07273d7511d54b88b18936d1c8590b03" member_type: VOTER last_known_addr { host: "127.1.135.193" port: 33215 } } peers { permanent_uuid: "7a422c36ebff4c2384978dbc5c2b9afc" member_type: VOTER last_known_addr { host: "127.1.135.194" port: 46503 } } peers { permanent_uuid: "f36bf50226164a81aa65fea3b9d4c07d" member_type: VOTER last_known_addr { host: "127.1.135.195" port: 40361 } }
I20260923 22:12:54.395560 9578 raft_consensus.cc:374] T 5e1cdb94d0b4481bb14d9cccb195eb50 P f36bf50226164a81aa65fea3b9d4c07d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260923 22:12:54.395581 9578 raft_consensus.cc:729] T 5e1cdb94d0b4481bb14d9cccb195eb50 P f36bf50226164a81aa65fea3b9d4c07d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: f36bf50226164a81aa65fea3b9d4c07d, State: Initialized, Role: FOLLOWER
I20260923 22:12:54.395629 9578 consensus_queue.cc:260] T 5e1cdb94d0b4481bb14d9cccb195eb50 P f36bf50226164a81aa65fea3b9d4c07d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "07273d7511d54b88b18936d1c8590b03" member_type: VOTER last_known_addr { host: "127.1.135.193" port: 33215 } } peers { permanent_uuid: "7a422c36ebff4c2384978dbc5c2b9afc" member_type: VOTER last_known_addr { host: "127.1.135.194" port: 46503 } } peers { permanent_uuid: "f36bf50226164a81aa65fea3b9d4c07d" member_type: VOTER last_known_addr { host: "127.1.135.195" port: 40361 } }
I20260923 22:12:54.395651 9579 tablet_bootstrap.cc:492] T 5e1cdb94d0b4481bb14d9cccb195eb50 P 07273d7511d54b88b18936d1c8590b03: Bootstrap starting.
I20260923 22:12:54.395710 9578 ts_tablet_manager.cc:1428] T 5e1cdb94d0b4481bb14d9cccb195eb50 P f36bf50226164a81aa65fea3b9d4c07d: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20260923 22:12:54.395926 9579 tablet_bootstrap.cc:654] T 5e1cdb94d0b4481bb14d9cccb195eb50 P 07273d7511d54b88b18936d1c8590b03: Neither blocks nor log segments found. Creating new log.
I20260923 22:12:54.395918 9052 catalog_manager.cc:4841] TS 7a422c36ebff4c2384978dbc5c2b9afc (127.1.135.194:46503): tablet 89768fbac223427bbdeb22824fb8248c (table test-table [id=44b462cee4034ee79bb2a13aef3f1c79]) successfully deleted
I20260923 22:12:54.396416 9577 tablet_bootstrap.cc:492] T 5e1cdb94d0b4481bb14d9cccb195eb50 P 7a422c36ebff4c2384978dbc5c2b9afc: No bootstrap required, opened a new log
I20260923 22:12:54.396495 9577 ts_tablet_manager.cc:1397] T 5e1cdb94d0b4481bb14d9cccb195eb50 P 7a422c36ebff4c2384978dbc5c2b9afc: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20260923 22:12:54.396677 9577 raft_consensus.cc:348] T 5e1cdb94d0b4481bb14d9cccb195eb50 P 7a422c36ebff4c2384978dbc5c2b9afc [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "07273d7511d54b88b18936d1c8590b03" member_type: VOTER last_known_addr { host: "127.1.135.193" port: 33215 } } peers { permanent_uuid: "7a422c36ebff4c2384978dbc5c2b9afc" member_type: VOTER last_known_addr { host: "127.1.135.194" port: 46503 } } peers { permanent_uuid: "f36bf50226164a81aa65fea3b9d4c07d" member_type: VOTER last_known_addr { host: "127.1.135.195" port: 40361 } }
I20260923 22:12:54.396759 9577 raft_consensus.cc:374] T 5e1cdb94d0b4481bb14d9cccb195eb50 P 7a422c36ebff4c2384978dbc5c2b9afc [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260923 22:12:54.396786 9577 raft_consensus.cc:729] T 5e1cdb94d0b4481bb14d9cccb195eb50 P 7a422c36ebff4c2384978dbc5c2b9afc [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 7a422c36ebff4c2384978dbc5c2b9afc, State: Initialized, Role: FOLLOWER
I20260923 22:12:54.396831 9577 consensus_queue.cc:260] T 5e1cdb94d0b4481bb14d9cccb195eb50 P 7a422c36ebff4c2384978dbc5c2b9afc [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "07273d7511d54b88b18936d1c8590b03" member_type: VOTER last_known_addr { host: "127.1.135.193" port: 33215 } } peers { permanent_uuid: "7a422c36ebff4c2384978dbc5c2b9afc" member_type: VOTER last_known_addr { host: "127.1.135.194" port: 46503 } } peers { permanent_uuid: "f36bf50226164a81aa65fea3b9d4c07d" member_type: VOTER last_known_addr { host: "127.1.135.195" port: 40361 } }
I20260923 22:12:54.396934 9577 ts_tablet_manager.cc:1428] T 5e1cdb94d0b4481bb14d9cccb195eb50 P 7a422c36ebff4c2384978dbc5c2b9afc: Time spent starting tablet: real 0.000s user 0.001s sys 0.000s
I20260923 22:12:54.397734 9579 tablet_bootstrap.cc:492] T 5e1cdb94d0b4481bb14d9cccb195eb50 P 07273d7511d54b88b18936d1c8590b03: No bootstrap required, opened a new log
I20260923 22:12:54.397787 9579 ts_tablet_manager.cc:1397] T 5e1cdb94d0b4481bb14d9cccb195eb50 P 07273d7511d54b88b18936d1c8590b03: Time spent bootstrapping tablet: real 0.002s user 0.000s sys 0.000s
I20260923 22:12:54.397893 9579 raft_consensus.cc:348] T 5e1cdb94d0b4481bb14d9cccb195eb50 P 07273d7511d54b88b18936d1c8590b03 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "07273d7511d54b88b18936d1c8590b03" member_type: VOTER last_known_addr { host: "127.1.135.193" port: 33215 } } peers { permanent_uuid: "7a422c36ebff4c2384978dbc5c2b9afc" member_type: VOTER last_known_addr { host: "127.1.135.194" port: 46503 } } peers { permanent_uuid: "f36bf50226164a81aa65fea3b9d4c07d" member_type: VOTER last_known_addr { host: "127.1.135.195" port: 40361 } }
I20260923 22:12:54.397961 9579 raft_consensus.cc:374] T 5e1cdb94d0b4481bb14d9cccb195eb50 P 07273d7511d54b88b18936d1c8590b03 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260923 22:12:54.397986 9579 raft_consensus.cc:729] T 5e1cdb94d0b4481bb14d9cccb195eb50 P 07273d7511d54b88b18936d1c8590b03 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 07273d7511d54b88b18936d1c8590b03, State: Initialized, Role: FOLLOWER
I20260923 22:12:54.398025 9579 consensus_queue.cc:260] T 5e1cdb94d0b4481bb14d9cccb195eb50 P 07273d7511d54b88b18936d1c8590b03 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "07273d7511d54b88b18936d1c8590b03" member_type: VOTER last_known_addr { host: "127.1.135.193" port: 33215 } } peers { permanent_uuid: "7a422c36ebff4c2384978dbc5c2b9afc" member_type: VOTER last_known_addr { host: "127.1.135.194" port: 46503 } } peers { permanent_uuid: "f36bf50226164a81aa65fea3b9d4c07d" member_type: VOTER last_known_addr { host: "127.1.135.195" port: 40361 } }
I20260923 22:12:54.398113 9579 ts_tablet_manager.cc:1428] T 5e1cdb94d0b4481bb14d9cccb195eb50 P 07273d7511d54b88b18936d1c8590b03: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20260923 22:12:54.565291 9595 raft_consensus.cc:482] T 5e1cdb94d0b4481bb14d9cccb195eb50 P f36bf50226164a81aa65fea3b9d4c07d [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260923 22:12:54.565419 9595 raft_consensus.cc:504] T 5e1cdb94d0b4481bb14d9cccb195eb50 P f36bf50226164a81aa65fea3b9d4c07d [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "07273d7511d54b88b18936d1c8590b03" member_type: VOTER last_known_addr { host: "127.1.135.193" port: 33215 } } peers { permanent_uuid: "7a422c36ebff4c2384978dbc5c2b9afc" member_type: VOTER last_known_addr { host: "127.1.135.194" port: 46503 } } peers { permanent_uuid: "f36bf50226164a81aa65fea3b9d4c07d" member_type: VOTER last_known_addr { host: "127.1.135.195" port: 40361 } }
I20260923 22:12:54.565634 9595 leader_election.cc:288] T 5e1cdb94d0b4481bb14d9cccb195eb50 P f36bf50226164a81aa65fea3b9d4c07d [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 07273d7511d54b88b18936d1c8590b03 (127.1.135.193:33215), 7a422c36ebff4c2384978dbc5c2b9afc (127.1.135.194:46503)
I20260923 22:12:54.565851 9363 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "5e1cdb94d0b4481bb14d9cccb195eb50" candidate_uuid: "f36bf50226164a81aa65fea3b9d4c07d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7a422c36ebff4c2384978dbc5c2b9afc" is_pre_election: true
I20260923 22:12:54.565850 9223 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "5e1cdb94d0b4481bb14d9cccb195eb50" candidate_uuid: "f36bf50226164a81aa65fea3b9d4c07d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "07273d7511d54b88b18936d1c8590b03" is_pre_election: true
I20260923 22:12:54.565933 9363 raft_consensus.cc:2454] T 5e1cdb94d0b4481bb14d9cccb195eb50 P 7a422c36ebff4c2384978dbc5c2b9afc [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f36bf50226164a81aa65fea3b9d4c07d in term 0.
I20260923 22:12:54.565987 9223 raft_consensus.cc:2454] T 5e1cdb94d0b4481bb14d9cccb195eb50 P 07273d7511d54b88b18936d1c8590b03 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f36bf50226164a81aa65fea3b9d4c07d in term 0.
I20260923 22:12:54.566133 9436 leader_election.cc:302] T 5e1cdb94d0b4481bb14d9cccb195eb50 P f36bf50226164a81aa65fea3b9d4c07d [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 7a422c36ebff4c2384978dbc5c2b9afc, f36bf50226164a81aa65fea3b9d4c07d; no voters:
I20260923 22:12:54.566242 9595 raft_consensus.cc:2781] T 5e1cdb94d0b4481bb14d9cccb195eb50 P f36bf50226164a81aa65fea3b9d4c07d [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260923 22:12:54.566294 9595 raft_consensus.cc:482] T 5e1cdb94d0b4481bb14d9cccb195eb50 P f36bf50226164a81aa65fea3b9d4c07d [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260923 22:12:54.566318 9595 raft_consensus.cc:3037] T 5e1cdb94d0b4481bb14d9cccb195eb50 P f36bf50226164a81aa65fea3b9d4c07d [term 0 FOLLOWER]: Advancing to term 1
I20260923 22:12:54.566994 9595 raft_consensus.cc:504] T 5e1cdb94d0b4481bb14d9cccb195eb50 P f36bf50226164a81aa65fea3b9d4c07d [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "07273d7511d54b88b18936d1c8590b03" member_type: VOTER last_known_addr { host: "127.1.135.193" port: 33215 } } peers { permanent_uuid: "7a422c36ebff4c2384978dbc5c2b9afc" member_type: VOTER last_known_addr { host: "127.1.135.194" port: 46503 } } peers { permanent_uuid: "f36bf50226164a81aa65fea3b9d4c07d" member_type: VOTER last_known_addr { host: "127.1.135.195" port: 40361 } }
I20260923 22:12:54.567155 9595 leader_election.cc:288] T 5e1cdb94d0b4481bb14d9cccb195eb50 P f36bf50226164a81aa65fea3b9d4c07d [CANDIDATE]: Term 1 election: Requested vote from peers 07273d7511d54b88b18936d1c8590b03 (127.1.135.193:33215), 7a422c36ebff4c2384978dbc5c2b9afc (127.1.135.194:46503)
I20260923 22:12:54.567289 9223 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "5e1cdb94d0b4481bb14d9cccb195eb50" candidate_uuid: "f36bf50226164a81aa65fea3b9d4c07d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "07273d7511d54b88b18936d1c8590b03"
I20260923 22:12:54.567299 9363 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "5e1cdb94d0b4481bb14d9cccb195eb50" candidate_uuid: "f36bf50226164a81aa65fea3b9d4c07d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7a422c36ebff4c2384978dbc5c2b9afc"
I20260923 22:12:54.567368 9223 raft_consensus.cc:3037] T 5e1cdb94d0b4481bb14d9cccb195eb50 P 07273d7511d54b88b18936d1c8590b03 [term 0 FOLLOWER]: Advancing to term 1
I20260923 22:12:54.567389 9363 raft_consensus.cc:3037] T 5e1cdb94d0b4481bb14d9cccb195eb50 P 7a422c36ebff4c2384978dbc5c2b9afc [term 0 FOLLOWER]: Advancing to term 1
I20260923 22:12:54.567972 9223 raft_consensus.cc:2454] T 5e1cdb94d0b4481bb14d9cccb195eb50 P 07273d7511d54b88b18936d1c8590b03 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f36bf50226164a81aa65fea3b9d4c07d in term 1.
I20260923 22:12:54.567976 9363 raft_consensus.cc:2454] T 5e1cdb94d0b4481bb14d9cccb195eb50 P 7a422c36ebff4c2384978dbc5c2b9afc [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f36bf50226164a81aa65fea3b9d4c07d in term 1.
I20260923 22:12:54.568159 9436 leader_election.cc:302] T 5e1cdb94d0b4481bb14d9cccb195eb50 P f36bf50226164a81aa65fea3b9d4c07d [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 7a422c36ebff4c2384978dbc5c2b9afc, f36bf50226164a81aa65fea3b9d4c07d; no voters:
I20260923 22:12:54.568292 9595 raft_consensus.cc:2781] T 5e1cdb94d0b4481bb14d9cccb195eb50 P f36bf50226164a81aa65fea3b9d4c07d [term 1 FOLLOWER]: Leader election won for term 1
I20260923 22:12:54.568383 9595 raft_consensus.cc:686] T 5e1cdb94d0b4481bb14d9cccb195eb50 P f36bf50226164a81aa65fea3b9d4c07d [term 1 LEADER]: Becoming Leader. State: Replica: f36bf50226164a81aa65fea3b9d4c07d, State: Running, Role: LEADER
I20260923 22:12:54.568446 9595 consensus_queue.cc:237] T 5e1cdb94d0b4481bb14d9cccb195eb50 P f36bf50226164a81aa65fea3b9d4c07d [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "07273d7511d54b88b18936d1c8590b03" member_type: VOTER last_known_addr { host: "127.1.135.193" port: 33215 } } peers { permanent_uuid: "7a422c36ebff4c2384978dbc5c2b9afc" member_type: VOTER last_known_addr { host: "127.1.135.194" port: 46503 } } peers { permanent_uuid: "f36bf50226164a81aa65fea3b9d4c07d" member_type: VOTER last_known_addr { host: "127.1.135.195" port: 40361 } }
I20260923 22:12:54.568979 9063 catalog_manager.cc:5495] T 5e1cdb94d0b4481bb14d9cccb195eb50 P f36bf50226164a81aa65fea3b9d4c07d reported cstate change: term changed from 0 to 1, leader changed from <none> to f36bf50226164a81aa65fea3b9d4c07d (127.1.135.195). New cstate: current_term: 1 leader_uuid: "f36bf50226164a81aa65fea3b9d4c07d" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "07273d7511d54b88b18936d1c8590b03" member_type: VOTER last_known_addr { host: "127.1.135.193" port: 33215 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "7a422c36ebff4c2384978dbc5c2b9afc" member_type: VOTER last_known_addr { host: "127.1.135.194" port: 46503 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "f36bf50226164a81aa65fea3b9d4c07d" member_type: VOTER last_known_addr { host: "127.1.135.195" port: 40361 } health_report { overall_health: HEALTHY } } }
I20260923 22:12:54.581493 9576 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 22:12:54.579511 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58736 (local address 127.1.135.195:40361)
0923 22:12:54.579564 (+ 53us) server_negotiation.cc:206] Beginning negotiation
0923 22:12:54.579566 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0923 22:12:54.579575 (+ 9us) server_negotiation.cc:407] Connection header received
0923 22:12:54.579603 (+ 28us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 22:12:54.579603 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 22:12:54.579618 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 22:12:54.579678 (+ 60us) server_negotiation.cc:226] Negotiated authn=TOKEN
0923 22:12:54.579807 (+ 129us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 22:12:54.580331 (+ 524us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 22:12:54.580969 (+ 638us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 22:12:54.581146 (+ 177us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 22:12:54.581197 (+ 51us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 22:12:54.581302 (+ 105us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0923 22:12:54.581326 (+ 24us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0923 22:12:54.581362 (+ 36us) server_negotiation.cc:1035] Waiting for connection context
0923 22:12:54.581407 (+ 45us) server_negotiation.cc:299] Negotiation successful
0923 22:12:54.581422 (+ 15us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":33}
I20260923 22:12:54.585884 9663 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 22:12:54.583602 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.135.193:33215 (local address 127.1.135.195:37729)
0923 22:12:54.583796 (+ 194us) negotiation.cc:107] Waiting for socket to connect
0923 22:12:54.583804 (+ 8us) client_negotiation.cc:174] Beginning negotiation
0923 22:12:54.583851 (+ 47us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0923 22:12:54.583954 (+ 103us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0923 22:12:54.583955 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0923 22:12:54.583956 (+ 1us) client_negotiation.cc:189] Negotiated authn=TOKEN
0923 22:12:54.584000 (+ 44us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 22:12:54.584001 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 22:12:54.584622 (+ 621us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 22:12:54.584623 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 22:12:54.585171 (+ 548us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 22:12:54.585172 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 22:12:54.585483 (+ 311us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 22:12:54.585484 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 22:12:54.585552 (+ 68us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 22:12:54.585564 (+ 12us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0923 22:12:54.585754 (+ 190us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0923 22:12:54.585755 (+ 1us) client_negotiation.cc:769] Sending connection context
0923 22:12:54.585788 (+ 33us) client_negotiation.cc:240] Negotiation successful
0923 22:12:54.585825 (+ 37us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":163,"thread_start_us":85,"threads_started":1}
I20260923 22:12:54.585883 9573 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 22:12:54.583719 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.135.195:37729 (local address 127.1.135.193:33215)
0923 22:12:54.583780 (+ 61us) server_negotiation.cc:206] Beginning negotiation
0923 22:12:54.583783 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0923 22:12:54.583841 (+ 58us) server_negotiation.cc:407] Connection header received
0923 22:12:54.583890 (+ 49us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 22:12:54.583891 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 22:12:54.583899 (+ 8us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 22:12:54.583950 (+ 51us) server_negotiation.cc:226] Negotiated authn=TOKEN
0923 22:12:54.584043 (+ 93us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 22:12:54.584557 (+ 514us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 22:12:54.585242 (+ 685us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 22:12:54.585418 (+ 176us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 22:12:54.585470 (+ 52us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 22:12:54.585656 (+ 186us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0923 22:12:54.585685 (+ 29us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0923 22:12:54.585752 (+ 67us) server_negotiation.cc:1035] Waiting for connection context
0923 22:12:54.585792 (+ 40us) server_negotiation.cc:299] Negotiation successful
0923 22:12:54.585812 (+ 20us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":36}
I20260923 22:12:54.587766 9223 raft_consensus.cc:1261] T 5e1cdb94d0b4481bb14d9cccb195eb50 P 07273d7511d54b88b18936d1c8590b03 [term 1 FOLLOWER]: Refusing update from remote peer f36bf50226164a81aa65fea3b9d4c07d: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260923 22:12:54.587767 9363 raft_consensus.cc:1261] T 5e1cdb94d0b4481bb14d9cccb195eb50 P 7a422c36ebff4c2384978dbc5c2b9afc [term 1 FOLLOWER]: Refusing update from remote peer f36bf50226164a81aa65fea3b9d4c07d: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260923 22:12:54.587971 9659 consensus_queue.cc:1035] T 5e1cdb94d0b4481bb14d9cccb195eb50 P f36bf50226164a81aa65fea3b9d4c07d [LEADER]: Connected to new peer: Peer: permanent_uuid: "7a422c36ebff4c2384978dbc5c2b9afc" member_type: VOTER last_known_addr { host: "127.1.135.194" port: 46503 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260923 22:12:54.588127 9595 consensus_queue.cc:1035] T 5e1cdb94d0b4481bb14d9cccb195eb50 P f36bf50226164a81aa65fea3b9d4c07d [LEADER]: Connected to new peer: Peer: permanent_uuid: "07273d7511d54b88b18936d1c8590b03" member_type: VOTER last_known_addr { host: "127.1.135.193" port: 33215 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260923 22:12:54.588991 9590 mvcc.cc:205] Tried to move back new op lower bound from 7332665649510240256 to 7332665649432674304. Current Snapshot: MvccSnapshot[applied={T|T < 7332665649510240256}]
I20260923 22:12:54.594156 9576 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 22:12:54.592118 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.135.193:50519 (local address 127.1.135.195:40361)
0923 22:12:54.592150 (+ 32us) server_negotiation.cc:206] Beginning negotiation
0923 22:12:54.592152 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0923 22:12:54.592159 (+ 7us) server_negotiation.cc:407] Connection header received
0923 22:12:54.592224 (+ 65us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 22:12:54.592225 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 22:12:54.592234 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 22:12:54.592282 (+ 48us) server_negotiation.cc:226] Negotiated authn=TOKEN
0923 22:12:54.592377 (+ 95us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 22:12:54.593006 (+ 629us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 22:12:54.593647 (+ 641us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 22:12:54.593806 (+ 159us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 22:12:54.593850 (+ 44us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 22:12:54.593947 (+ 97us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0923 22:12:54.593967 (+ 20us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0923 22:12:54.593998 (+ 31us) server_negotiation.cc:1035] Waiting for connection context
0923 22:12:54.594072 (+ 74us) server_negotiation.cc:299] Negotiation successful
0923 22:12:54.594089 (+ 17us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":16}
I20260923 22:12:54.594173 9627 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 22:12:54.592046 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.135.195:40361 (local address 127.1.135.193:50519)
0923 22:12:54.592125 (+ 79us) negotiation.cc:107] Waiting for socket to connect
0923 22:12:54.592135 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0923 22:12:54.592173 (+ 38us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0923 22:12:54.592286 (+ 113us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0923 22:12:54.592286 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0923 22:12:54.592287 (+ 1us) client_negotiation.cc:189] Negotiated authn=TOKEN
0923 22:12:54.592324 (+ 37us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 22:12:54.592325 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 22:12:54.593073 (+ 748us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 22:12:54.593074 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 22:12:54.593579 (+ 505us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 22:12:54.593581 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 22:12:54.593863 (+ 282us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 22:12:54.593864 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 22:12:54.593902 (+ 38us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 22:12:54.593907 (+ 5us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0923 22:12:54.594013 (+ 106us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0923 22:12:54.594014 (+ 1us) client_negotiation.cc:769] Sending connection context
0923 22:12:54.594057 (+ 43us) client_negotiation.cc:240] Negotiation successful
0923 22:12:54.594096 (+ 39us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":57}
I20260923 22:12:54.600816 9063 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:48922:
table { table_name: "test-table" } modify_external_catalogs: true
I20260923 22:12:54.600922 9063 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:48922:
table { table_name: "test-table" } modify_external_catalogs: true
I20260923 22:12:54.601550 9063 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 53d3bf4260ad47b69a41891f4fa3a6e5: Sending DeleteTablet for 3 replicas of tablet 5e1cdb94d0b4481bb14d9cccb195eb50
I20260923 22:12:54.601790 9483 tablet_service.cc:1514] Processing DeleteTablet for tablet 5e1cdb94d0b4481bb14d9cccb195eb50 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-23 22:12:54 UTC) from {username='oryx', principal='oryx/127.1.135.254@KRBTEST.COM'} at 127.0.0.1:34286
I20260923 22:12:54.601801 9343 tablet_service.cc:1514] Processing DeleteTablet for tablet 5e1cdb94d0b4481bb14d9cccb195eb50 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-23 22:12:54 UTC) from {username='oryx', principal='oryx/127.1.135.254@KRBTEST.COM'} at 127.0.0.1:32868
I20260923 22:12:54.601893 9600 tablet_replica.cc:330] T 5e1cdb94d0b4481bb14d9cccb195eb50 P 7a422c36ebff4c2384978dbc5c2b9afc: stopping tablet replica
I20260923 22:12:54.601890 9602 tablet_replica.cc:330] T 5e1cdb94d0b4481bb14d9cccb195eb50 P f36bf50226164a81aa65fea3b9d4c07d: stopping tablet replica
I20260923 22:12:54.601944 9600 raft_consensus.cc:2229] T 5e1cdb94d0b4481bb14d9cccb195eb50 P 7a422c36ebff4c2384978dbc5c2b9afc [term 1 FOLLOWER]: Raft consensus shutting down.
I20260923 22:12:54.601943 9602 raft_consensus.cc:2229] T 5e1cdb94d0b4481bb14d9cccb195eb50 P f36bf50226164a81aa65fea3b9d4c07d [term 1 LEADER]: Raft consensus shutting down.
I20260923 22:12:54.601994 9602 raft_consensus.cc:2258] T 5e1cdb94d0b4481bb14d9cccb195eb50 P f36bf50226164a81aa65fea3b9d4c07d [term 1 FOLLOWER]: Raft consensus is shut down!
I20260923 22:12:54.602005 9600 raft_consensus.cc:2258] T 5e1cdb94d0b4481bb14d9cccb195eb50 P 7a422c36ebff4c2384978dbc5c2b9afc [term 1 FOLLOWER]: Raft consensus is shut down!
I20260923 22:12:54.602155 9203 tablet_service.cc:1514] Processing DeleteTablet for tablet 5e1cdb94d0b4481bb14d9cccb195eb50 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-23 22:12:54 UTC) from {username='oryx', principal='oryx/127.1.135.254@KRBTEST.COM'} at 127.0.0.1:40636
I20260923 22:12:54.602243 9601 tablet_replica.cc:330] T 5e1cdb94d0b4481bb14d9cccb195eb50 P 07273d7511d54b88b18936d1c8590b03: stopping tablet replica
I20260923 22:12:54.602260 9602 ts_tablet_manager.cc:1905] T 5e1cdb94d0b4481bb14d9cccb195eb50 P f36bf50226164a81aa65fea3b9d4c07d: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260923 22:12:54.602303 9601 raft_consensus.cc:2229] T 5e1cdb94d0b4481bb14d9cccb195eb50 P 07273d7511d54b88b18936d1c8590b03 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260923 22:12:54.602336 9601 raft_consensus.cc:2258] T 5e1cdb94d0b4481bb14d9cccb195eb50 P 07273d7511d54b88b18936d1c8590b03 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260923 22:12:54.602339 1567 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskRiQe_a/build/release/bin/kudu with pid 9138
I20260923 22:12:54.604125 9602 ts_tablet_manager.cc:1918] T 5e1cdb94d0b4481bb14d9cccb195eb50 P f36bf50226164a81aa65fea3b9d4c07d: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260923 22:12:54.604187 9602 log.cc:1196] T 5e1cdb94d0b4481bb14d9cccb195eb50 P f36bf50226164a81aa65fea3b9d4c07d: Deleting WAL directory at /tmp/dist-test-taskRiQe_a/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790201539513534-1567-0/minicluster-data/ts-2/wal/wals/5e1cdb94d0b4481bb14d9cccb195eb50
I20260923 22:12:54.604386 9602 ts_tablet_manager.cc:1939] T 5e1cdb94d0b4481bb14d9cccb195eb50 P f36bf50226164a81aa65fea3b9d4c07d: Deleting consensus metadata
I20260923 22:12:54.604676 9600 ts_tablet_manager.cc:1905] T 5e1cdb94d0b4481bb14d9cccb195eb50 P 7a422c36ebff4c2384978dbc5c2b9afc: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260923 22:12:54.604719 9051 catalog_manager.cc:4841] TS f36bf50226164a81aa65fea3b9d4c07d (127.1.135.195:40361): tablet 5e1cdb94d0b4481bb14d9cccb195eb50 (table test-table [id=24dee7017ecf4a468b1213ea485b4fcf]) successfully deleted
I20260923 22:12:54.605851 9600 ts_tablet_manager.cc:1918] T 5e1cdb94d0b4481bb14d9cccb195eb50 P 7a422c36ebff4c2384978dbc5c2b9afc: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260923 22:12:54.605914 9600 log.cc:1196] T 5e1cdb94d0b4481bb14d9cccb195eb50 P 7a422c36ebff4c2384978dbc5c2b9afc: Deleting WAL directory at /tmp/dist-test-taskRiQe_a/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790201539513534-1567-0/minicluster-data/ts-1/wal/wals/5e1cdb94d0b4481bb14d9cccb195eb50
I20260923 22:12:54.606124 9600 ts_tablet_manager.cc:1939] T 5e1cdb94d0b4481bb14d9cccb195eb50 P 7a422c36ebff4c2384978dbc5c2b9afc: Deleting consensus metadata
I20260923 22:12:54.606422 9052 catalog_manager.cc:4841] TS 7a422c36ebff4c2384978dbc5c2b9afc (127.1.135.194:46503): tablet 5e1cdb94d0b4481bb14d9cccb195eb50 (table test-table [id=24dee7017ecf4a468b1213ea485b4fcf]) successfully deleted
W20260923 22:12:54.610410 9051 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.1.135.193:33215 (error 108)
I20260923 22:12:54.610708 1567 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskRiQe_a/build/release/bin/kudu with pid 9278
I20260923 22:12:54.611735 9668 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 22:12:54.610770 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.135.193:33215 (local address 127.0.0.1:51534)
0923 22:12:54.611664 (+ 894us) negotiation.cc:107] Waiting for socket to connect
0923 22:12:54.611706 (+ 42us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.135.193:33215: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":879,"thread_start_us":36,"threads_started":1}
W20260923 22:12:54.611842 9051 catalog_manager.cc:4584] TS 07273d7511d54b88b18936d1c8590b03 (127.1.135.193:33215): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet 5e1cdb94d0b4481bb14d9cccb195eb50: Network error: Client connection negotiation failed: client connection to 127.1.135.193:33215: connect: Connection refused (error 111)
I20260923 22:12:54.616751 1567 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskRiQe_a/build/release/bin/kudu with pid 9418
I20260923 22:12:54.622300 1567 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskRiQe_a/build/release/bin/kudu with pid 9034
2026-09-23T22:12:54Z chronyd exiting
[ OK ] SecurityITest.TestNonDefaultPrincipal (2750 ms)
[ RUN ] SecurityITest.TestNonExistentPrincipal
Loading random data
Initializing database '/tmp/dist-test-taskRiQe_a/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1790201539513534-1567-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Sep 23 22:12:54 dist-test-slave-s2fj krb5kdc[9671](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Sep 23 22:12:54 dist-test-slave-s2fj krb5kdc[9671](info): set up 2 sockets
Sep 23 22:12:54 dist-test-slave-s2fj krb5kdc[9671](info): commencing operation
krb5kdc: starting...
W20260923 22:12:55.652606 1567 mini_kdc.cc:121] Time spent starting KDC: real 1.010s user 0.003s sys 0.002s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskRiQe_a/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1790201539513534-1567-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskRiQe_a/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1790201539513534-1567-0/krb5kdc/test-user.keytab.
Sep 23 22:12:55 dist-test-slave-s2fj krb5kdc[9671](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790201575, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-09-23T22:12:55Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-09-23T22:12:55Z Disabled control of system clock
WARNING: no policy specified for kudu/127.1.135.254@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.1.135.254@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.1.135.254 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskRiQe_a/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1790201539513534-1567-0/krb5kdc/kudu_127.1.135.254.keytab.
Entry for principal kudu/127.1.135.254 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskRiQe_a/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1790201539513534-1567-0/krb5kdc/kudu_127.1.135.254.keytab.
I20260923 22:12:55.763115 1567 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskRiQe_a/build/release/bin/kudu
/tmp/dist-test-taskRiQe_a/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskRiQe_a/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1790201539513534-1567-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskRiQe_a/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1790201539513534-1567-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskRiQe_a/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1790201539513534-1567-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskRiQe_a/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1790201539513534-1567-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.1.135.254:43645
--webserver_interface=127.1.135.254
--webserver_port=0
--keytab_file=/tmp/dist-test-taskRiQe_a/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1790201539513534-1567-0/krb5kdc/kudu_127.1.135.254.keytab
--principal=kudu/127.1.135.254
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.1.135.212:35843
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.1.135.254:43645
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--principal=oryx with env {KRB5CCNAME=/tmp/dist-test-taskRiQe_a/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1790201539513534-1567-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskRiQe_a/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1790201539513534-1567-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskRiQe_a/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1790201539513534-1567-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260923 22:12:55.840695 9684 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260923 22:12:55.840889 9684 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260923 22:12:55.840911 9684 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260923 22:12:55.842072 9684 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260923 22:12:55.842113 9684 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260923 22:12:55.842128 9684 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260923 22:12:55.842163 9684 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260923 22:12:55.842191 9684 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260923 22:12:55.843686 9684 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.1.135.212:35843
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskRiQe_a/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1790201539513534-1567-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskRiQe_a/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1790201539513534-1567-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.1.135.254:43645
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskRiQe_a/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1790201539513534-1567-0/krb5kdc/kudu_127.1.135.254.keytab
--principal=oryx
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.1.135.254:43645
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskRiQe_a/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1790201539513534-1567-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.1.135.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.9684
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskRiQe_a/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1790201539513534-1567-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 23 Sep 2026 20:53:37 UTC on ccycloud.quasar-rkviyx.root.comops.site
build id 4668083
I20260923 22:12:55.843936 9684 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260923 22:12:55.844183 9684 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260923 22:12:55.848763 9692 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260923 22:12:55.848769 9689 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260923 22:12:55.848774 9690 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260923 22:12:55.848984 9684 server_base.cc:1034] running on GCE node
I20260923 22:12:55.849237 9684 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260923 22:12:55.849538 9684 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260923 22:12:55.850701 9684 hybrid_clock.cc:648] HybridClock initialized: now 1790201575850683 us; error 29 us; skew 500 ppm
Runtime error: kinit failed: unable to login from keytab: Keytab contains no suitable keys for oryx@KRBTEST.COM
2026-09-23T22:12:55Z chronyd exiting
[ OK ] SecurityITest.TestNonExistentPrincipal (1237 ms)
[ RUN ] SecurityITest.TestMismatchingPrincipals
Loading random data
Initializing database '/tmp/dist-test-taskRiQe_a/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790201539513534-1567-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Sep 23 22:12:55 dist-test-slave-s2fj krb5kdc[9697](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Sep 23 22:12:55 dist-test-slave-s2fj krb5kdc[9697](info): set up 2 sockets
Sep 23 22:12:55 dist-test-slave-s2fj krb5kdc[9697](info): commencing operation
krb5kdc: starting...
W20260923 22:12:56.904616 1567 mini_kdc.cc:121] Time spent starting KDC: real 1.024s user 0.002s sys 0.003s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskRiQe_a/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790201539513534-1567-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskRiQe_a/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790201539513534-1567-0/krb5kdc/test-user.keytab.
Sep 23 22:12:56 dist-test-slave-s2fj krb5kdc[9697](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790201576, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-09-23T22:12:56Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-09-23T22:12:56Z Disabled control of system clock
WARNING: no policy specified for kudu/127.1.135.254@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.1.135.254@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.1.135.254 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskRiQe_a/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790201539513534-1567-0/krb5kdc/kudu_127.1.135.254.keytab.
Entry for principal kudu/127.1.135.254 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskRiQe_a/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790201539513534-1567-0/krb5kdc/kudu_127.1.135.254.keytab.
I20260923 22:12:57.018234 1567 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskRiQe_a/build/release/bin/kudu
/tmp/dist-test-taskRiQe_a/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskRiQe_a/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790201539513534-1567-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskRiQe_a/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790201539513534-1567-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskRiQe_a/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790201539513534-1567-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskRiQe_a/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790201539513534-1567-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.1.135.254:38637
--webserver_interface=127.1.135.254
--webserver_port=0
--keytab_file=/tmp/dist-test-taskRiQe_a/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790201539513534-1567-0/krb5kdc/kudu_127.1.135.254.keytab
--principal=kudu/127.1.135.254
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.1.135.212:36077
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.1.135.254:38637
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskRiQe_a/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790201539513534-1567-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskRiQe_a/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790201539513534-1567-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskRiQe_a/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790201539513534-1567-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260923 22:12:57.096869 9710 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260923 22:12:57.097062 9710 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260923 22:12:57.097082 9710 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260923 22:12:57.098237 9710 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260923 22:12:57.098280 9710 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260923 22:12:57.098295 9710 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260923 22:12:57.098307 9710 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260923 22:12:57.098357 9710 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260923 22:12:57.099757 9710 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.1.135.212:36077
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskRiQe_a/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790201539513534-1567-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskRiQe_a/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790201539513534-1567-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.1.135.254:38637
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskRiQe_a/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790201539513534-1567-0/krb5kdc/kudu_127.1.135.254.keytab
--principal=kudu/127.1.135.254
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.1.135.254:38637
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskRiQe_a/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790201539513534-1567-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.1.135.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.9710
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskRiQe_a/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790201539513534-1567-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 23 Sep 2026 20:53:37 UTC on ccycloud.quasar-rkviyx.root.comops.site
build id 4668083
I20260923 22:12:57.099998 9710 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260923 22:12:57.100239 9710 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260923 22:12:57.104660 9716 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260923 22:12:57.104714 9718 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260923 22:12:57.104725 9715 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260923 22:12:57.104911 9710 server_base.cc:1034] running on GCE node
I20260923 22:12:57.105142 9710 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260923 22:12:57.105445 9710 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260923 22:12:57.106601 9710 hybrid_clock.cc:648] HybridClock initialized: now 1790201577106596 us; error 36 us; skew 500 ppm
Sep 23 22:12:57 dist-test-slave-s2fj krb5kdc[9697](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790201577, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.135.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260923 22:12:57.112483 9710 init.cc:377] Logged in from keytab as kudu/127.1.135.254@KRBTEST.COM (short username kudu)
I20260923 22:12:57.112830 9710 webserver.cc:460] Webserver started at http://127.1.135.254:38727/ using document root <none> and password file <none>
I20260923 22:12:57.113003 9710 fs_manager.cc:359] Metadata directory not provided
I20260923 22:12:57.113042 9710 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260923 22:12:57.113122 9710 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260923 22:12:57.114082 9710 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskRiQe_a/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790201539513534-1567-0/minicluster-data/master-0/data/instance:
uuid: "e156218e8e7e492596e7b347ed8d854b"
format_stamp: "Formatted at 2026-09-23 22:12:57 on dist-test-slave-s2fj"
server_key: "526f8bc9db07a3d4e4662741e5da64d1"
server_key_iv: "126203d9ad107429041fe29da337bb24"
server_key_version: "encryptionkey@0"
I20260923 22:12:57.114372 9710 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskRiQe_a/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790201539513534-1567-0/minicluster-data/master-0/wal/instance:
uuid: "e156218e8e7e492596e7b347ed8d854b"
format_stamp: "Formatted at 2026-09-23 22:12:57 on dist-test-slave-s2fj"
server_key: "526f8bc9db07a3d4e4662741e5da64d1"
server_key_iv: "126203d9ad107429041fe29da337bb24"
server_key_version: "encryptionkey@0"
I20260923 22:12:57.115661 9710 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.000s
I20260923 22:12:57.116336 9725 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260923 22:12:57.116533 9710 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260923 22:12:57.116617 9710 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskRiQe_a/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790201539513534-1567-0/minicluster-data/master-0/data,/tmp/dist-test-taskRiQe_a/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790201539513534-1567-0/minicluster-data/master-0/wal
uuid: "e156218e8e7e492596e7b347ed8d854b"
format_stamp: "Formatted at 2026-09-23 22:12:57 on dist-test-slave-s2fj"
server_key: "526f8bc9db07a3d4e4662741e5da64d1"
server_key_iv: "126203d9ad107429041fe29da337bb24"
server_key_version: "encryptionkey@0"
I20260923 22:12:57.116668 9710 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskRiQe_a/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790201539513534-1567-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskRiQe_a/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790201539513534-1567-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskRiQe_a/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790201539513534-1567-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260923 22:12:57.129441 9710 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260923 22:12:57.131139 9710 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260923 22:12:57.131242 9710 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260923 22:12:57.134841 9710 rpc_server.cc:307] RPC server started. Bound to: 127.1.135.254:38637
I20260923 22:12:57.134932 9777 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.1.135.254:38637 every 8 connection(s)
I20260923 22:12:57.135205 9710 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskRiQe_a/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790201539513534-1567-0/minicluster-data/master-0/data/info.pb
I20260923 22:12:57.136193 9778 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260923 22:12:57.138334 9778 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P e156218e8e7e492596e7b347ed8d854b: Bootstrap starting.
I20260923 22:12:57.138895 9778 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P e156218e8e7e492596e7b347ed8d854b: Neither blocks nor log segments found. Creating new log.
I20260923 22:12:57.139191 9778 log.cc:824] T 00000000000000000000000000000000 P e156218e8e7e492596e7b347ed8d854b: Log is configured to *not* fsync() on all Append() calls
I20260923 22:12:57.139844 9778 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P e156218e8e7e492596e7b347ed8d854b: No bootstrap required, opened a new log
I20260923 22:12:57.141412 9778 raft_consensus.cc:348] T 00000000000000000000000000000000 P e156218e8e7e492596e7b347ed8d854b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e156218e8e7e492596e7b347ed8d854b" member_type: VOTER last_known_addr { host: "127.1.135.254" port: 38637 } }
I20260923 22:12:57.141582 9778 raft_consensus.cc:374] T 00000000000000000000000000000000 P e156218e8e7e492596e7b347ed8d854b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260923 22:12:57.141631 9778 raft_consensus.cc:729] T 00000000000000000000000000000000 P e156218e8e7e492596e7b347ed8d854b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e156218e8e7e492596e7b347ed8d854b, State: Initialized, Role: FOLLOWER
I20260923 22:12:57.141774 9778 consensus_queue.cc:260] T 00000000000000000000000000000000 P e156218e8e7e492596e7b347ed8d854b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e156218e8e7e492596e7b347ed8d854b" member_type: VOTER last_known_addr { host: "127.1.135.254" port: 38637 } }
I20260923 22:12:57.141852 9778 raft_consensus.cc:388] T 00000000000000000000000000000000 P e156218e8e7e492596e7b347ed8d854b [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260923 22:12:57.141896 9778 raft_consensus.cc:482] T 00000000000000000000000000000000 P e156218e8e7e492596e7b347ed8d854b [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260923 22:12:57.141948 9778 raft_consensus.cc:3037] T 00000000000000000000000000000000 P e156218e8e7e492596e7b347ed8d854b [term 0 FOLLOWER]: Advancing to term 1
I20260923 22:12:57.142458 9778 raft_consensus.cc:504] T 00000000000000000000000000000000 P e156218e8e7e492596e7b347ed8d854b [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e156218e8e7e492596e7b347ed8d854b" member_type: VOTER last_known_addr { host: "127.1.135.254" port: 38637 } }
I20260923 22:12:57.142570 9778 leader_election.cc:302] T 00000000000000000000000000000000 P e156218e8e7e492596e7b347ed8d854b [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: e156218e8e7e492596e7b347ed8d854b; no voters:
I20260923 22:12:57.142688 1567 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskRiQe_a/build/release/bin/kudu as pid 9710
I20260923 22:12:57.142755 9778 leader_election.cc:288] T 00000000000000000000000000000000 P e156218e8e7e492596e7b347ed8d854b [CANDIDATE]: Term 1 election: Requested vote from peers
I20260923 22:12:57.142781 9781 raft_consensus.cc:2781] T 00000000000000000000000000000000 P e156218e8e7e492596e7b347ed8d854b [term 1 FOLLOWER]: Leader election won for term 1
I20260923 22:12:57.142808 1567 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskRiQe_a/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790201539513534-1567-0/minicluster-data/master-0/wal/instance
I20260923 22:12:57.142896 9781 raft_consensus.cc:686] T 00000000000000000000000000000000 P e156218e8e7e492596e7b347ed8d854b [term 1 LEADER]: Becoming Leader. State: Replica: e156218e8e7e492596e7b347ed8d854b, State: Running, Role: LEADER
I20260923 22:12:57.142987 9781 consensus_queue.cc:237] T 00000000000000000000000000000000 P e156218e8e7e492596e7b347ed8d854b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e156218e8e7e492596e7b347ed8d854b" member_type: VOTER last_known_addr { host: "127.1.135.254" port: 38637 } }
I20260923 22:12:57.143021 9778 sys_catalog.cc:564] T 00000000000000000000000000000000 P e156218e8e7e492596e7b347ed8d854b [sys.catalog]: configured and running, proceeding with master startup.
I20260923 22:12:57.143041 1567 external_mini_cluster.cc:1428] Setting key 7845a1e3f12d89fece4c0d6bcff04efb
I20260923 22:12:57.143282 9781 sys_catalog.cc:455] T 00000000000000000000000000000000 P e156218e8e7e492596e7b347ed8d854b [sys.catalog]: SysCatalogTable state changed. Reason: New leader e156218e8e7e492596e7b347ed8d854b. Latest consensus state: current_term: 1 leader_uuid: "e156218e8e7e492596e7b347ed8d854b" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e156218e8e7e492596e7b347ed8d854b" member_type: VOTER last_known_addr { host: "127.1.135.254" port: 38637 } } }
I20260923 22:12:57.143369 9781 sys_catalog.cc:458] T 00000000000000000000000000000000 P e156218e8e7e492596e7b347ed8d854b [sys.catalog]: This master's current role is: LEADER
I20260923 22:12:57.143291 9782 sys_catalog.cc:455] T 00000000000000000000000000000000 P e156218e8e7e492596e7b347ed8d854b [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "e156218e8e7e492596e7b347ed8d854b" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e156218e8e7e492596e7b347ed8d854b" member_type: VOTER last_known_addr { host: "127.1.135.254" port: 38637 } } }
I20260923 22:12:57.143420 9782 sys_catalog.cc:458] T 00000000000000000000000000000000 P e156218e8e7e492596e7b347ed8d854b [sys.catalog]: This master's current role is: LEADER
I20260923 22:12:57.143643 9786 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260923 22:12:57.144438 9786 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260923 22:12:57.146198 9786 catalog_manager.cc:1347] Generated new cluster ID: ce591194a02346b4bf0fd0301d0285d2
I20260923 22:12:57.146253 9786 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
Sep 23 22:12:57 dist-test-slave-s2fj krb5kdc[9697](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790201576, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.1.135.254@KRBTEST.COM
I20260923 22:12:57.153071 9787 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 22:12:57.143658 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46740 (local address 127.1.135.254:38637)
0923 22:12:57.144052 (+ 394us) server_negotiation.cc:206] Beginning negotiation
0923 22:12:57.144078 (+ 26us) server_negotiation.cc:399] Waiting for connection header
0923 22:12:57.144116 (+ 38us) server_negotiation.cc:407] Connection header received
0923 22:12:57.144232 (+ 116us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 22:12:57.144238 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 22:12:57.144294 (+ 56us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 22:12:57.145148 (+ 854us) server_negotiation.cc:226] Negotiated authn=SASL
0923 22:12:57.145246 (+ 98us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 22:12:57.146331 (+ 1085us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 22:12:57.147095 (+ 764us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 22:12:57.147514 (+ 419us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 22:12:57.147631 (+ 117us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 22:12:57.149933 (+ 2302us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0923 22:12:57.149937 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0923 22:12:57.149938 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0923 22:12:57.149959 (+ 21us) server_negotiation.cc:922] calling sasl_server_start()
0923 22:12:57.151574 (+ 1615us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0923 22:12:57.151944 (+ 370us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0923 22:12:57.151947 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0923 22:12:57.151947 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0923 22:12:57.152049 (+ 102us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0923 22:12:57.152285 (+ 236us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0923 22:12:57.152286 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0923 22:12:57.152286 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0923 22:12:57.152514 (+ 228us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0923 22:12:57.152693 (+ 179us) server_negotiation.cc:1035] Waiting for connection context
0923 22:12:57.152758 (+ 65us) server_negotiation.cc:299] Negotiation successful
0923 22:12:57.152896 (+ 138us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":347,"thread_start_us":63,"threads_started":1}
I20260923 22:12:57.157496 9786 catalog_manager.cc:1370] Generated new certificate authority record
I20260923 22:12:57.158031 9786 catalog_manager.cc:1504] Loading token signing keys...
I20260923 22:12:57.166435 9786 catalog_manager.cc:5868] T 00000000000000000000000000000000 P e156218e8e7e492596e7b347ed8d854b: Generated new TSK 0
I20260923 22:12:57.166618 9786 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.1.135.193@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.1.135.193@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.1.135.193 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskRiQe_a/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790201539513534-1567-0/krb5kdc/kudu_127.1.135.193.keytab.
Entry for principal kudu/127.1.135.193 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskRiQe_a/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790201539513534-1567-0/krb5kdc/kudu_127.1.135.193.keytab.
I20260923 22:12:57.195112 1567 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskRiQe_a/build/release/bin/kudu
/tmp/dist-test-taskRiQe_a/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskRiQe_a/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790201539513534-1567-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskRiQe_a/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790201539513534-1567-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskRiQe_a/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790201539513534-1567-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskRiQe_a/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790201539513534-1567-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.1.135.193:0
--local_ip_for_outbound_sockets=127.1.135.193
--webserver_interface=127.1.135.193
--webserver_port=0
--tserver_master_addrs=127.1.135.254:38637
--keytab_file=/tmp/dist-test-taskRiQe_a/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790201539513534-1567-0/krb5kdc/kudu_127.1.135.193.keytab
--principal=kudu/127.1.135.193
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.1.135.212:36077
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskRiQe_a/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790201539513534-1567-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskRiQe_a/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790201539513534-1567-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskRiQe_a/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790201539513534-1567-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260923 22:12:57.293495 9804 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260923 22:12:57.293686 9804 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260923 22:12:57.293708 9804 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260923 22:12:57.294886 9804 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260923 22:12:57.294960 9804 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260923 22:12:57.294988 9804 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.1.135.193
I20260923 22:12:57.296408 9804 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.1.135.212:36077
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskRiQe_a/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790201539513534-1567-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskRiQe_a/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790201539513534-1567-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskRiQe_a/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790201539513534-1567-0/krb5kdc/kudu_127.1.135.193.keytab
--principal=kudu/127.1.135.193
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.1.135.193:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskRiQe_a/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790201539513534-1567-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.1.135.193
--webserver_port=0
--tserver_master_addrs=127.1.135.254:38637
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.9804
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.1.135.193
--log_dir=/tmp/dist-test-taskRiQe_a/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790201539513534-1567-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 23 Sep 2026 20:53:37 UTC on ccycloud.quasar-rkviyx.root.comops.site
build id 4668083
I20260923 22:12:57.296630 9804 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260923 22:12:57.296861 9804 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260923 22:12:57.301687 9810 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260923 22:12:57.301707 9809 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260923 22:12:57.301687 9812 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260923 22:12:57.301918 9804 server_base.cc:1034] running on GCE node
I20260923 22:12:57.302155 9804 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260923 22:12:57.302472 9804 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260923 22:12:57.303625 9804 hybrid_clock.cc:648] HybridClock initialized: now 1790201577303606 us; error 36 us; skew 500 ppm
Sep 23 22:12:57 dist-test-slave-s2fj krb5kdc[9697](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790201577, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.135.193@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260923 22:12:57.306005 9804 init.cc:377] Logged in from keytab as kudu/127.1.135.193@KRBTEST.COM (short username kudu)
I20260923 22:12:57.306478 9804 webserver.cc:460] Webserver started at http://127.1.135.193:34721/ using document root <none> and password file <none>
I20260923 22:12:57.306685 9804 fs_manager.cc:359] Metadata directory not provided
I20260923 22:12:57.306748 9804 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260923 22:12:57.306919 9804 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260923 22:12:57.307936 9804 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskRiQe_a/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790201539513534-1567-0/minicluster-data/ts-0/data/instance:
uuid: "b52106de424b4a678b18b2c8707d75bc"
format_stamp: "Formatted at 2026-09-23 22:12:57 on dist-test-slave-s2fj"
server_key: "1eb169d16415f5199330c7f824e3c3da"
server_key_iv: "ac5b66b161f703fae57ed6549823ff58"
server_key_version: "encryptionkey@0"
I20260923 22:12:57.308337 9804 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskRiQe_a/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790201539513534-1567-0/minicluster-data/ts-0/wal/instance:
uuid: "b52106de424b4a678b18b2c8707d75bc"
format_stamp: "Formatted at 2026-09-23 22:12:57 on dist-test-slave-s2fj"
server_key: "1eb169d16415f5199330c7f824e3c3da"
server_key_iv: "ac5b66b161f703fae57ed6549823ff58"
server_key_version: "encryptionkey@0"
I20260923 22:12:57.309588 9804 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.001s sys 0.000s
I20260923 22:12:57.310292 9819 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260923 22:12:57.310456 9804 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260923 22:12:57.310554 9804 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskRiQe_a/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790201539513534-1567-0/minicluster-data/ts-0/data,/tmp/dist-test-taskRiQe_a/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790201539513534-1567-0/minicluster-data/ts-0/wal
uuid: "b52106de424b4a678b18b2c8707d75bc"
format_stamp: "Formatted at 2026-09-23 22:12:57 on dist-test-slave-s2fj"
server_key: "1eb169d16415f5199330c7f824e3c3da"
server_key_iv: "ac5b66b161f703fae57ed6549823ff58"
server_key_version: "encryptionkey@0"
I20260923 22:12:57.310644 9804 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskRiQe_a/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790201539513534-1567-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskRiQe_a/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790201539513534-1567-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskRiQe_a/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790201539513534-1567-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260923 22:12:57.326640 9804 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260923 22:12:57.328387 9804 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260923 22:12:57.328544 9804 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260923 22:12:57.328758 9804 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260923 22:12:57.329074 9804 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260923 22:12:57.329123 9804 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260923 22:12:57.329162 9804 ts_tablet_manager.cc:610] Registered 0 tablets
I20260923 22:12:57.329187 9804 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260923 22:12:57.335927 9804 rpc_server.cc:307] RPC server started. Bound to: 127.1.135.193:36515
I20260923 22:12:57.335973 9932 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.1.135.193:36515 every 8 connection(s)
I20260923 22:12:57.336339 9804 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskRiQe_a/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790201539513534-1567-0/minicluster-data/ts-0/data/info.pb
I20260923 22:12:57.340519 1567 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskRiQe_a/build/release/bin/kudu as pid 9804
I20260923 22:12:57.340611 1567 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskRiQe_a/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790201539513534-1567-0/minicluster-data/ts-0/wal/instance
I20260923 22:12:57.340842 1567 external_mini_cluster.cc:1428] Setting key 349b43fb4e3fdf33b91aedd20ec9e9f0
Sep 23 22:12:57 dist-test-slave-s2fj krb5kdc[9697](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790201577, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.135.193@KRBTEST.COM for kudu/127.1.135.254@KRBTEST.COM
I20260923 22:12:57.346552 9935 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 22:12:57.336991 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.135.254:38637 (local address 127.1.135.193:33693)
0923 22:12:57.337163 (+ 172us) negotiation.cc:107] Waiting for socket to connect
0923 22:12:57.337197 (+ 34us) client_negotiation.cc:174] Beginning negotiation
0923 22:12:57.337684 (+ 487us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0923 22:12:57.337924 (+ 240us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0923 22:12:57.337928 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0923 22:12:57.338487 (+ 559us) client_negotiation.cc:189] Negotiated authn=SASL
0923 22:12:57.338666 (+ 179us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 22:12:57.338670 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 22:12:57.339380 (+ 710us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 22:12:57.339381 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 22:12:57.340269 (+ 888us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 22:12:57.340272 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 22:12:57.340745 (+ 473us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 22:12:57.340747 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 22:12:57.340904 (+ 157us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 22:12:57.341520 (+ 616us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0923 22:12:57.341530 (+ 10us) client_negotiation.cc:656] calling sasl_client_start()
0923 22:12:57.343310 (+ 1780us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0923 22:12:57.345255 (+ 1945us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0923 22:12:57.345260 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0923 22:12:57.345262 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0923 22:12:57.345475 (+ 213us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0923 22:12:57.345728 (+ 253us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0923 22:12:57.345729 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0923 22:12:57.345729 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0923 22:12:57.345824 (+ 95us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0923 22:12:57.346098 (+ 274us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0923 22:12:57.346102 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0923 22:12:57.346206 (+ 104us) client_negotiation.cc:769] Sending connection context
0923 22:12:57.346287 (+ 81us) client_negotiation.cc:240] Negotiation successful
0923 22:12:57.346399 (+ 112us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":127,"thread_start_us":99,"threads_started":1}
I20260923 22:12:57.346585 9787 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 22:12:57.337015 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.135.193:33693 (local address 127.1.135.254:38637)
0923 22:12:57.337077 (+ 62us) server_negotiation.cc:206] Beginning negotiation
0923 22:12:57.337079 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0923 22:12:57.337278 (+ 199us) server_negotiation.cc:407] Connection header received
0923 22:12:57.337789 (+ 511us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 22:12:57.337790 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 22:12:57.337805 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 22:12:57.337854 (+ 49us) server_negotiation.cc:226] Negotiated authn=SASL
0923 22:12:57.338775 (+ 921us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 22:12:57.339281 (+ 506us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 22:12:57.340407 (+ 1126us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 22:12:57.340658 (+ 251us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 22:12:57.340719 (+ 61us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 22:12:57.343415 (+ 2696us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0923 22:12:57.343432 (+ 17us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0923 22:12:57.343433 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0923 22:12:57.343440 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0923 22:12:57.345172 (+ 1732us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0923 22:12:57.345551 (+ 379us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0923 22:12:57.345552 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0923 22:12:57.345553 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0923 22:12:57.345616 (+ 63us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0923 22:12:57.345891 (+ 275us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0923 22:12:57.345892 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0923 22:12:57.345892 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0923 22:12:57.346034 (+ 142us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0923 22:12:57.346106 (+ 72us) server_negotiation.cc:1035] Waiting for connection context
0923 22:12:57.346354 (+ 248us) server_negotiation.cc:299] Negotiation successful
0923 22:12:57.346455 (+ 101us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":37}
I20260923 22:12:57.346997 9933 heartbeater.cc:347] Connected to a master server at 127.1.135.254:38637
I20260923 22:12:57.347093 9933 heartbeater.cc:464] Registering TS with master...
I20260923 22:12:57.347283 9933 heartbeater.cc:511] Master 127.1.135.254:38637 requested a full tablet report, sending...
I20260923 22:12:57.347764 9742 ts_manager.cc:194] Registered new tserver with Master: b52106de424b4a678b18b2c8707d75bc (127.1.135.193:36515)
I20260923 22:12:57.348438 9742 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.1.135.193@KRBTEST.COM'} at 127.1.135.193:33693
WARNING: no policy specified for kudu/127.1.135.194@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.1.135.194@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.1.135.194 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskRiQe_a/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790201539513534-1567-0/krb5kdc/kudu_127.1.135.194.keytab.
Entry for principal kudu/127.1.135.194 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskRiQe_a/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790201539513534-1567-0/krb5kdc/kudu_127.1.135.194.keytab.
I20260923 22:12:57.365144 1567 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskRiQe_a/build/release/bin/kudu
/tmp/dist-test-taskRiQe_a/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskRiQe_a/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790201539513534-1567-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskRiQe_a/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790201539513534-1567-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskRiQe_a/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790201539513534-1567-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskRiQe_a/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790201539513534-1567-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.1.135.194:0
--local_ip_for_outbound_sockets=127.1.135.194
--webserver_interface=127.1.135.194
--webserver_port=0
--tserver_master_addrs=127.1.135.254:38637
--keytab_file=/tmp/dist-test-taskRiQe_a/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790201539513534-1567-0/krb5kdc/kudu_127.1.135.194.keytab
--principal=kudu/127.1.135.194
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.1.135.212:36077
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskRiQe_a/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790201539513534-1567-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskRiQe_a/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790201539513534-1567-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskRiQe_a/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790201539513534-1567-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260923 22:12:57.444612 9938 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260923 22:12:57.444795 9938 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260923 22:12:57.444818 9938 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260923 22:12:57.445917 9938 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260923 22:12:57.445957 9938 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260923 22:12:57.445983 9938 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.1.135.194
I20260923 22:12:57.447283 9938 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.1.135.212:36077
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskRiQe_a/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790201539513534-1567-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskRiQe_a/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790201539513534-1567-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskRiQe_a/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790201539513534-1567-0/krb5kdc/kudu_127.1.135.194.keytab
--principal=kudu/127.1.135.194
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.1.135.194:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskRiQe_a/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790201539513534-1567-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.1.135.194
--webserver_port=0
--tserver_master_addrs=127.1.135.254:38637
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.9938
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.1.135.194
--log_dir=/tmp/dist-test-taskRiQe_a/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790201539513534-1567-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 23 Sep 2026 20:53:37 UTC on ccycloud.quasar-rkviyx.root.comops.site
build id 4668083
I20260923 22:12:57.447443 9938 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260923 22:12:57.447604 9938 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260923 22:12:57.452240 9944 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260923 22:12:57.452306 9946 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260923 22:12:57.452417 9938 server_base.cc:1034] running on GCE node
W20260923 22:12:57.452239 9943 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260923 22:12:57.452700 9938 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260923 22:12:57.452888 9938 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260923 22:12:57.454044 9938 hybrid_clock.cc:648] HybridClock initialized: now 1790201577454032 us; error 27 us; skew 500 ppm
Sep 23 22:12:57 dist-test-slave-s2fj krb5kdc[9697](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790201577, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.135.194@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260923 22:12:57.456321 9938 init.cc:377] Logged in from keytab as kudu/127.1.135.194@KRBTEST.COM (short username kudu)
I20260923 22:12:57.456765 9938 webserver.cc:460] Webserver started at http://127.1.135.194:32885/ using document root <none> and password file <none>
I20260923 22:12:57.456995 9938 fs_manager.cc:359] Metadata directory not provided
I20260923 22:12:57.457062 9938 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260923 22:12:57.457213 9938 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260923 22:12:57.458204 9938 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskRiQe_a/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790201539513534-1567-0/minicluster-data/ts-1/data/instance:
uuid: "f45403e0b77b4fbca524bf4ef8d8a1ca"
format_stamp: "Formatted at 2026-09-23 22:12:57 on dist-test-slave-s2fj"
server_key: "fbf10af433d233810441927c823de5cf"
server_key_iv: "95fe80671fb302d0a0b84e71ce180aab"
server_key_version: "encryptionkey@0"
I20260923 22:12:57.458549 9938 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskRiQe_a/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790201539513534-1567-0/minicluster-data/ts-1/wal/instance:
uuid: "f45403e0b77b4fbca524bf4ef8d8a1ca"
format_stamp: "Formatted at 2026-09-23 22:12:57 on dist-test-slave-s2fj"
server_key: "fbf10af433d233810441927c823de5cf"
server_key_iv: "95fe80671fb302d0a0b84e71ce180aab"
server_key_version: "encryptionkey@0"
I20260923 22:12:57.459782 9938 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260923 22:12:57.460528 9953 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260923 22:12:57.460692 9938 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.000s
I20260923 22:12:57.460781 9938 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskRiQe_a/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790201539513534-1567-0/minicluster-data/ts-1/data,/tmp/dist-test-taskRiQe_a/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790201539513534-1567-0/minicluster-data/ts-1/wal
uuid: "f45403e0b77b4fbca524bf4ef8d8a1ca"
format_stamp: "Formatted at 2026-09-23 22:12:57 on dist-test-slave-s2fj"
server_key: "fbf10af433d233810441927c823de5cf"
server_key_iv: "95fe80671fb302d0a0b84e71ce180aab"
server_key_version: "encryptionkey@0"
I20260923 22:12:57.460860 9938 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskRiQe_a/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790201539513534-1567-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskRiQe_a/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790201539513534-1567-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskRiQe_a/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790201539513534-1567-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260923 22:12:57.472294 9938 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260923 22:12:57.473976 9938 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260923 22:12:57.474124 9938 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260923 22:12:57.474354 9938 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260923 22:12:57.474650 9938 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260923 22:12:57.474702 9938 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260923 22:12:57.474740 9938 ts_tablet_manager.cc:610] Registered 0 tablets
I20260923 22:12:57.474769 9938 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260923 22:12:57.480546 9938 rpc_server.cc:307] RPC server started. Bound to: 127.1.135.194:35425
I20260923 22:12:57.480593 10066 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.1.135.194:35425 every 8 connection(s)
I20260923 22:12:57.480863 9938 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskRiQe_a/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790201539513534-1567-0/minicluster-data/ts-1/data/info.pb
Sep 23 22:12:57 dist-test-slave-s2fj krb5kdc[9697](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790201577, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.135.194@KRBTEST.COM for kudu/127.1.135.254@KRBTEST.COM
I20260923 22:12:57.489317 9787 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 22:12:57.481591 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.135.194:59845 (local address 127.1.135.254:38637)
0923 22:12:57.481662 (+ 71us) server_negotiation.cc:206] Beginning negotiation
0923 22:12:57.481665 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0923 22:12:57.481848 (+ 183us) server_negotiation.cc:407] Connection header received
0923 22:12:57.482268 (+ 420us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 22:12:57.482269 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 22:12:57.482279 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 22:12:57.482349 (+ 70us) server_negotiation.cc:226] Negotiated authn=SASL
0923 22:12:57.483180 (+ 831us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 22:12:57.483712 (+ 532us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 22:12:57.484648 (+ 936us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 22:12:57.484860 (+ 212us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 22:12:57.484921 (+ 61us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 22:12:57.487038 (+ 2117us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0923 22:12:57.487039 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0923 22:12:57.487039 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0923 22:12:57.487045 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0923 22:12:57.488245 (+ 1200us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0923 22:12:57.488578 (+ 333us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0923 22:12:57.488579 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0923 22:12:57.488579 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0923 22:12:57.488627 (+ 48us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0923 22:12:57.488823 (+ 196us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0923 22:12:57.488823 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0923 22:12:57.488824 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0923 22:12:57.488921 (+ 97us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0923 22:12:57.488981 (+ 60us) server_negotiation.cc:1035] Waiting for connection context
0923 22:12:57.489183 (+ 202us) server_negotiation.cc:299] Negotiation successful
0923 22:12:57.489241 (+ 58us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":44}
I20260923 22:12:57.489512 10069 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 22:12:57.481571 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.135.254:38637 (local address 127.1.135.194:59845)
0923 22:12:57.481720 (+ 149us) negotiation.cc:107] Waiting for socket to connect
0923 22:12:57.481758 (+ 38us) client_negotiation.cc:174] Beginning negotiation
0923 22:12:57.482190 (+ 432us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0923 22:12:57.482397 (+ 207us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0923 22:12:57.482400 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0923 22:12:57.482915 (+ 515us) client_negotiation.cc:189] Negotiated authn=SASL
0923 22:12:57.483101 (+ 186us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 22:12:57.483105 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 22:12:57.483789 (+ 684us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 22:12:57.483790 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 22:12:57.484567 (+ 777us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 22:12:57.484569 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 22:12:57.484934 (+ 365us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 22:12:57.484934 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 22:12:57.485032 (+ 98us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 22:12:57.485437 (+ 405us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0923 22:12:57.485446 (+ 9us) client_negotiation.cc:656] calling sasl_client_start()
0923 22:12:57.486956 (+ 1510us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0923 22:12:57.488310 (+ 1354us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0923 22:12:57.488315 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0923 22:12:57.488316 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0923 22:12:57.488516 (+ 200us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0923 22:12:57.488683 (+ 167us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0923 22:12:57.488683 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0923 22:12:57.488683 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0923 22:12:57.488769 (+ 86us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0923 22:12:57.488978 (+ 209us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0923 22:12:57.488982 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0923 22:12:57.489078 (+ 96us) client_negotiation.cc:769] Sending connection context
0923 22:12:57.489155 (+ 77us) client_negotiation.cc:240] Negotiation successful
0923 22:12:57.489261 (+ 106us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":111,"thread_start_us":87,"threads_started":1}
I20260923 22:12:57.489500 1567 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskRiQe_a/build/release/bin/kudu as pid 9938
I20260923 22:12:57.489609 1567 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskRiQe_a/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790201539513534-1567-0/minicluster-data/ts-1/wal/instance
I20260923 22:12:57.489780 1567 external_mini_cluster.cc:1428] Setting key d1db20de19f819ab2e6bb856a817cfe5
I20260923 22:12:57.490124 10067 heartbeater.cc:347] Connected to a master server at 127.1.135.254:38637
I20260923 22:12:57.490208 10067 heartbeater.cc:464] Registering TS with master...
I20260923 22:12:57.490413 10067 heartbeater.cc:511] Master 127.1.135.254:38637 requested a full tablet report, sending...
I20260923 22:12:57.490797 9742 ts_manager.cc:194] Registered new tserver with Master: f45403e0b77b4fbca524bf4ef8d8a1ca (127.1.135.194:35425)
I20260923 22:12:57.491179 9742 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.1.135.194@KRBTEST.COM'} at 127.1.135.194:59845
WARNING: no policy specified for kudu/127.1.135.195@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.1.135.195@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.1.135.195 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskRiQe_a/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790201539513534-1567-0/krb5kdc/kudu_127.1.135.195.keytab.
Entry for principal kudu/127.1.135.195 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskRiQe_a/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790201539513534-1567-0/krb5kdc/kudu_127.1.135.195.keytab.
I20260923 22:12:57.514146 1567 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskRiQe_a/build/release/bin/kudu
/tmp/dist-test-taskRiQe_a/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskRiQe_a/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790201539513534-1567-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskRiQe_a/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790201539513534-1567-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskRiQe_a/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790201539513534-1567-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskRiQe_a/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790201539513534-1567-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.1.135.195:0
--local_ip_for_outbound_sockets=127.1.135.195
--webserver_interface=127.1.135.195
--webserver_port=0
--tserver_master_addrs=127.1.135.254:38637
--keytab_file=/tmp/dist-test-taskRiQe_a/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790201539513534-1567-0/krb5kdc/kudu_127.1.135.195.keytab
--principal=kudu/127.1.135.195
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.1.135.212:36077
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskRiQe_a/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790201539513534-1567-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskRiQe_a/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790201539513534-1567-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskRiQe_a/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790201539513534-1567-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260923 22:12:57.592762 10072 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260923 22:12:57.593010 10072 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260923 22:12:57.593050 10072 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260923 22:12:57.594141 10072 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260923 22:12:57.594203 10072 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260923 22:12:57.594247 10072 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.1.135.195
I20260923 22:12:57.595665 10072 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.1.135.212:36077
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskRiQe_a/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790201539513534-1567-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskRiQe_a/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790201539513534-1567-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskRiQe_a/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790201539513534-1567-0/krb5kdc/kudu_127.1.135.195.keytab
--principal=kudu/127.1.135.195
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.1.135.195:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskRiQe_a/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790201539513534-1567-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.1.135.195
--webserver_port=0
--tserver_master_addrs=127.1.135.254:38637
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.10072
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.1.135.195
--log_dir=/tmp/dist-test-taskRiQe_a/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790201539513534-1567-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 23 Sep 2026 20:53:37 UTC on ccycloud.quasar-rkviyx.root.comops.site
build id 4668083
I20260923 22:12:57.595927 10072 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260923 22:12:57.596210 10072 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260923 22:12:57.600940 10078 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260923 22:12:57.600973 10080 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260923 22:12:57.600968 10077 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260923 22:12:57.601262 10072 server_base.cc:1034] running on GCE node
I20260923 22:12:57.601505 10072 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260923 22:12:57.601725 10072 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260923 22:12:57.602869 10072 hybrid_clock.cc:648] HybridClock initialized: now 1790201577602859 us; error 26 us; skew 500 ppm
Sep 23 22:12:57 dist-test-slave-s2fj krb5kdc[9697](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790201577, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.135.195@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260923 22:12:57.605433 10072 init.cc:377] Logged in from keytab as kudu/127.1.135.195@KRBTEST.COM (short username kudu)
I20260923 22:12:57.605841 10072 webserver.cc:460] Webserver started at http://127.1.135.195:33823/ using document root <none> and password file <none>
I20260923 22:12:57.606015 10072 fs_manager.cc:359] Metadata directory not provided
I20260923 22:12:57.606076 10072 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260923 22:12:57.606155 10072 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260923 22:12:57.607110 10072 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskRiQe_a/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790201539513534-1567-0/minicluster-data/ts-2/data/instance:
uuid: "bd4dac402de8439f865e36d987a8b151"
format_stamp: "Formatted at 2026-09-23 22:12:57 on dist-test-slave-s2fj"
server_key: "ab40aeab9db52c37193c4f9ee4b84869"
server_key_iv: "47cd8ad09140cd87337a6c8fb17e7ae0"
server_key_version: "encryptionkey@0"
I20260923 22:12:57.607404 10072 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskRiQe_a/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790201539513534-1567-0/minicluster-data/ts-2/wal/instance:
uuid: "bd4dac402de8439f865e36d987a8b151"
format_stamp: "Formatted at 2026-09-23 22:12:57 on dist-test-slave-s2fj"
server_key: "ab40aeab9db52c37193c4f9ee4b84869"
server_key_iv: "47cd8ad09140cd87337a6c8fb17e7ae0"
server_key_version: "encryptionkey@0"
I20260923 22:12:57.608731 10072 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.001s sys 0.001s
I20260923 22:12:57.609352 10087 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260923 22:12:57.609555 10072 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260923 22:12:57.609647 10072 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskRiQe_a/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790201539513534-1567-0/minicluster-data/ts-2/data,/tmp/dist-test-taskRiQe_a/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790201539513534-1567-0/minicluster-data/ts-2/wal
uuid: "bd4dac402de8439f865e36d987a8b151"
format_stamp: "Formatted at 2026-09-23 22:12:57 on dist-test-slave-s2fj"
server_key: "ab40aeab9db52c37193c4f9ee4b84869"
server_key_iv: "47cd8ad09140cd87337a6c8fb17e7ae0"
server_key_version: "encryptionkey@0"
I20260923 22:12:57.609719 10072 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskRiQe_a/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790201539513534-1567-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskRiQe_a/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790201539513534-1567-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskRiQe_a/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790201539513534-1567-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260923 22:12:57.635782 10072 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260923 22:12:57.638244 10072 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260923 22:12:57.638415 10072 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260923 22:12:57.638643 10072 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260923 22:12:57.638957 10072 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260923 22:12:57.639011 10072 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260923 22:12:57.639053 10072 ts_tablet_manager.cc:610] Registered 0 tablets
I20260923 22:12:57.639081 10072 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260923 22:12:57.644989 10072 rpc_server.cc:307] RPC server started. Bound to: 127.1.135.195:38051
I20260923 22:12:57.645066 10200 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.1.135.195:38051 every 8 connection(s)
I20260923 22:12:57.645358 10072 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskRiQe_a/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790201539513534-1567-0/minicluster-data/ts-2/data/info.pb
I20260923 22:12:57.648518 1567 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskRiQe_a/build/release/bin/kudu as pid 10072
I20260923 22:12:57.648655 1567 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskRiQe_a/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790201539513534-1567-0/minicluster-data/ts-2/wal/instance
I20260923 22:12:57.648857 1567 external_mini_cluster.cc:1428] Setting key 816a8481b79f061d331665b4ce926243
Sep 23 22:12:57 dist-test-slave-s2fj krb5kdc[9697](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790201577, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.135.195@KRBTEST.COM for kudu/127.1.135.254@KRBTEST.COM
I20260923 22:12:57.654484 9787 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 22:12:57.646035 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.135.195:51053 (local address 127.1.135.254:38637)
0923 22:12:57.646105 (+ 70us) server_negotiation.cc:206] Beginning negotiation
0923 22:12:57.646107 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0923 22:12:57.646297 (+ 190us) server_negotiation.cc:407] Connection header received
0923 22:12:57.646752 (+ 455us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 22:12:57.646753 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 22:12:57.646763 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 22:12:57.646826 (+ 63us) server_negotiation.cc:226] Negotiated authn=SASL
0923 22:12:57.647666 (+ 840us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 22:12:57.648261 (+ 595us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 22:12:57.649560 (+ 1299us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 22:12:57.649762 (+ 202us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 22:12:57.649839 (+ 77us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 22:12:57.651881 (+ 2042us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0923 22:12:57.651882 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0923 22:12:57.651883 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0923 22:12:57.651890 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0923 22:12:57.653098 (+ 1208us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0923 22:12:57.653518 (+ 420us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0923 22:12:57.653520 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0923 22:12:57.653520 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0923 22:12:57.653582 (+ 62us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0923 22:12:57.653830 (+ 248us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0923 22:12:57.653831 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0923 22:12:57.653831 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0923 22:12:57.653955 (+ 124us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0923 22:12:57.654049 (+ 94us) server_negotiation.cc:1035] Waiting for connection context
0923 22:12:57.654319 (+ 270us) server_negotiation.cc:299] Negotiation successful
0923 22:12:57.654397 (+ 78us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":44}
I20260923 22:12:57.654659 10203 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 22:12:57.646002 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.135.254:38637 (local address 127.1.135.195:51053)
0923 22:12:57.646172 (+ 170us) negotiation.cc:107] Waiting for socket to connect
0923 22:12:57.646212 (+ 40us) client_negotiation.cc:174] Beginning negotiation
0923 22:12:57.646674 (+ 462us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0923 22:12:57.646879 (+ 205us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0923 22:12:57.646882 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0923 22:12:57.647412 (+ 530us) client_negotiation.cc:189] Negotiated authn=SASL
0923 22:12:57.647592 (+ 180us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 22:12:57.647597 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 22:12:57.648349 (+ 752us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 22:12:57.648350 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 22:12:57.649436 (+ 1086us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 22:12:57.649439 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 22:12:57.649845 (+ 406us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 22:12:57.649846 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 22:12:57.649979 (+ 133us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 22:12:57.650387 (+ 408us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0923 22:12:57.650395 (+ 8us) client_negotiation.cc:656] calling sasl_client_start()
0923 22:12:57.651756 (+ 1361us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0923 22:12:57.653181 (+ 1425us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0923 22:12:57.653187 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0923 22:12:57.653189 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0923 22:12:57.653421 (+ 232us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0923 22:12:57.653650 (+ 229us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0923 22:12:57.653651 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0923 22:12:57.653651 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0923 22:12:57.653761 (+ 110us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0923 22:12:57.654039 (+ 278us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0923 22:12:57.654045 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0923 22:12:57.654182 (+ 137us) client_negotiation.cc:769] Sending connection context
0923 22:12:57.654287 (+ 105us) client_negotiation.cc:240] Negotiation successful
0923 22:12:57.654432 (+ 145us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":133,"thread_start_us":106,"threads_started":1}
I20260923 22:12:57.655114 10201 heartbeater.cc:347] Connected to a master server at 127.1.135.254:38637
I20260923 22:12:57.655201 10201 heartbeater.cc:464] Registering TS with master...
I20260923 22:12:57.655345 10201 heartbeater.cc:511] Master 127.1.135.254:38637 requested a full tablet report, sending...
I20260923 22:12:57.655689 9742 ts_manager.cc:194] Registered new tserver with Master: bd4dac402de8439f865e36d987a8b151 (127.1.135.195:38051)
I20260923 22:12:57.656004 9742 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.1.135.195@KRBTEST.COM'} at 127.1.135.195:51053
I20260923 22:12:57.660557 1567 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
WARNING: no policy specified for oryx/127.1.135.193@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.1.135.193@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.1.135.193 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskRiQe_a/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790201539513534-1567-0/krb5kdc/oryx_127.1.135.193.keytab.
Entry for principal oryx/127.1.135.193 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskRiQe_a/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790201539513534-1567-0/krb5kdc/oryx_127.1.135.193.keytab.
WARNING: no policy specified for oryx/127.1.135.194@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.1.135.194@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.1.135.194 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskRiQe_a/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790201539513534-1567-0/krb5kdc/oryx_127.1.135.194.keytab.
Entry for principal oryx/127.1.135.194 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskRiQe_a/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790201539513534-1567-0/krb5kdc/oryx_127.1.135.194.keytab.
WARNING: no policy specified for oryx/127.1.135.195@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.1.135.195@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.1.135.195 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskRiQe_a/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790201539513534-1567-0/krb5kdc/oryx_127.1.135.195.keytab.
Entry for principal oryx/127.1.135.195 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskRiQe_a/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790201539513534-1567-0/krb5kdc/oryx_127.1.135.195.keytab.
I20260923 22:12:57.724519 1567 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskRiQe_a/build/release/bin/kudu with pid 9804
I20260923 22:12:57.728694 1567 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskRiQe_a/build/release/bin/kudu with pid 9938
I20260923 22:12:57.732916 1567 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskRiQe_a/build/release/bin/kudu with pid 10072
I20260923 22:12:57.736841 1567 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskRiQe_a/build/release/bin/kudu with pid 9710
I20260923 22:12:57.741596 1567 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskRiQe_a/build/release/bin/kudu
/tmp/dist-test-taskRiQe_a/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskRiQe_a/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790201539513534-1567-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskRiQe_a/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790201539513534-1567-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskRiQe_a/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790201539513534-1567-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskRiQe_a/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790201539513534-1567-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.1.135.254:38637
--webserver_interface=127.1.135.254
--webserver_port=38727
--keytab_file=/tmp/dist-test-taskRiQe_a/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790201539513534-1567-0/krb5kdc/kudu_127.1.135.254.keytab
--principal=kudu/127.1.135.254
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.1.135.212:36077
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.1.135.254:38637
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskRiQe_a/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790201539513534-1567-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskRiQe_a/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790201539513534-1567-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskRiQe_a/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790201539513534-1567-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260923 22:12:57.819864 10210 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260923 22:12:57.820101 10210 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260923 22:12:57.820137 10210 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260923 22:12:57.821234 10210 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260923 22:12:57.821271 10210 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260923 22:12:57.821286 10210 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260923 22:12:57.821298 10210 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260923 22:12:57.821310 10210 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260923 22:12:57.822640 10210 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.1.135.212:36077
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskRiQe_a/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790201539513534-1567-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskRiQe_a/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790201539513534-1567-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.1.135.254:38637
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskRiQe_a/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790201539513534-1567-0/krb5kdc/kudu_127.1.135.254.keytab
--principal=kudu/127.1.135.254
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.1.135.254:38637
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskRiQe_a/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790201539513534-1567-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.1.135.254
--webserver_port=38727
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.10210
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskRiQe_a/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790201539513534-1567-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 23 Sep 2026 20:53:37 UTC on ccycloud.quasar-rkviyx.root.comops.site
build id 4668083
I20260923 22:12:57.822820 10210 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260923 22:12:57.822979 10210 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260923 22:12:57.827586 10216 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260923 22:12:57.827677 10210 server_base.cc:1034] running on GCE node
W20260923 22:12:57.827622 10215 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260923 22:12:57.827646 10218 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260923 22:12:57.827996 10210 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260923 22:12:57.828290 10210 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260923 22:12:57.829427 10210 hybrid_clock.cc:648] HybridClock initialized: now 1790201577829413 us; error 29 us; skew 500 ppm
Sep 23 22:12:57 dist-test-slave-s2fj krb5kdc[9697](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790201577, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.135.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260923 22:12:57.831655 10210 init.cc:377] Logged in from keytab as kudu/127.1.135.254@KRBTEST.COM (short username kudu)
I20260923 22:12:57.832149 10210 webserver.cc:460] Webserver started at http://127.1.135.254:38727/ using document root <none> and password file <none>
I20260923 22:12:57.832363 10210 fs_manager.cc:359] Metadata directory not provided
I20260923 22:12:57.832448 10210 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260923 22:12:57.833724 10210 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.000s sys 0.001s
I20260923 22:12:57.834343 10225 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260923 22:12:57.834507 10210 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.001s
I20260923 22:12:57.834604 10210 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskRiQe_a/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790201539513534-1567-0/minicluster-data/master-0/data,/tmp/dist-test-taskRiQe_a/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790201539513534-1567-0/minicluster-data/master-0/wal
uuid: "e156218e8e7e492596e7b347ed8d854b"
format_stamp: "Formatted at 2026-09-23 22:12:57 on dist-test-slave-s2fj"
server_key: "526f8bc9db07a3d4e4662741e5da64d1"
server_key_iv: "126203d9ad107429041fe29da337bb24"
server_key_version: "encryptionkey@0"
I20260923 22:12:57.834934 10210 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskRiQe_a/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790201539513534-1567-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskRiQe_a/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790201539513534-1567-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskRiQe_a/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790201539513534-1567-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260923 22:12:57.847915 10210 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260923 22:12:57.849661 10210 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260923 22:12:57.849824 10210 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260923 22:12:57.853756 10210 rpc_server.cc:307] RPC server started. Bound to: 127.1.135.254:38637
I20260923 22:12:57.853789 10277 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.1.135.254:38637 every 8 connection(s)
I20260923 22:12:57.854135 10210 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskRiQe_a/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790201539513534-1567-0/minicluster-data/master-0/data/info.pb
I20260923 22:12:57.855465 10278 sys_catalog.cc:263] Verifying existing consensus state
I20260923 22:12:57.856132 10278 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P e156218e8e7e492596e7b347ed8d854b: Bootstrap starting.
I20260923 22:12:57.856279 1567 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskRiQe_a/build/release/bin/kudu as pid 10210
I20260923 22:12:57.856571 1567 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskRiQe_a/build/release/bin/kudu
/tmp/dist-test-taskRiQe_a/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskRiQe_a/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790201539513534-1567-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskRiQe_a/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790201539513534-1567-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskRiQe_a/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790201539513534-1567-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskRiQe_a/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790201539513534-1567-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.1.135.193:36515
--local_ip_for_outbound_sockets=127.1.135.193
--tserver_master_addrs=127.1.135.254:38637
--webserver_port=34721
--webserver_interface=127.1.135.193
--keytab_file=/tmp/dist-test-taskRiQe_a/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790201539513534-1567-0/krb5kdc/kudu_127.1.135.193.keytab
--principal=kudu/127.1.135.193
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.1.135.212:36077
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--principal=oryx/127.1.135.193
--keytab_file=/tmp/dist-test-taskRiQe_a/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790201539513534-1567-0/krb5kdc/oryx_127.1.135.193.keytab with env {KRB5CCNAME=/tmp/dist-test-taskRiQe_a/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790201539513534-1567-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskRiQe_a/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790201539513534-1567-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskRiQe_a/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790201539513534-1567-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260923 22:12:57.864009 10278 log.cc:824] T 00000000000000000000000000000000 P e156218e8e7e492596e7b347ed8d854b: Log is configured to *not* fsync() on all Append() calls
I20260923 22:12:57.865715 10278 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P e156218e8e7e492596e7b347ed8d854b: Bootstrap replayed 1/1 log segments. Stats: ops{read=4 overwritten=0 applied=4 ignored=0} inserts{seen=3 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260923 22:12:57.866019 10278 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P e156218e8e7e492596e7b347ed8d854b: Bootstrap complete.
I20260923 22:12:57.868301 10278 raft_consensus.cc:348] T 00000000000000000000000000000000 P e156218e8e7e492596e7b347ed8d854b [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e156218e8e7e492596e7b347ed8d854b" member_type: VOTER last_known_addr { host: "127.1.135.254" port: 38637 } }
I20260923 22:12:57.868530 10278 raft_consensus.cc:729] T 00000000000000000000000000000000 P e156218e8e7e492596e7b347ed8d854b [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: e156218e8e7e492596e7b347ed8d854b, State: Initialized, Role: FOLLOWER
I20260923 22:12:57.868706 10278 consensus_queue.cc:260] T 00000000000000000000000000000000 P e156218e8e7e492596e7b347ed8d854b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e156218e8e7e492596e7b347ed8d854b" member_type: VOTER last_known_addr { host: "127.1.135.254" port: 38637 } }
I20260923 22:12:57.868773 10278 raft_consensus.cc:388] T 00000000000000000000000000000000 P e156218e8e7e492596e7b347ed8d854b [term 1 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260923 22:12:57.868839 10278 raft_consensus.cc:482] T 00000000000000000000000000000000 P e156218e8e7e492596e7b347ed8d854b [term 1 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260923 22:12:57.868886 10278 raft_consensus.cc:3037] T 00000000000000000000000000000000 P e156218e8e7e492596e7b347ed8d854b [term 1 FOLLOWER]: Advancing to term 2
I20260923 22:12:57.869894 10278 raft_consensus.cc:504] T 00000000000000000000000000000000 P e156218e8e7e492596e7b347ed8d854b [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e156218e8e7e492596e7b347ed8d854b" member_type: VOTER last_known_addr { host: "127.1.135.254" port: 38637 } }
I20260923 22:12:57.870010 10278 leader_election.cc:302] T 00000000000000000000000000000000 P e156218e8e7e492596e7b347ed8d854b [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: e156218e8e7e492596e7b347ed8d854b; no voters:
I20260923 22:12:57.870203 10278 leader_election.cc:288] T 00000000000000000000000000000000 P e156218e8e7e492596e7b347ed8d854b [CANDIDATE]: Term 2 election: Requested vote from peers
I20260923 22:12:57.870239 10282 raft_consensus.cc:2781] T 00000000000000000000000000000000 P e156218e8e7e492596e7b347ed8d854b [term 2 FOLLOWER]: Leader election won for term 2
I20260923 22:12:57.870352 10282 raft_consensus.cc:686] T 00000000000000000000000000000000 P e156218e8e7e492596e7b347ed8d854b [term 2 LEADER]: Becoming Leader. State: Replica: e156218e8e7e492596e7b347ed8d854b, State: Running, Role: LEADER
I20260923 22:12:57.870435 10282 consensus_queue.cc:237] T 00000000000000000000000000000000 P e156218e8e7e492596e7b347ed8d854b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 2, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e156218e8e7e492596e7b347ed8d854b" member_type: VOTER last_known_addr { host: "127.1.135.254" port: 38637 } }
I20260923 22:12:57.870481 10278 sys_catalog.cc:564] T 00000000000000000000000000000000 P e156218e8e7e492596e7b347ed8d854b [sys.catalog]: configured and running, proceeding with master startup.
I20260923 22:12:57.870678 10282 sys_catalog.cc:455] T 00000000000000000000000000000000 P e156218e8e7e492596e7b347ed8d854b [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 2 leader_uuid: "e156218e8e7e492596e7b347ed8d854b" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e156218e8e7e492596e7b347ed8d854b" member_type: VOTER last_known_addr { host: "127.1.135.254" port: 38637 } } }
I20260923 22:12:57.870744 10282 sys_catalog.cc:458] T 00000000000000000000000000000000 P e156218e8e7e492596e7b347ed8d854b [sys.catalog]: This master's current role is: LEADER
I20260923 22:12:57.870846 10282 sys_catalog.cc:455] T 00000000000000000000000000000000 P e156218e8e7e492596e7b347ed8d854b [sys.catalog]: SysCatalogTable state changed. Reason: New leader e156218e8e7e492596e7b347ed8d854b. Latest consensus state: current_term: 2 leader_uuid: "e156218e8e7e492596e7b347ed8d854b" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e156218e8e7e492596e7b347ed8d854b" member_type: VOTER last_known_addr { host: "127.1.135.254" port: 38637 } } }
I20260923 22:12:57.870915 10282 sys_catalog.cc:458] T 00000000000000000000000000000000 P e156218e8e7e492596e7b347ed8d854b [sys.catalog]: This master's current role is: LEADER
I20260923 22:12:57.871295 10286 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260923 22:12:57.871795 10286 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260923 22:12:57.871955 10286 catalog_manager.cc:1259] Loaded cluster ID: ce591194a02346b4bf0fd0301d0285d2
I20260923 22:12:57.872171 10286 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260923 22:12:57.872982 10286 catalog_manager.cc:1504] Loading token signing keys...
I20260923 22:12:57.873169 10286 catalog_manager.cc:5879] T 00000000000000000000000000000000 P e156218e8e7e492596e7b347ed8d854b: Loaded TSK: 0
I20260923 22:12:57.873382 10286 catalog_manager.cc:1514] Initializing in-progress tserver states...
W20260923 22:12:57.951601 10280 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260923 22:12:57.951797 10280 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260923 22:12:57.951818 10280 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260923 22:12:57.952958 10280 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260923 22:12:57.952997 10280 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260923 22:12:57.953023 10280 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.1.135.193
I20260923 22:12:57.954396 10280 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.1.135.212:36077
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskRiQe_a/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790201539513534-1567-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskRiQe_a/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790201539513534-1567-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskRiQe_a/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790201539513534-1567-0/krb5kdc/oryx_127.1.135.193.keytab
--principal=oryx/127.1.135.193
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.1.135.193:36515
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskRiQe_a/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790201539513534-1567-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.1.135.193
--webserver_port=34721
--tserver_master_addrs=127.1.135.254:38637
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.10280
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.1.135.193
--log_dir=/tmp/dist-test-taskRiQe_a/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790201539513534-1567-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 23 Sep 2026 20:53:37 UTC on ccycloud.quasar-rkviyx.root.comops.site
build id 4668083
I20260923 22:12:57.954618 10280 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260923 22:12:57.954842 10280 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260923 22:12:57.960507 10304 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260923 22:12:57.960481 10305 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260923 22:12:57.960489 10307 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260923 22:12:57.960696 10280 server_base.cc:1034] running on GCE node
I20260923 22:12:57.960866 10280 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260923 22:12:57.961107 10280 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260923 22:12:57.962260 10280 hybrid_clock.cc:648] HybridClock initialized: now 1790201577962238 us; error 38 us; skew 500 ppm
Sep 23 22:12:57 dist-test-slave-s2fj krb5kdc[9697](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790201577, etypes {rep=17 tkt=17 ses=17}, oryx/127.1.135.193@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260923 22:12:57.964934 10280 init.cc:377] Logged in from keytab as oryx/127.1.135.193@KRBTEST.COM (short username oryx)
I20260923 22:12:57.965485 10280 webserver.cc:460] Webserver started at http://127.1.135.193:34721/ using document root <none> and password file <none>
I20260923 22:12:57.965760 10280 fs_manager.cc:359] Metadata directory not provided
I20260923 22:12:57.965844 10280 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260923 22:12:57.967535 10280 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.001s sys 0.001s
I20260923 22:12:57.968178 10314 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260923 22:12:57.968372 10280 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.000s
I20260923 22:12:57.968477 10280 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskRiQe_a/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790201539513534-1567-0/minicluster-data/ts-0/data,/tmp/dist-test-taskRiQe_a/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790201539513534-1567-0/minicluster-data/ts-0/wal
uuid: "b52106de424b4a678b18b2c8707d75bc"
format_stamp: "Formatted at 2026-09-23 22:12:57 on dist-test-slave-s2fj"
server_key: "1eb169d16415f5199330c7f824e3c3da"
server_key_iv: "ac5b66b161f703fae57ed6549823ff58"
server_key_version: "encryptionkey@0"
I20260923 22:12:57.968899 10280 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskRiQe_a/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790201539513534-1567-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskRiQe_a/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790201539513534-1567-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskRiQe_a/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790201539513534-1567-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260923 22:12:57.988152 10280 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260923 22:12:57.990445 10280 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260923 22:12:57.990599 10280 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260923 22:12:57.990839 10280 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260923 22:12:57.991165 10280 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260923 22:12:57.991214 10280 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260923 22:12:57.991245 10280 ts_tablet_manager.cc:610] Registered 0 tablets
I20260923 22:12:57.991271 10280 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260923 22:12:57.998423 10280 rpc_server.cc:307] RPC server started. Bound to: 127.1.135.193:36515
I20260923 22:12:57.998485 10427 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.1.135.193:36515 every 8 connection(s)
I20260923 22:12:57.998879 10280 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskRiQe_a/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790201539513534-1567-0/minicluster-data/ts-0/data/info.pb
I20260923 22:12:58.001801 1567 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskRiQe_a/build/release/bin/kudu as pid 10280
I20260923 22:12:58.002111 1567 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskRiQe_a/build/release/bin/kudu
/tmp/dist-test-taskRiQe_a/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskRiQe_a/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790201539513534-1567-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskRiQe_a/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790201539513534-1567-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskRiQe_a/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790201539513534-1567-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskRiQe_a/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790201539513534-1567-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.1.135.194:35425
--local_ip_for_outbound_sockets=127.1.135.194
--tserver_master_addrs=127.1.135.254:38637
--webserver_port=32885
--webserver_interface=127.1.135.194
--keytab_file=/tmp/dist-test-taskRiQe_a/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790201539513534-1567-0/krb5kdc/kudu_127.1.135.194.keytab
--principal=kudu/127.1.135.194
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.1.135.212:36077
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--principal=oryx/127.1.135.194
--keytab_file=/tmp/dist-test-taskRiQe_a/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790201539513534-1567-0/krb5kdc/oryx_127.1.135.194.keytab with env {KRB5CCNAME=/tmp/dist-test-taskRiQe_a/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790201539513534-1567-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskRiQe_a/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790201539513534-1567-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskRiQe_a/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790201539513534-1567-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
Sep 23 22:12:58 dist-test-slave-s2fj krb5kdc[9697](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.1.135.193@KRBTEST.COM for oryx/127.1.135.254@KRBTEST.COM, Server not found in Kerberos database
Sep 23 22:12:58 dist-test-slave-s2fj krb5kdc[9697](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.1.135.193@KRBTEST.COM for oryx/127.1.135.254@KRBTEST.COM, Server not found in Kerberos database
I20260923 22:12:58.009407 10430 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 22:12:57.999650 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.135.254:38637 (local address 127.1.135.193:46481)
0923 22:12:57.999840 (+ 190us) negotiation.cc:107] Waiting for socket to connect
0923 22:12:57.999888 (+ 48us) client_negotiation.cc:174] Beginning negotiation
0923 22:12:58.000450 (+ 562us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0923 22:12:58.000885 (+ 435us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0923 22:12:58.000888 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0923 22:12:58.001466 (+ 578us) client_negotiation.cc:189] Negotiated authn=SASL
0923 22:12:58.001687 (+ 221us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 22:12:58.001693 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 22:12:58.003306 (+ 1613us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 22:12:58.003307 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 22:12:58.004597 (+ 1290us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 22:12:58.004600 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 22:12:58.005452 (+ 852us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 22:12:58.005453 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 22:12:58.005603 (+ 150us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 22:12:58.006203 (+ 600us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0923 22:12:58.006212 (+ 9us) client_negotiation.cc:656] calling sasl_client_start()
0923 22:12:58.009168 (+ 2956us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.1.135.254:38637: Server oryx/127.1.135.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":144,"thread_start_us":99,"threads_started":1}
W20260923 22:12:58.009546 10431 negotiation.cc:337] Failed RPC negotiation. Trace:
0923 22:12:57.999716 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.135.193:46481 (local address 127.1.135.254:38637)
0923 22:12:57.999902 (+ 186us) server_negotiation.cc:206] Beginning negotiation
0923 22:12:57.999910 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0923 22:12:57.999981 (+ 71us) server_negotiation.cc:407] Connection header received
0923 22:12:58.000675 (+ 694us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 22:12:58.000681 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 22:12:58.000722 (+ 41us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 22:12:58.000864 (+ 142us) server_negotiation.cc:226] Negotiated authn=SASL
0923 22:12:58.001828 (+ 964us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 22:12:58.003182 (+ 1354us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 22:12:58.004691 (+ 1509us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 22:12:58.005345 (+ 654us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 22:12:58.005473 (+ 128us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 22:12:58.009337 (+ 3864us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.1.135.193:46481: BlockingRecv error: recv got EOF from 127.1.135.193:46481 (error 108)
Metrics: {"server-negotiator.queue_time_us":146,"thread_start_us":85,"threads_started":1}
W20260923 22:12:58.010099 10428 heartbeater.cc:644] Failed to heartbeat to 127.1.135.254:38637 (0 consecutive failures): Not authorized: Failed to ping master at 127.1.135.254:38637: Client connection negotiation failed: client connection to 127.1.135.254:38637: Server oryx/127.1.135.254@KRBTEST.COM not found in Kerberos database
Sep 23 22:12:58 dist-test-slave-s2fj krb5kdc[9697](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.1.135.193@KRBTEST.COM for oryx/127.1.135.254@KRBTEST.COM, Server not found in Kerberos database
Sep 23 22:12:58 dist-test-slave-s2fj krb5kdc[9697](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.1.135.193@KRBTEST.COM for oryx/127.1.135.254@KRBTEST.COM, Server not found in Kerberos database
I20260923 22:12:58.015516 10430 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 22:12:58.010344 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.135.254:38637 (local address 127.1.135.193:36107)
0923 22:12:58.010388 (+ 44us) negotiation.cc:107] Waiting for socket to connect
0923 22:12:58.010398 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0923 22:12:58.010446 (+ 48us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0923 22:12:58.010603 (+ 157us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0923 22:12:58.010604 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0923 22:12:58.010764 (+ 160us) client_negotiation.cc:189] Negotiated authn=SASL
0923 22:12:58.010819 (+ 55us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 22:12:58.010821 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 22:12:58.011722 (+ 901us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 22:12:58.011723 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 22:12:58.012660 (+ 937us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 22:12:58.012663 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 22:12:58.013116 (+ 453us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 22:12:58.013116 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 22:12:58.013179 (+ 63us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 22:12:58.013613 (+ 434us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0923 22:12:58.013618 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0923 22:12:58.015424 (+ 1806us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.1.135.254:38637: Server oryx/127.1.135.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":29}
W20260923 22:12:58.015939 10431 negotiation.cc:337] Failed RPC negotiation. Trace:
0923 22:12:58.010376 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.135.193:36107 (local address 127.1.135.254:38637)
0923 22:12:58.010422 (+ 46us) server_negotiation.cc:206] Beginning negotiation
0923 22:12:58.010424 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0923 22:12:58.010433 (+ 9us) server_negotiation.cc:407] Connection header received
0923 22:12:58.010511 (+ 78us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 22:12:58.010513 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 22:12:58.010524 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 22:12:58.010588 (+ 64us) server_negotiation.cc:226] Negotiated authn=SASL
0923 22:12:58.010902 (+ 314us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 22:12:58.011638 (+ 736us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 22:12:58.012747 (+ 1109us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 22:12:58.013031 (+ 284us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 22:12:58.013110 (+ 79us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 22:12:58.015886 (+ 2776us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.1.135.193:36107: BlockingRecv error: recv got EOF from 127.1.135.193:36107 (error 108)
Metrics: {"server-negotiator.queue_time_us":25}
Sep 23 22:12:58 dist-test-slave-s2fj krb5kdc[9697](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.1.135.193@KRBTEST.COM for oryx/127.1.135.254@KRBTEST.COM, Server not found in Kerberos database
Sep 23 22:12:58 dist-test-slave-s2fj krb5kdc[9697](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.1.135.193@KRBTEST.COM for oryx/127.1.135.254@KRBTEST.COM, Server not found in Kerberos database
I20260923 22:12:58.021394 10433 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 22:12:58.015719 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.135.254:38637 (local address 127.1.135.193:43547)
0923 22:12:58.015940 (+ 221us) negotiation.cc:107] Waiting for socket to connect
0923 22:12:58.015951 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0923 22:12:58.016145 (+ 194us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0923 22:12:58.016298 (+ 153us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0923 22:12:58.016298 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0923 22:12:58.016474 (+ 176us) client_negotiation.cc:189] Negotiated authn=SASL
0923 22:12:58.016529 (+ 55us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 22:12:58.016531 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 22:12:58.017384 (+ 853us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 22:12:58.017385 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 22:12:58.018217 (+ 832us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 22:12:58.018220 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 22:12:58.018689 (+ 469us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 22:12:58.018690 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 22:12:58.018757 (+ 67us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 22:12:58.019250 (+ 493us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0923 22:12:58.019255 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0923 22:12:58.021313 (+ 2058us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.1.135.254:38637: Server oryx/127.1.135.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":204,"thread_start_us":72,"threads_started":1}
W20260923 22:12:58.021482 10431 negotiation.cc:337] Failed RPC negotiation. Trace:
0923 22:12:58.015780 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.135.193:43547 (local address 127.1.135.254:38637)
0923 22:12:58.016018 (+ 238us) server_negotiation.cc:206] Beginning negotiation
0923 22:12:58.016020 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0923 22:12:58.016028 (+ 8us) server_negotiation.cc:407] Connection header received
0923 22:12:58.016213 (+ 185us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 22:12:58.016214 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 22:12:58.016224 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 22:12:58.016281 (+ 57us) server_negotiation.cc:226] Negotiated authn=SASL
0923 22:12:58.016600 (+ 319us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 22:12:58.017301 (+ 701us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 22:12:58.018302 (+ 1001us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 22:12:58.018595 (+ 293us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 22:12:58.018660 (+ 65us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 22:12:58.021419 (+ 2759us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.1.135.193:43547: BlockingRecv error: recv got EOF from 127.1.135.193:43547 (error 108)
Metrics: {"server-negotiator.queue_time_us":225,"thread_start_us":94,"threads_started":1}
W20260923 22:12:58.021708 10428 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
W20260923 22:12:58.084923 10432 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260923 22:12:58.085143 10432 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260923 22:12:58.085179 10432 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260923 22:12:58.086308 10432 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260923 22:12:58.086370 10432 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260923 22:12:58.086414 10432 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.1.135.194
I20260923 22:12:58.087811 10432 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.1.135.212:36077
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskRiQe_a/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790201539513534-1567-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskRiQe_a/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790201539513534-1567-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskRiQe_a/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790201539513534-1567-0/krb5kdc/oryx_127.1.135.194.keytab
--principal=oryx/127.1.135.194
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.1.135.194:35425
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskRiQe_a/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790201539513534-1567-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.1.135.194
--webserver_port=32885
--tserver_master_addrs=127.1.135.254:38637
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.10432
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.1.135.194
--log_dir=/tmp/dist-test-taskRiQe_a/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790201539513534-1567-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 23 Sep 2026 20:53:37 UTC on ccycloud.quasar-rkviyx.root.comops.site
build id 4668083
I20260923 22:12:58.088102 10432 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260923 22:12:58.088313 10432 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260923 22:12:58.092995 10440 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260923 22:12:58.092989 10439 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260923 22:12:58.093211 10432 server_base.cc:1034] running on GCE node
W20260923 22:12:58.092989 10442 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260923 22:12:58.093441 10432 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260923 22:12:58.093659 10432 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260923 22:12:58.094813 10432 hybrid_clock.cc:648] HybridClock initialized: now 1790201578094810 us; error 34 us; skew 500 ppm
Sep 23 22:12:58 dist-test-slave-s2fj krb5kdc[9697](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790201578, etypes {rep=17 tkt=17 ses=17}, oryx/127.1.135.194@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260923 22:12:58.097285 10432 init.cc:377] Logged in from keytab as oryx/127.1.135.194@KRBTEST.COM (short username oryx)
I20260923 22:12:58.097694 10432 webserver.cc:460] Webserver started at http://127.1.135.194:32885/ using document root <none> and password file <none>
I20260923 22:12:58.097930 10432 fs_manager.cc:359] Metadata directory not provided
I20260923 22:12:58.098013 10432 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260923 22:12:58.099308 10432 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.001s sys 0.000s
I20260923 22:12:58.099818 10449 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260923 22:12:58.100013 10432 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260923 22:12:58.100126 10432 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskRiQe_a/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790201539513534-1567-0/minicluster-data/ts-1/data,/tmp/dist-test-taskRiQe_a/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790201539513534-1567-0/minicluster-data/ts-1/wal
uuid: "f45403e0b77b4fbca524bf4ef8d8a1ca"
format_stamp: "Formatted at 2026-09-23 22:12:57 on dist-test-slave-s2fj"
server_key: "fbf10af433d233810441927c823de5cf"
server_key_iv: "95fe80671fb302d0a0b84e71ce180aab"
server_key_version: "encryptionkey@0"
I20260923 22:12:58.100520 10432 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskRiQe_a/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790201539513534-1567-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskRiQe_a/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790201539513534-1567-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskRiQe_a/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790201539513534-1567-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260923 22:12:58.111791 10432 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260923 22:12:58.113858 10432 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260923 22:12:58.113968 10432 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260923 22:12:58.114125 10432 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260923 22:12:58.114362 10432 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260923 22:12:58.114388 10432 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260923 22:12:58.114408 10432 ts_tablet_manager.cc:610] Registered 0 tablets
I20260923 22:12:58.114421 10432 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260923 22:12:58.119989 10432 rpc_server.cc:307] RPC server started. Bound to: 127.1.135.194:35425
I20260923 22:12:58.120026 10562 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.1.135.194:35425 every 8 connection(s)
I20260923 22:12:58.120404 10432 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskRiQe_a/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790201539513534-1567-0/minicluster-data/ts-1/data/info.pb
Sep 23 22:12:58 dist-test-slave-s2fj krb5kdc[9697](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.1.135.194@KRBTEST.COM for oryx/127.1.135.254@KRBTEST.COM, Server not found in Kerberos database
Sep 23 22:12:58 dist-test-slave-s2fj krb5kdc[9697](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.1.135.194@KRBTEST.COM for oryx/127.1.135.254@KRBTEST.COM, Server not found in Kerberos database
I20260923 22:12:58.126981 1567 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskRiQe_a/build/release/bin/kudu as pid 10432
W20260923 22:12:58.127224 10431 negotiation.cc:337] Failed RPC negotiation. Trace:
0923 22:12:58.121068 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.135.194:41501 (local address 127.1.135.254:38637)
0923 22:12:58.121114 (+ 46us) server_negotiation.cc:206] Beginning negotiation
0923 22:12:58.121116 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0923 22:12:58.121332 (+ 216us) server_negotiation.cc:407] Connection header received
0923 22:12:58.121731 (+ 399us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 22:12:58.121732 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 22:12:58.121746 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 22:12:58.121804 (+ 58us) server_negotiation.cc:226] Negotiated authn=SASL
0923 22:12:58.122601 (+ 797us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 22:12:58.123263 (+ 662us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 22:12:58.124202 (+ 939us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 22:12:58.124405 (+ 203us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 22:12:58.124458 (+ 53us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 22:12:58.127179 (+ 2721us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.1.135.194:41501: BlockingRecv error: recv got EOF from 127.1.135.194:41501 (error 108)
Metrics: {"server-negotiator.queue_time_us":18}
I20260923 22:12:58.127305 1567 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskRiQe_a/build/release/bin/kudu
/tmp/dist-test-taskRiQe_a/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskRiQe_a/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790201539513534-1567-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskRiQe_a/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790201539513534-1567-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskRiQe_a/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790201539513534-1567-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskRiQe_a/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790201539513534-1567-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.1.135.195:38051
--local_ip_for_outbound_sockets=127.1.135.195
--tserver_master_addrs=127.1.135.254:38637
--webserver_port=33823
--webserver_interface=127.1.135.195
--keytab_file=/tmp/dist-test-taskRiQe_a/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790201539513534-1567-0/krb5kdc/kudu_127.1.135.195.keytab
--principal=kudu/127.1.135.195
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.1.135.212:36077
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--principal=oryx/127.1.135.195
--keytab_file=/tmp/dist-test-taskRiQe_a/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790201539513534-1567-0/krb5kdc/oryx_127.1.135.195.keytab with env {KRB5CCNAME=/tmp/dist-test-taskRiQe_a/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790201539513534-1567-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskRiQe_a/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790201539513534-1567-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskRiQe_a/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790201539513534-1567-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260923 22:12:58.127342 10565 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 22:12:58.121063 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.135.254:38637 (local address 127.1.135.194:41501)
0923 22:12:58.121206 (+ 143us) negotiation.cc:107] Waiting for socket to connect
0923 22:12:58.121237 (+ 31us) client_negotiation.cc:174] Beginning negotiation
0923 22:12:58.121657 (+ 420us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0923 22:12:58.121848 (+ 191us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0923 22:12:58.121851 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0923 22:12:58.122354 (+ 503us) client_negotiation.cc:189] Negotiated authn=SASL
0923 22:12:58.122529 (+ 175us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 22:12:58.122533 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 22:12:58.123342 (+ 809us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 22:12:58.123343 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 22:12:58.124128 (+ 785us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 22:12:58.124130 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 22:12:58.124481 (+ 351us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 22:12:58.124482 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 22:12:58.124576 (+ 94us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 22:12:58.124969 (+ 393us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0923 22:12:58.124977 (+ 8us) client_negotiation.cc:656] calling sasl_client_start()
0923 22:12:58.127088 (+ 2111us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.1.135.254:38637: Server oryx/127.1.135.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":109,"thread_start_us":84,"threads_started":1}
W20260923 22:12:58.127563 10563 heartbeater.cc:644] Failed to heartbeat to 127.1.135.254:38637 (0 consecutive failures): Not authorized: Failed to ping master at 127.1.135.254:38637: Client connection negotiation failed: client connection to 127.1.135.254:38637: Server oryx/127.1.135.254@KRBTEST.COM not found in Kerberos database
Sep 23 22:12:58 dist-test-slave-s2fj krb5kdc[9697](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.1.135.194@KRBTEST.COM for oryx/127.1.135.254@KRBTEST.COM, Server not found in Kerberos database
Sep 23 22:12:58 dist-test-slave-s2fj krb5kdc[9697](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.1.135.194@KRBTEST.COM for oryx/127.1.135.254@KRBTEST.COM, Server not found in Kerberos database
I20260923 22:12:58.132373 10565 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 22:12:58.127772 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.135.254:38637 (local address 127.1.135.194:37151)
0923 22:12:58.127812 (+ 40us) negotiation.cc:107] Waiting for socket to connect
0923 22:12:58.127821 (+ 9us) client_negotiation.cc:174] Beginning negotiation
0923 22:12:58.127858 (+ 37us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0923 22:12:58.127998 (+ 140us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0923 22:12:58.127998 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0923 22:12:58.128159 (+ 161us) client_negotiation.cc:189] Negotiated authn=SASL
0923 22:12:58.128205 (+ 46us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 22:12:58.128206 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 22:12:58.128981 (+ 775us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 22:12:58.128982 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 22:12:58.129517 (+ 535us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 22:12:58.129519 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 22:12:58.129842 (+ 323us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 22:12:58.129843 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 22:12:58.129887 (+ 44us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 22:12:58.130293 (+ 406us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0923 22:12:58.130297 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0923 22:12:58.132277 (+ 1980us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.1.135.254:38637: Server oryx/127.1.135.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":28}
W20260923 22:12:58.132429 10431 negotiation.cc:337] Failed RPC negotiation. Trace:
0923 22:12:58.127840 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.135.194:37151 (local address 127.1.135.254:38637)
0923 22:12:58.127881 (+ 41us) server_negotiation.cc:206] Beginning negotiation
0923 22:12:58.127883 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0923 22:12:58.127892 (+ 9us) server_negotiation.cc:407] Connection header received
0923 22:12:58.127915 (+ 23us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 22:12:58.127916 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 22:12:58.127925 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 22:12:58.127986 (+ 61us) server_negotiation.cc:226] Negotiated authn=SASL
0923 22:12:58.128290 (+ 304us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 22:12:58.128898 (+ 608us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 22:12:58.129593 (+ 695us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 22:12:58.129770 (+ 177us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 22:12:58.129820 (+ 50us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 22:12:58.132362 (+ 2542us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.1.135.194:37151: BlockingRecv error: recv got EOF from 127.1.135.194:37151 (error 108)
Metrics: {"server-negotiator.queue_time_us":19}
Sep 23 22:12:58 dist-test-slave-s2fj krb5kdc[9697](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.1.135.194@KRBTEST.COM for oryx/127.1.135.254@KRBTEST.COM, Server not found in Kerberos database
Sep 23 22:12:58 dist-test-slave-s2fj krb5kdc[9697](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.1.135.194@KRBTEST.COM for oryx/127.1.135.254@KRBTEST.COM, Server not found in Kerberos database
I20260923 22:12:58.136932 10565 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 22:12:58.132748 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.135.254:38637 (local address 127.1.135.194:57923)
0923 22:12:58.132802 (+ 54us) negotiation.cc:107] Waiting for socket to connect
0923 22:12:58.132813 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0923 22:12:58.132857 (+ 44us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0923 22:12:58.132973 (+ 116us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0923 22:12:58.132974 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0923 22:12:58.133102 (+ 128us) client_negotiation.cc:189] Negotiated authn=SASL
0923 22:12:58.133128 (+ 26us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 22:12:58.133129 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 22:12:58.133731 (+ 602us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 22:12:58.133732 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 22:12:58.134232 (+ 500us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 22:12:58.134234 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 22:12:58.134601 (+ 367us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 22:12:58.134602 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 22:12:58.134653 (+ 51us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 22:12:58.135051 (+ 398us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0923 22:12:58.135056 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0923 22:12:58.136847 (+ 1791us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.1.135.254:38637: Server oryx/127.1.135.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":37}
W20260923 22:12:58.136979 10431 negotiation.cc:337] Failed RPC negotiation. Trace:
0923 22:12:58.132831 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.135.194:57923 (local address 127.1.135.254:38637)
0923 22:12:58.132867 (+ 36us) server_negotiation.cc:206] Beginning negotiation
0923 22:12:58.132869 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0923 22:12:58.132879 (+ 10us) server_negotiation.cc:407] Connection header received
0923 22:12:58.132902 (+ 23us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 22:12:58.132903 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 22:12:58.132913 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 22:12:58.132969 (+ 56us) server_negotiation.cc:226] Negotiated authn=SASL
0923 22:12:58.133209 (+ 240us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 22:12:58.133655 (+ 446us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 22:12:58.134327 (+ 672us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 22:12:58.134517 (+ 190us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 22:12:58.134591 (+ 74us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 22:12:58.136915 (+ 2324us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.1.135.194:57923: BlockingRecv error: recv got EOF from 127.1.135.194:57923 (error 108)
Metrics: {"server-negotiator.queue_time_us":18}
W20260923 22:12:58.137192 10563 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
W20260923 22:12:58.207921 10566 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260923 22:12:58.208151 10566 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260923 22:12:58.208189 10566 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260923 22:12:58.209338 10566 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260923 22:12:58.209398 10566 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260923 22:12:58.209453 10566 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.1.135.195
I20260923 22:12:58.210767 10566 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.1.135.212:36077
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskRiQe_a/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790201539513534-1567-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskRiQe_a/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790201539513534-1567-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskRiQe_a/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790201539513534-1567-0/krb5kdc/oryx_127.1.135.195.keytab
--principal=oryx/127.1.135.195
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.1.135.195:38051
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskRiQe_a/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790201539513534-1567-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.1.135.195
--webserver_port=33823
--tserver_master_addrs=127.1.135.254:38637
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.10566
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.1.135.195
--log_dir=/tmp/dist-test-taskRiQe_a/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790201539513534-1567-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 23 Sep 2026 20:53:37 UTC on ccycloud.quasar-rkviyx.root.comops.site
build id 4668083
I20260923 22:12:58.211007 10566 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260923 22:12:58.211211 10566 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260923 22:12:58.215806 10574 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260923 22:12:58.215814 10572 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260923 22:12:58.215814 10571 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260923 22:12:58.216014 10566 server_base.cc:1034] running on GCE node
I20260923 22:12:58.216262 10566 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260923 22:12:58.216511 10566 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260923 22:12:58.217657 10566 hybrid_clock.cc:648] HybridClock initialized: now 1790201578217644 us; error 25 us; skew 500 ppm
Sep 23 22:12:58 dist-test-slave-s2fj krb5kdc[9697](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790201578, etypes {rep=17 tkt=17 ses=17}, oryx/127.1.135.195@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260923 22:12:58.219995 10566 init.cc:377] Logged in from keytab as oryx/127.1.135.195@KRBTEST.COM (short username oryx)
I20260923 22:12:58.220474 10566 webserver.cc:460] Webserver started at http://127.1.135.195:33823/ using document root <none> and password file <none>
I20260923 22:12:58.220649 10566 fs_manager.cc:359] Metadata directory not provided
I20260923 22:12:58.220709 10566 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260923 22:12:58.221887 10566 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.001s sys 0.000s
I20260923 22:12:58.222384 10581 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260923 22:12:58.222558 10566 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260923 22:12:58.222635 10566 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskRiQe_a/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790201539513534-1567-0/minicluster-data/ts-2/data,/tmp/dist-test-taskRiQe_a/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790201539513534-1567-0/minicluster-data/ts-2/wal
uuid: "bd4dac402de8439f865e36d987a8b151"
format_stamp: "Formatted at 2026-09-23 22:12:57 on dist-test-slave-s2fj"
server_key: "ab40aeab9db52c37193c4f9ee4b84869"
server_key_iv: "47cd8ad09140cd87337a6c8fb17e7ae0"
server_key_version: "encryptionkey@0"
I20260923 22:12:58.222924 10566 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskRiQe_a/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790201539513534-1567-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskRiQe_a/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790201539513534-1567-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskRiQe_a/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790201539513534-1567-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260923 22:12:58.234068 10566 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260923 22:12:58.236219 10566 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260923 22:12:58.236322 10566 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260923 22:12:58.236502 10566 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260923 22:12:58.236733 10566 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260923 22:12:58.236761 10566 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260923 22:12:58.236780 10566 ts_tablet_manager.cc:610] Registered 0 tablets
I20260923 22:12:58.236845 10566 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260923 22:12:58.242581 10566 rpc_server.cc:307] RPC server started. Bound to: 127.1.135.195:38051
I20260923 22:12:58.242674 10694 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.1.135.195:38051 every 8 connection(s)
I20260923 22:12:58.242990 10566 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskRiQe_a/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790201539513534-1567-0/minicluster-data/ts-2/data/info.pb
Sep 23 22:12:58 dist-test-slave-s2fj krb5kdc[9697](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.1.135.195@KRBTEST.COM for oryx/127.1.135.254@KRBTEST.COM, Server not found in Kerberos database
Sep 23 22:12:58 dist-test-slave-s2fj krb5kdc[9697](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.1.135.195@KRBTEST.COM for oryx/127.1.135.254@KRBTEST.COM, Server not found in Kerberos database
W20260923 22:12:58.249577 10431 negotiation.cc:337] Failed RPC negotiation. Trace:
0923 22:12:58.243666 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.135.195:39699 (local address 127.1.135.254:38637)
0923 22:12:58.243726 (+ 60us) server_negotiation.cc:206] Beginning negotiation
0923 22:12:58.243728 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0923 22:12:58.243919 (+ 191us) server_negotiation.cc:407] Connection header received
0923 22:12:58.244458 (+ 539us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 22:12:58.244459 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 22:12:58.244470 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 22:12:58.244521 (+ 51us) server_negotiation.cc:226] Negotiated authn=SASL
0923 22:12:58.245351 (+ 830us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 22:12:58.245820 (+ 469us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 22:12:58.246731 (+ 911us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 22:12:58.246907 (+ 176us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 22:12:58.246957 (+ 50us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 22:12:58.249523 (+ 2566us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.1.135.195:39699: BlockingRecv error: recv got EOF from 127.1.135.195:39699 (error 108)
Metrics: {"server-negotiator.queue_time_us":19}
I20260923 22:12:58.249637 10697 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 22:12:58.243637 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.135.254:38637 (local address 127.1.135.195:39699)
0923 22:12:58.243805 (+ 168us) negotiation.cc:107] Waiting for socket to connect
0923 22:12:58.243836 (+ 31us) client_negotiation.cc:174] Beginning negotiation
0923 22:12:58.244361 (+ 525us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0923 22:12:58.244571 (+ 210us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0923 22:12:58.244574 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0923 22:12:58.245106 (+ 532us) client_negotiation.cc:189] Negotiated authn=SASL
0923 22:12:58.245280 (+ 174us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 22:12:58.245285 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 22:12:58.245893 (+ 608us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 22:12:58.245894 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 22:12:58.246656 (+ 762us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 22:12:58.246658 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 22:12:58.246980 (+ 322us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 22:12:58.246981 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 22:12:58.247076 (+ 95us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 22:12:58.247470 (+ 394us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0923 22:12:58.247478 (+ 8us) client_negotiation.cc:656] calling sasl_client_start()
0923 22:12:58.249454 (+ 1976us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.1.135.254:38637: Server oryx/127.1.135.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":123,"thread_start_us":98,"threads_started":1}
W20260923 22:12:58.249846 10695 heartbeater.cc:644] Failed to heartbeat to 127.1.135.254:38637 (0 consecutive failures): Not authorized: Failed to ping master at 127.1.135.254:38637: Client connection negotiation failed: client connection to 127.1.135.254:38637: Server oryx/127.1.135.254@KRBTEST.COM not found in Kerberos database
I20260923 22:12:58.252724 1567 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskRiQe_a/build/release/bin/kudu as pid 10566
Sep 23 22:12:58 dist-test-slave-s2fj krb5kdc[9697](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.1.135.195@KRBTEST.COM for oryx/127.1.135.254@KRBTEST.COM, Server not found in Kerberos database
Sep 23 22:12:58 dist-test-slave-s2fj krb5kdc[9697](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.1.135.195@KRBTEST.COM for oryx/127.1.135.254@KRBTEST.COM, Server not found in Kerberos database
I20260923 22:12:58.254101 10697 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 22:12:58.250059 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.135.254:38637 (local address 127.1.135.195:36375)
0923 22:12:58.250106 (+ 47us) negotiation.cc:107] Waiting for socket to connect
0923 22:12:58.250116 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0923 22:12:58.250188 (+ 72us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0923 22:12:58.250390 (+ 202us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0923 22:12:58.250391 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0923 22:12:58.250523 (+ 132us) client_negotiation.cc:189] Negotiated authn=SASL
0923 22:12:58.250563 (+ 40us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 22:12:58.250564 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 22:12:58.251162 (+ 598us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 22:12:58.251163 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 22:12:58.251671 (+ 508us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 22:12:58.251672 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 22:12:58.252028 (+ 356us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 22:12:58.252029 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 22:12:58.252113 (+ 84us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 22:12:58.252462 (+ 349us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0923 22:12:58.252466 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0923 22:12:58.254062 (+ 1596us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.1.135.254:38637: Server oryx/127.1.135.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":32}
W20260923 22:12:58.254209 10431 negotiation.cc:337] Failed RPC negotiation. Trace:
0923 22:12:58.250239 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.135.195:36375 (local address 127.1.135.254:38637)
0923 22:12:58.250294 (+ 55us) server_negotiation.cc:206] Beginning negotiation
0923 22:12:58.250296 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0923 22:12:58.250302 (+ 6us) server_negotiation.cc:407] Connection header received
0923 22:12:58.250319 (+ 17us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 22:12:58.250319 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 22:12:58.250326 (+ 7us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 22:12:58.250388 (+ 62us) server_negotiation.cc:226] Negotiated authn=SASL
0923 22:12:58.250643 (+ 255us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 22:12:58.251088 (+ 445us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 22:12:58.251749 (+ 661us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 22:12:58.251954 (+ 205us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 22:12:58.252031 (+ 77us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 22:12:58.254156 (+ 2125us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.1.135.195:36375: BlockingRecv error: recv got EOF from 127.1.135.195:36375 (error 108)
Metrics: {"server-negotiator.queue_time_us":38}
Sep 23 22:12:58 dist-test-slave-s2fj krb5kdc[9697](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.1.135.195@KRBTEST.COM for oryx/127.1.135.254@KRBTEST.COM, Server not found in Kerberos database
Sep 23 22:12:58 dist-test-slave-s2fj krb5kdc[9697](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.1.135.195@KRBTEST.COM for oryx/127.1.135.254@KRBTEST.COM, Server not found in Kerberos database
I20260923 22:12:58.258569 10697 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 22:12:58.254522 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.135.254:38637 (local address 127.1.135.195:34893)
0923 22:12:58.254569 (+ 47us) negotiation.cc:107] Waiting for socket to connect
0923 22:12:58.254578 (+ 9us) client_negotiation.cc:174] Beginning negotiation
0923 22:12:58.254612 (+ 34us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0923 22:12:58.254725 (+ 113us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0923 22:12:58.254725 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0923 22:12:58.254859 (+ 134us) client_negotiation.cc:189] Negotiated authn=SASL
0923 22:12:58.254887 (+ 28us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 22:12:58.254888 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 22:12:58.255411 (+ 523us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 22:12:58.255412 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 22:12:58.256062 (+ 650us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 22:12:58.256064 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 22:12:58.256513 (+ 449us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 22:12:58.256514 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 22:12:58.256570 (+ 56us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 22:12:58.256878 (+ 308us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0923 22:12:58.256884 (+ 6us) client_negotiation.cc:656] calling sasl_client_start()
0923 22:12:58.258495 (+ 1611us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.1.135.254:38637: Server oryx/127.1.135.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":33}
W20260923 22:12:58.258620 10431 negotiation.cc:337] Failed RPC negotiation. Trace:
0923 22:12:58.254575 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.135.195:34893 (local address 127.1.135.254:38637)
0923 22:12:58.254612 (+ 37us) server_negotiation.cc:206] Beginning negotiation
0923 22:12:58.254614 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0923 22:12:58.254622 (+ 8us) server_negotiation.cc:407] Connection header received
0923 22:12:58.254661 (+ 39us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 22:12:58.254662 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 22:12:58.254670 (+ 8us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 22:12:58.254721 (+ 51us) server_negotiation.cc:226] Negotiated authn=SASL
0923 22:12:58.254956 (+ 235us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 22:12:58.255342 (+ 386us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 22:12:58.256189 (+ 847us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 22:12:58.256445 (+ 256us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 22:12:58.256495 (+ 50us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 22:12:58.258558 (+ 2063us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.1.135.195:34893: BlockingRecv error: recv got EOF from 127.1.135.195:34893 (error 108)
Metrics: {"server-negotiator.queue_time_us":17}
W20260923 22:12:58.258941 10695 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
I20260923 22:12:58.260149 10434 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 22:12:58.253131 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46750 (local address 127.1.135.254:38637)
0923 22:12:58.253181 (+ 50us) server_negotiation.cc:206] Beginning negotiation
0923 22:12:58.253183 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0923 22:12:58.253330 (+ 147us) server_negotiation.cc:407] Connection header received
0923 22:12:58.253371 (+ 41us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 22:12:58.253372 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 22:12:58.253385 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 22:12:58.253458 (+ 73us) server_negotiation.cc:226] Negotiated authn=SASL
0923 22:12:58.254165 (+ 707us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 22:12:58.254811 (+ 646us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 22:12:58.255646 (+ 835us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 22:12:58.255814 (+ 168us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 22:12:58.255859 (+ 45us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 22:12:58.257275 (+ 1416us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0923 22:12:58.257280 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0923 22:12:58.257280 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0923 22:12:58.257289 (+ 9us) server_negotiation.cc:922] calling sasl_server_start()
0923 22:12:58.259046 (+ 1757us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0923 22:12:58.259342 (+ 296us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0923 22:12:58.259344 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0923 22:12:58.259344 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0923 22:12:58.259433 (+ 89us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0923 22:12:58.259567 (+ 134us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0923 22:12:58.259567 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0923 22:12:58.259567 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0923 22:12:58.259758 (+ 191us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0923 22:12:58.259849 (+ 91us) server_negotiation.cc:1035] Waiting for connection context
0923 22:12:58.259940 (+ 91us) server_negotiation.cc:299] Negotiation successful
0923 22:12:58.260013 (+ 73us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":24}
Sep 23 22:12:59 dist-test-slave-s2fj krb5kdc[9697](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.1.135.193@KRBTEST.COM for oryx/127.1.135.254@KRBTEST.COM, Server not found in Kerberos database
Sep 23 22:12:59 dist-test-slave-s2fj krb5kdc[9697](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.1.135.193@KRBTEST.COM for oryx/127.1.135.254@KRBTEST.COM, Server not found in Kerberos database
I20260923 22:12:59.026523 10699 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 22:12:59.022097 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.135.254:38637 (local address 127.1.135.193:34623)
0923 22:12:59.022234 (+ 137us) negotiation.cc:107] Waiting for socket to connect
0923 22:12:59.022245 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0923 22:12:59.022297 (+ 52us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0923 22:12:59.022415 (+ 118us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0923 22:12:59.022416 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0923 22:12:59.022586 (+ 170us) client_negotiation.cc:189] Negotiated authn=SASL
0923 22:12:59.022635 (+ 49us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 22:12:59.022636 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 22:12:59.023269 (+ 633us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 22:12:59.023269 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 22:12:59.023783 (+ 514us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 22:12:59.023785 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 22:12:59.024126 (+ 341us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 22:12:59.024127 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 22:12:59.024178 (+ 51us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 22:12:59.024610 (+ 432us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0923 22:12:59.024615 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0923 22:12:59.026434 (+ 1819us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.1.135.254:38637: Server oryx/127.1.135.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":119,"thread_start_us":78,"threads_started":1}
W20260923 22:12:59.026662 10700 negotiation.cc:337] Failed RPC negotiation. Trace:
0923 22:12:59.022181 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.135.193:34623 (local address 127.1.135.254:38637)
0923 22:12:59.022300 (+ 119us) server_negotiation.cc:206] Beginning negotiation
0923 22:12:59.022302 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0923 22:12:59.022317 (+ 15us) server_negotiation.cc:407] Connection header received
0923 22:12:59.022338 (+ 21us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 22:12:59.022338 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 22:12:59.022347 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 22:12:59.022401 (+ 54us) server_negotiation.cc:226] Negotiated authn=SASL
0923 22:12:59.022712 (+ 311us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 22:12:59.023208 (+ 496us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 22:12:59.023849 (+ 641us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 22:12:59.024026 (+ 177us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 22:12:59.024089 (+ 63us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 22:12:59.026548 (+ 2459us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.1.135.193:34623: BlockingRecv error: recv got EOF from 127.1.135.193:34623 (error 108)
Metrics: {"server-negotiator.queue_time_us":97,"thread_start_us":69,"threads_started":1}
Sep 23 22:12:59 dist-test-slave-s2fj krb5kdc[9697](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.1.135.194@KRBTEST.COM for oryx/127.1.135.254@KRBTEST.COM, Server not found in Kerberos database
Sep 23 22:12:59 dist-test-slave-s2fj krb5kdc[9697](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.1.135.194@KRBTEST.COM for oryx/127.1.135.254@KRBTEST.COM, Server not found in Kerberos database
I20260923 22:12:59.141855 10701 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 22:12:59.137557 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.135.254:38637 (local address 127.1.135.194:48997)
0923 22:12:59.137702 (+ 145us) negotiation.cc:107] Waiting for socket to connect
0923 22:12:59.137713 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0923 22:12:59.137771 (+ 58us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0923 22:12:59.137894 (+ 123us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0923 22:12:59.137895 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0923 22:12:59.138101 (+ 206us) client_negotiation.cc:189] Negotiated authn=SASL
0923 22:12:59.138148 (+ 47us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 22:12:59.138150 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 22:12:59.138831 (+ 681us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 22:12:59.138832 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 22:12:59.139364 (+ 532us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 22:12:59.139365 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 22:12:59.139667 (+ 302us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 22:12:59.139668 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 22:12:59.139705 (+ 37us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 22:12:59.140178 (+ 473us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0923 22:12:59.140183 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0923 22:12:59.141790 (+ 1607us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.1.135.254:38637: Server oryx/127.1.135.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":128,"thread_start_us":91,"threads_started":1}
W20260923 22:12:59.141976 10700 negotiation.cc:337] Failed RPC negotiation. Trace:
0923 22:12:59.137625 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.135.194:48997 (local address 127.1.135.254:38637)
0923 22:12:59.137677 (+ 52us) server_negotiation.cc:206] Beginning negotiation
0923 22:12:59.137679 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0923 22:12:59.137756 (+ 77us) server_negotiation.cc:407] Connection header received
0923 22:12:59.137822 (+ 66us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 22:12:59.137822 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 22:12:59.137833 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 22:12:59.137876 (+ 43us) server_negotiation.cc:226] Negotiated authn=SASL
0923 22:12:59.138209 (+ 333us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 22:12:59.138766 (+ 557us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 22:12:59.139429 (+ 663us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 22:12:59.139606 (+ 177us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 22:12:59.139654 (+ 48us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 22:12:59.141879 (+ 2225us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.1.135.194:48997: BlockingRecv error: recv got EOF from 127.1.135.194:48997 (error 108)
Metrics: {"server-negotiator.queue_time_us":22}
Sep 23 22:12:59 dist-test-slave-s2fj krb5kdc[9697](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.1.135.195@KRBTEST.COM for oryx/127.1.135.254@KRBTEST.COM, Server not found in Kerberos database
Sep 23 22:12:59 dist-test-slave-s2fj krb5kdc[9697](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.1.135.195@KRBTEST.COM for oryx/127.1.135.254@KRBTEST.COM, Server not found in Kerberos database
I20260923 22:12:59.263923 10702 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 22:12:59.259426 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.135.254:38637 (local address 127.1.135.195:34771)
0923 22:12:59.259533 (+ 107us) negotiation.cc:107] Waiting for socket to connect
0923 22:12:59.259545 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0923 22:12:59.259603 (+ 58us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0923 22:12:59.259715 (+ 112us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0923 22:12:59.259716 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0923 22:12:59.259863 (+ 147us) client_negotiation.cc:189] Negotiated authn=SASL
0923 22:12:59.259909 (+ 46us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 22:12:59.259911 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 22:12:59.260569 (+ 658us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 22:12:59.260569 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 22:12:59.261090 (+ 521us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 22:12:59.261092 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 22:12:59.261431 (+ 339us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 22:12:59.261431 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 22:12:59.261469 (+ 38us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 22:12:59.261889 (+ 420us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0923 22:12:59.261893 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0923 22:12:59.263820 (+ 1927us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.1.135.254:38637: Server oryx/127.1.135.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":89,"thread_start_us":62,"threads_started":1}
W20260923 22:12:59.263960 10700 negotiation.cc:337] Failed RPC negotiation. Trace:
0923 22:12:59.259504 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.135.195:34771 (local address 127.1.135.254:38637)
0923 22:12:59.259579 (+ 75us) server_negotiation.cc:206] Beginning negotiation
0923 22:12:59.259581 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0923 22:12:59.259592 (+ 11us) server_negotiation.cc:407] Connection header received
0923 22:12:59.259657 (+ 65us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 22:12:59.259658 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 22:12:59.259670 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 22:12:59.259705 (+ 35us) server_negotiation.cc:226] Negotiated authn=SASL
0923 22:12:59.259966 (+ 261us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 22:12:59.260504 (+ 538us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 22:12:59.261164 (+ 660us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 22:12:59.261356 (+ 192us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 22:12:59.261418 (+ 62us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 22:12:59.263897 (+ 2479us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.1.135.195:34771: BlockingRecv error: recv got EOF from 127.1.135.195:34771 (error 108)
Metrics: {"server-negotiator.queue_time_us":40}
Sep 23 22:13:00 dist-test-slave-s2fj krb5kdc[9697](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.1.135.193@KRBTEST.COM for oryx/127.1.135.254@KRBTEST.COM, Server not found in Kerberos database
Sep 23 22:13:00 dist-test-slave-s2fj krb5kdc[9697](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.1.135.193@KRBTEST.COM for oryx/127.1.135.254@KRBTEST.COM, Server not found in Kerberos database
I20260923 22:13:00.031383 10703 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 22:13:00.027023 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.135.254:38637 (local address 127.1.135.193:33091)
0923 22:13:00.027132 (+ 109us) negotiation.cc:107] Waiting for socket to connect
0923 22:13:00.027149 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0923 22:13:00.027190 (+ 41us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0923 22:13:00.027401 (+ 211us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0923 22:13:00.027402 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0923 22:13:00.027551 (+ 149us) client_negotiation.cc:189] Negotiated authn=SASL
0923 22:13:00.027599 (+ 48us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 22:13:00.027601 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 22:13:00.028270 (+ 669us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 22:13:00.028271 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 22:13:00.028807 (+ 536us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 22:13:00.028809 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 22:13:00.029132 (+ 323us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 22:13:00.029133 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 22:13:00.029173 (+ 40us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 22:13:00.029596 (+ 423us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0923 22:13:00.029601 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0923 22:13:00.031326 (+ 1725us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.1.135.254:38637: Server oryx/127.1.135.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":90,"thread_start_us":63,"threads_started":1}
W20260923 22:13:00.031539 10704 negotiation.cc:337] Failed RPC negotiation. Trace:
0923 22:13:00.027077 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.135.193:33091 (local address 127.1.135.254:38637)
0923 22:13:00.027205 (+ 128us) server_negotiation.cc:206] Beginning negotiation
0923 22:13:00.027207 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0923 22:13:00.027216 (+ 9us) server_negotiation.cc:407] Connection header received
0923 22:13:00.027268 (+ 52us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 22:13:00.027269 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 22:13:00.027283 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 22:13:00.027363 (+ 80us) server_negotiation.cc:226] Negotiated authn=SASL
0923 22:13:00.027663 (+ 300us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 22:13:00.028174 (+ 511us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 22:13:00.028885 (+ 711us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 22:13:00.029068 (+ 183us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 22:13:00.029117 (+ 49us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 22:13:00.031449 (+ 2332us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.1.135.193:33091: BlockingRecv error: recv got EOF from 127.1.135.193:33091 (error 108)
Metrics: {"server-negotiator.queue_time_us":107,"thread_start_us":79,"threads_started":1}
Sep 23 22:13:00 dist-test-slave-s2fj krb5kdc[9697](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.1.135.194@KRBTEST.COM for oryx/127.1.135.254@KRBTEST.COM, Server not found in Kerberos database
Sep 23 22:13:00 dist-test-slave-s2fj krb5kdc[9697](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.1.135.194@KRBTEST.COM for oryx/127.1.135.254@KRBTEST.COM, Server not found in Kerberos database
I20260923 22:13:00.146731 10705 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 22:13:00.142287 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.135.254:38637 (local address 127.1.135.194:40915)
0923 22:13:00.142417 (+ 130us) negotiation.cc:107] Waiting for socket to connect
0923 22:13:00.142432 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0923 22:13:00.142478 (+ 46us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0923 22:13:00.142656 (+ 178us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0923 22:13:00.142656 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0923 22:13:00.142812 (+ 156us) client_negotiation.cc:189] Negotiated authn=SASL
0923 22:13:00.142857 (+ 45us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 22:13:00.142859 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 22:13:00.143467 (+ 608us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 22:13:00.143468 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 22:13:00.143976 (+ 508us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 22:13:00.143977 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 22:13:00.144316 (+ 339us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 22:13:00.144316 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 22:13:00.144356 (+ 40us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 22:13:00.144763 (+ 407us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0923 22:13:00.144767 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0923 22:13:00.146674 (+ 1907us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.1.135.254:38637: Server oryx/127.1.135.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":110,"thread_start_us":71,"threads_started":1}
W20260923 22:13:00.146857 10704 negotiation.cc:337] Failed RPC negotiation. Trace:
0923 22:13:00.142471 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.135.194:40915 (local address 127.1.135.254:38637)
0923 22:13:00.142536 (+ 65us) server_negotiation.cc:206] Beginning negotiation
0923 22:13:00.142539 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0923 22:13:00.142547 (+ 8us) server_negotiation.cc:407] Connection header received
0923 22:13:00.142570 (+ 23us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 22:13:00.142570 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 22:13:00.142580 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 22:13:00.142639 (+ 59us) server_negotiation.cc:226] Negotiated authn=SASL
0923 22:13:00.142921 (+ 282us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 22:13:00.143404 (+ 483us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 22:13:00.144044 (+ 640us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 22:13:00.144251 (+ 207us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 22:13:00.144300 (+ 49us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 22:13:00.146760 (+ 2460us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.1.135.194:40915: BlockingRecv error: recv got EOF from 127.1.135.194:40915 (error 108)
Metrics: {"server-negotiator.queue_time_us":35}
Sep 23 22:13:00 dist-test-slave-s2fj krb5kdc[9697](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.1.135.195@KRBTEST.COM for oryx/127.1.135.254@KRBTEST.COM, Server not found in Kerberos database
Sep 23 22:13:00 dist-test-slave-s2fj krb5kdc[9697](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.1.135.195@KRBTEST.COM for oryx/127.1.135.254@KRBTEST.COM, Server not found in Kerberos database
I20260923 22:13:00.268749 10706 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 22:13:00.264481 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.135.254:38637 (local address 127.1.135.195:51735)
0923 22:13:00.264588 (+ 107us) negotiation.cc:107] Waiting for socket to connect
0923 22:13:00.264599 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0923 22:13:00.264653 (+ 54us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0923 22:13:00.264807 (+ 154us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0923 22:13:00.264808 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0923 22:13:00.264999 (+ 191us) client_negotiation.cc:189] Negotiated authn=SASL
0923 22:13:00.265047 (+ 48us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 22:13:00.265049 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 22:13:00.265694 (+ 645us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 22:13:00.265694 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 22:13:00.266191 (+ 497us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 22:13:00.266192 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 22:13:00.266531 (+ 339us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 22:13:00.266532 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 22:13:00.266568 (+ 36us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 22:13:00.266986 (+ 418us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0923 22:13:00.266990 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0923 22:13:00.268695 (+ 1705us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.1.135.254:38637: Server oryx/127.1.135.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":87,"thread_start_us":60,"threads_started":1}
W20260923 22:13:00.268857 10704 negotiation.cc:337] Failed RPC negotiation. Trace:
0923 22:13:00.264579 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.135.195:51735 (local address 127.1.135.254:38637)
0923 22:13:00.264645 (+ 66us) server_negotiation.cc:206] Beginning negotiation
0923 22:13:00.264647 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0923 22:13:00.264655 (+ 8us) server_negotiation.cc:407] Connection header received
0923 22:13:00.264711 (+ 56us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 22:13:00.264712 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 22:13:00.264727 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 22:13:00.264777 (+ 50us) server_negotiation.cc:226] Negotiated authn=SASL
0923 22:13:00.265120 (+ 343us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 22:13:00.265626 (+ 506us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 22:13:00.266291 (+ 665us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 22:13:00.266467 (+ 176us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 22:13:00.266516 (+ 49us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 22:13:00.268782 (+ 2266us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.1.135.195:51735: BlockingRecv error: recv got EOF from 127.1.135.195:51735 (error 108)
Metrics: {"server-negotiator.queue_time_us":44}
Sep 23 22:13:01 dist-test-slave-s2fj krb5kdc[9697](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.1.135.193@KRBTEST.COM for oryx/127.1.135.254@KRBTEST.COM, Server not found in Kerberos database
Sep 23 22:13:01 dist-test-slave-s2fj krb5kdc[9697](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.1.135.193@KRBTEST.COM for oryx/127.1.135.254@KRBTEST.COM, Server not found in Kerberos database
I20260923 22:13:01.036494 10707 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 22:13:01.032019 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.135.254:38637 (local address 127.1.135.193:42107)
0923 22:13:01.032179 (+ 160us) negotiation.cc:107] Waiting for socket to connect
0923 22:13:01.032188 (+ 9us) client_negotiation.cc:174] Beginning negotiation
0923 22:13:01.032248 (+ 60us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0923 22:13:01.032394 (+ 146us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0923 22:13:01.032394 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0923 22:13:01.032536 (+ 142us) client_negotiation.cc:189] Negotiated authn=SASL
0923 22:13:01.032582 (+ 46us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 22:13:01.032584 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 22:13:01.033165 (+ 581us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 22:13:01.033165 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 22:13:01.033670 (+ 505us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 22:13:01.033672 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 22:13:01.034060 (+ 388us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 22:13:01.034061 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 22:13:01.034104 (+ 43us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 22:13:01.034626 (+ 522us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0923 22:13:01.034631 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0923 22:13:01.036383 (+ 1752us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.1.135.254:38637: Server oryx/127.1.135.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":142,"thread_start_us":106,"threads_started":1}
W20260923 22:13:01.036520 10708 negotiation.cc:337] Failed RPC negotiation. Trace:
0923 22:13:01.032098 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.135.193:42107 (local address 127.1.135.254:38637)
0923 22:13:01.032202 (+ 104us) server_negotiation.cc:206] Beginning negotiation
0923 22:13:01.032203 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0923 22:13:01.032244 (+ 41us) server_negotiation.cc:407] Connection header received
0923 22:13:01.032302 (+ 58us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 22:13:01.032303 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 22:13:01.032313 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 22:13:01.032378 (+ 65us) server_negotiation.cc:226] Negotiated authn=SASL
0923 22:13:01.032640 (+ 262us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 22:13:01.033103 (+ 463us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 22:13:01.033734 (+ 631us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 22:13:01.033953 (+ 219us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 22:13:01.034009 (+ 56us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 22:13:01.036469 (+ 2460us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.1.135.193:42107: BlockingRecv error: recv got EOF from 127.1.135.193:42107 (error 108)
Metrics: {"server-negotiator.queue_time_us":79,"thread_start_us":46,"threads_started":1}
Sep 23 22:13:01 dist-test-slave-s2fj krb5kdc[9697](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.1.135.194@KRBTEST.COM for oryx/127.1.135.254@KRBTEST.COM, Server not found in Kerberos database
Sep 23 22:13:01 dist-test-slave-s2fj krb5kdc[9697](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.1.135.194@KRBTEST.COM for oryx/127.1.135.254@KRBTEST.COM, Server not found in Kerberos database
I20260923 22:13:01.151643 10709 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 22:13:01.147287 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.135.254:38637 (local address 127.1.135.194:45039)
0923 22:13:01.147420 (+ 133us) negotiation.cc:107] Waiting for socket to connect
0923 22:13:01.147431 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0923 22:13:01.147497 (+ 66us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0923 22:13:01.147638 (+ 141us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0923 22:13:01.147639 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0923 22:13:01.147795 (+ 156us) client_negotiation.cc:189] Negotiated authn=SASL
0923 22:13:01.147841 (+ 46us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 22:13:01.147843 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 22:13:01.148478 (+ 635us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 22:13:01.148479 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 22:13:01.148981 (+ 502us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 22:13:01.148982 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 22:13:01.149282 (+ 300us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 22:13:01.149282 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 22:13:01.149319 (+ 37us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 22:13:01.149813 (+ 494us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0923 22:13:01.149818 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0923 22:13:01.151546 (+ 1728us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.1.135.254:38637: Server oryx/127.1.135.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":114,"thread_start_us":83,"threads_started":1}
W20260923 22:13:01.151680 10708 negotiation.cc:337] Failed RPC negotiation. Trace:
0923 22:13:01.147354 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.135.194:45039 (local address 127.1.135.254:38637)
0923 22:13:01.147452 (+ 98us) server_negotiation.cc:206] Beginning negotiation
0923 22:13:01.147454 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0923 22:13:01.147492 (+ 38us) server_negotiation.cc:407] Connection header received
0923 22:13:01.147553 (+ 61us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 22:13:01.147554 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 22:13:01.147564 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 22:13:01.147619 (+ 55us) server_negotiation.cc:226] Negotiated authn=SASL
0923 22:13:01.147905 (+ 286us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 22:13:01.148415 (+ 510us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 22:13:01.149050 (+ 635us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 22:13:01.149222 (+ 172us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 22:13:01.149269 (+ 47us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 22:13:01.151620 (+ 2351us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.1.135.194:45039: BlockingRecv error: recv got EOF from 127.1.135.194:45039 (error 108)
Metrics: {"server-negotiator.queue_time_us":62}
Sep 23 22:13:01 dist-test-slave-s2fj krb5kdc[9697](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.1.135.195@KRBTEST.COM for oryx/127.1.135.254@KRBTEST.COM, Server not found in Kerberos database
Sep 23 22:13:01 dist-test-slave-s2fj krb5kdc[9697](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.1.135.195@KRBTEST.COM for oryx/127.1.135.254@KRBTEST.COM, Server not found in Kerberos database
I20260923 22:13:01.274269 10710 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 22:13:01.269288 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.135.254:38637 (local address 127.1.135.195:59735)
0923 22:13:01.269471 (+ 183us) negotiation.cc:107] Waiting for socket to connect
0923 22:13:01.269482 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0923 22:13:01.269571 (+ 89us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0923 22:13:01.269693 (+ 122us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0923 22:13:01.269693 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0923 22:13:01.269883 (+ 190us) client_negotiation.cc:189] Negotiated authn=SASL
0923 22:13:01.269927 (+ 44us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 22:13:01.269930 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 22:13:01.270630 (+ 700us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 22:13:01.270631 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 22:13:01.271158 (+ 527us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 22:13:01.271159 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 22:13:01.271622 (+ 463us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 22:13:01.271624 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 22:13:01.271724 (+ 100us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 22:13:01.272262 (+ 538us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0923 22:13:01.272268 (+ 6us) client_negotiation.cc:656] calling sasl_client_start()
0923 22:13:01.274175 (+ 1907us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.1.135.254:38637: Server oryx/127.1.135.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":165,"thread_start_us":96,"threads_started":1}
W20260923 22:13:01.274307 10708 negotiation.cc:337] Failed RPC negotiation. Trace:
0923 22:13:01.269382 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.135.195:59735 (local address 127.1.135.254:38637)
0923 22:13:01.269456 (+ 74us) server_negotiation.cc:206] Beginning negotiation
0923 22:13:01.269458 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0923 22:13:01.269553 (+ 95us) server_negotiation.cc:407] Connection header received
0923 22:13:01.269624 (+ 71us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 22:13:01.269625 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 22:13:01.269639 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 22:13:01.269689 (+ 50us) server_negotiation.cc:226] Negotiated authn=SASL
0923 22:13:01.270007 (+ 318us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 22:13:01.270512 (+ 505us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 22:13:01.271242 (+ 730us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 22:13:01.271536 (+ 294us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 22:13:01.271624 (+ 88us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 22:13:01.274247 (+ 2623us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.1.135.195:59735: BlockingRecv error: recv got EOF from 127.1.135.195:59735 (error 108)
Metrics: {"server-negotiator.queue_time_us":43}
Sep 23 22:13:02 dist-test-slave-s2fj krb5kdc[9697](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.1.135.193@KRBTEST.COM for oryx/127.1.135.254@KRBTEST.COM, Server not found in Kerberos database
Sep 23 22:13:02 dist-test-slave-s2fj krb5kdc[9697](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.1.135.193@KRBTEST.COM for oryx/127.1.135.254@KRBTEST.COM, Server not found in Kerberos database
I20260923 22:13:02.041857 10711 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 22:13:02.036916 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.135.254:38637 (local address 127.1.135.193:37523)
0923 22:13:02.037135 (+ 219us) negotiation.cc:107] Waiting for socket to connect
0923 22:13:02.037153 (+ 18us) client_negotiation.cc:174] Beginning negotiation
0923 22:13:02.037208 (+ 55us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0923 22:13:02.037341 (+ 133us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0923 22:13:02.037342 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0923 22:13:02.037556 (+ 214us) client_negotiation.cc:189] Negotiated authn=SASL
0923 22:13:02.037604 (+ 48us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 22:13:02.037606 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 22:13:02.038396 (+ 790us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 22:13:02.038397 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 22:13:02.039005 (+ 608us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 22:13:02.039007 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 22:13:02.039412 (+ 405us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 22:13:02.039413 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 22:13:02.039470 (+ 57us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 22:13:02.039968 (+ 498us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0923 22:13:02.039973 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0923 22:13:02.041786 (+ 1813us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.1.135.254:38637: Server oryx/127.1.135.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":200,"thread_start_us":172,"threads_started":1}
W20260923 22:13:02.041962 10712 negotiation.cc:337] Failed RPC negotiation. Trace:
0923 22:13:02.036974 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.135.193:37523 (local address 127.1.135.254:38637)
0923 22:13:02.037095 (+ 121us) server_negotiation.cc:206] Beginning negotiation
0923 22:13:02.037096 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0923 22:13:02.037208 (+ 112us) server_negotiation.cc:407] Connection header received
0923 22:13:02.037248 (+ 40us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 22:13:02.037249 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 22:13:02.037259 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 22:13:02.037326 (+ 67us) server_negotiation.cc:226] Negotiated authn=SASL
0923 22:13:02.037668 (+ 342us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 22:13:02.038316 (+ 648us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 22:13:02.039108 (+ 792us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 22:13:02.039325 (+ 217us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 22:13:02.039410 (+ 85us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 22:13:02.041872 (+ 2462us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.1.135.193:37523: BlockingRecv error: recv got EOF from 127.1.135.193:37523 (error 108)
Metrics: {"server-negotiator.queue_time_us":93,"thread_start_us":67,"threads_started":1}
Sep 23 22:13:02 dist-test-slave-s2fj krb5kdc[9697](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.1.135.194@KRBTEST.COM for oryx/127.1.135.254@KRBTEST.COM, Server not found in Kerberos database
Sep 23 22:13:02 dist-test-slave-s2fj krb5kdc[9697](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.1.135.194@KRBTEST.COM for oryx/127.1.135.254@KRBTEST.COM, Server not found in Kerberos database
I20260923 22:13:02.162109 10713 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 22:13:02.152309 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.135.254:38637 (local address 127.1.135.194:43887)
0923 22:13:02.152479 (+ 170us) negotiation.cc:107] Waiting for socket to connect
0923 22:13:02.152489 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0923 22:13:02.152552 (+ 63us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0923 22:13:02.152686 (+ 134us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0923 22:13:02.152687 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0923 22:13:02.152857 (+ 170us) client_negotiation.cc:189] Negotiated authn=SASL
0923 22:13:02.152903 (+ 46us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 22:13:02.152905 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 22:13:02.153514 (+ 609us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 22:13:02.153515 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 22:13:02.154026 (+ 511us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 22:13:02.154027 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 22:13:02.154327 (+ 300us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 22:13:02.154328 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 22:13:02.154364 (+ 36us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 22:13:02.160401 (+ 6037us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0923 22:13:02.160406 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0923 22:13:02.162052 (+ 1646us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.1.135.254:38637: Server oryx/127.1.135.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":153,"thread_start_us":83,"threads_started":1}
W20260923 22:13:02.162237 10712 negotiation.cc:337] Failed RPC negotiation. Trace:
0923 22:13:02.152422 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.135.194:43887 (local address 127.1.135.254:38637)
0923 22:13:02.152483 (+ 61us) server_negotiation.cc:206] Beginning negotiation
0923 22:13:02.152484 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0923 22:13:02.152547 (+ 63us) server_negotiation.cc:407] Connection header received
0923 22:13:02.152609 (+ 62us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 22:13:02.152610 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 22:13:02.152620 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 22:13:02.152664 (+ 44us) server_negotiation.cc:226] Negotiated authn=SASL
0923 22:13:02.152964 (+ 300us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 22:13:02.153450 (+ 486us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 22:13:02.154096 (+ 646us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 22:13:02.154262 (+ 166us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 22:13:02.154313 (+ 51us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 22:13:02.162143 (+ 7830us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.1.135.194:43887: BlockingRecv error: recv got EOF from 127.1.135.194:43887 (error 108)
Metrics: {"server-negotiator.queue_time_us":36}
Sep 23 22:13:02 dist-test-slave-s2fj krb5kdc[9697](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.1.135.195@KRBTEST.COM for oryx/127.1.135.254@KRBTEST.COM, Server not found in Kerberos database
Sep 23 22:13:02 dist-test-slave-s2fj krb5kdc[9697](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.1.135.195@KRBTEST.COM for oryx/127.1.135.254@KRBTEST.COM, Server not found in Kerberos database
I20260923 22:13:02.279434 10714 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 22:13:02.274716 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.135.254:38637 (local address 127.1.135.195:52003)
0923 22:13:02.274981 (+ 265us) negotiation.cc:107] Waiting for socket to connect
0923 22:13:02.274993 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0923 22:13:02.275054 (+ 61us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0923 22:13:02.275178 (+ 124us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0923 22:13:02.275178 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0923 22:13:02.275351 (+ 173us) client_negotiation.cc:189] Negotiated authn=SASL
0923 22:13:02.275394 (+ 43us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 22:13:02.275396 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 22:13:02.276047 (+ 651us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 22:13:02.276048 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 22:13:02.276691 (+ 643us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 22:13:02.276693 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 22:13:02.277067 (+ 374us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 22:13:02.277068 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 22:13:02.277127 (+ 59us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 22:13:02.277619 (+ 492us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0923 22:13:02.277624 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0923 22:13:02.279341 (+ 1717us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.1.135.254:38637: Server oryx/127.1.135.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":248,"thread_start_us":91,"threads_started":1}
W20260923 22:13:02.279488 10712 negotiation.cc:337] Failed RPC negotiation. Trace:
0923 22:13:02.274791 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.135.195:52003 (local address 127.1.135.254:38637)
0923 22:13:02.274967 (+ 176us) server_negotiation.cc:206] Beginning negotiation
0923 22:13:02.274969 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0923 22:13:02.275037 (+ 68us) server_negotiation.cc:407] Connection header received
0923 22:13:02.275105 (+ 68us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 22:13:02.275106 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 22:13:02.275120 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 22:13:02.275164 (+ 44us) server_negotiation.cc:226] Negotiated authn=SASL
0923 22:13:02.275473 (+ 309us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 22:13:02.275959 (+ 486us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 22:13:02.276776 (+ 817us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 22:13:02.276979 (+ 203us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 22:13:02.277065 (+ 86us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 22:13:02.279414 (+ 2349us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.1.135.195:52003: BlockingRecv error: recv got EOF from 127.1.135.195:52003 (error 108)
Metrics: {"server-negotiator.queue_time_us":148}
Sep 23 22:13:03 dist-test-slave-s2fj krb5kdc[9697](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.1.135.193@KRBTEST.COM for oryx/127.1.135.254@KRBTEST.COM, Server not found in Kerberos database
Sep 23 22:13:03 dist-test-slave-s2fj krb5kdc[9697](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.1.135.193@KRBTEST.COM for oryx/127.1.135.254@KRBTEST.COM, Server not found in Kerberos database
I20260923 22:13:03.046620 10715 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 22:13:03.042287 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.135.254:38637 (local address 127.1.135.193:44487)
0923 22:13:03.042442 (+ 155us) negotiation.cc:107] Waiting for socket to connect
0923 22:13:03.042453 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0923 22:13:03.042516 (+ 63us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0923 22:13:03.042670 (+ 154us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0923 22:13:03.042671 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0923 22:13:03.042837 (+ 166us) client_negotiation.cc:189] Negotiated authn=SASL
0923 22:13:03.042886 (+ 49us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 22:13:03.042887 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 22:13:03.043495 (+ 608us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 22:13:03.043496 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 22:13:03.044021 (+ 525us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 22:13:03.044023 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 22:13:03.044456 (+ 433us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 22:13:03.044457 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 22:13:03.044500 (+ 43us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 22:13:03.044933 (+ 433us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0923 22:13:03.044938 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0923 22:13:03.046566 (+ 1628us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.1.135.254:38637: Server oryx/127.1.135.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":135,"thread_start_us":86,"threads_started":1}
W20260923 22:13:03.046720 10716 negotiation.cc:337] Failed RPC negotiation. Trace:
0923 22:13:03.042355 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.135.193:44487 (local address 127.1.135.254:38637)
0923 22:13:03.042464 (+ 109us) server_negotiation.cc:206] Beginning negotiation
0923 22:13:03.042465 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0923 22:13:03.042501 (+ 36us) server_negotiation.cc:407] Connection header received
0923 22:13:03.042581 (+ 80us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 22:13:03.042581 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 22:13:03.042592 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 22:13:03.042649 (+ 57us) server_negotiation.cc:226] Negotiated authn=SASL
0923 22:13:03.042951 (+ 302us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 22:13:03.043435 (+ 484us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 22:13:03.044206 (+ 771us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 22:13:03.044391 (+ 185us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 22:13:03.044441 (+ 50us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 22:13:03.046647 (+ 2206us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.1.135.193:44487: BlockingRecv error: recv got EOF from 127.1.135.193:44487 (error 108)
Metrics: {"server-negotiator.queue_time_us":83,"thread_start_us":46,"threads_started":1}
Sep 23 22:13:03 dist-test-slave-s2fj krb5kdc[9697](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.1.135.194@KRBTEST.COM for oryx/127.1.135.254@KRBTEST.COM, Server not found in Kerberos database
Sep 23 22:13:03 dist-test-slave-s2fj krb5kdc[9697](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.1.135.194@KRBTEST.COM for oryx/127.1.135.254@KRBTEST.COM, Server not found in Kerberos database
I20260923 22:13:03.167372 10717 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 22:13:03.162668 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.135.254:38637 (local address 127.1.135.194:53903)
0923 22:13:03.162818 (+ 150us) negotiation.cc:107] Waiting for socket to connect
0923 22:13:03.162834 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0923 22:13:03.162892 (+ 58us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0923 22:13:03.163050 (+ 158us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0923 22:13:03.163051 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0923 22:13:03.163212 (+ 161us) client_negotiation.cc:189] Negotiated authn=SASL
0923 22:13:03.163254 (+ 42us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 22:13:03.163255 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 22:13:03.163890 (+ 635us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 22:13:03.163891 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 22:13:03.164511 (+ 620us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 22:13:03.164514 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 22:13:03.164868 (+ 354us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 22:13:03.164869 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 22:13:03.164924 (+ 55us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 22:13:03.165366 (+ 442us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0923 22:13:03.165370 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0923 22:13:03.167278 (+ 1908us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.1.135.254:38637: Server oryx/127.1.135.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":114,"thread_start_us":85,"threads_started":1}
W20260923 22:13:03.167428 10716 negotiation.cc:337] Failed RPC negotiation. Trace:
0923 22:13:03.162800 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.135.194:53903 (local address 127.1.135.254:38637)
0923 22:13:03.162876 (+ 76us) server_negotiation.cc:206] Beginning negotiation
0923 22:13:03.162878 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0923 22:13:03.162887 (+ 9us) server_negotiation.cc:407] Connection header received
0923 22:13:03.162956 (+ 69us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 22:13:03.162957 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 22:13:03.162975 (+ 18us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 22:13:03.163046 (+ 71us) server_negotiation.cc:226] Negotiated authn=SASL
0923 22:13:03.163340 (+ 294us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 22:13:03.163811 (+ 471us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 22:13:03.164604 (+ 793us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 22:13:03.164795 (+ 191us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 22:13:03.164871 (+ 76us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 22:13:03.167364 (+ 2493us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.1.135.194:53903: BlockingRecv error: recv got EOF from 127.1.135.194:53903 (error 108)
Metrics: {"server-negotiator.queue_time_us":50}
Sep 23 22:13:03 dist-test-slave-s2fj krb5kdc[9697](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.1.135.195@KRBTEST.COM for oryx/127.1.135.254@KRBTEST.COM, Server not found in Kerberos database
Sep 23 22:13:03 dist-test-slave-s2fj krb5kdc[9697](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.1.135.195@KRBTEST.COM for oryx/127.1.135.254@KRBTEST.COM, Server not found in Kerberos database
I20260923 22:13:03.284468 10718 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 22:13:03.279994 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.135.254:38637 (local address 127.1.135.195:43071)
0923 22:13:03.280152 (+ 158us) negotiation.cc:107] Waiting for socket to connect
0923 22:13:03.280162 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0923 22:13:03.280215 (+ 53us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0923 22:13:03.280347 (+ 132us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0923 22:13:03.280348 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0923 22:13:03.280499 (+ 151us) client_negotiation.cc:189] Negotiated authn=SASL
0923 22:13:03.280546 (+ 47us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 22:13:03.280548 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 22:13:03.281181 (+ 633us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 22:13:03.281181 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 22:13:03.281686 (+ 505us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 22:13:03.281687 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 22:13:03.281981 (+ 294us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 22:13:03.281982 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 22:13:03.282019 (+ 37us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 22:13:03.282527 (+ 508us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0923 22:13:03.282532 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0923 22:13:03.284374 (+ 1842us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.1.135.254:38637: Server oryx/127.1.135.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":136,"thread_start_us":100,"threads_started":1}
W20260923 22:13:03.284502 10716 negotiation.cc:337] Failed RPC negotiation. Trace:
0923 22:13:03.280093 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.135.195:43071 (local address 127.1.135.254:38637)
0923 22:13:03.280142 (+ 49us) server_negotiation.cc:206] Beginning negotiation
0923 22:13:03.280144 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0923 22:13:03.280203 (+ 59us) server_negotiation.cc:407] Connection header received
0923 22:13:03.280270 (+ 67us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 22:13:03.280271 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 22:13:03.280282 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 22:13:03.280331 (+ 49us) server_negotiation.cc:226] Negotiated authn=SASL
0923 22:13:03.280606 (+ 275us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 22:13:03.281121 (+ 515us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 22:13:03.281751 (+ 630us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 22:13:03.281922 (+ 171us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 22:13:03.281968 (+ 46us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 22:13:03.284444 (+ 2476us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.1.135.195:43071: BlockingRecv error: recv got EOF from 127.1.135.195:43071 (error 108)
Metrics: {"server-negotiator.queue_time_us":22}
Sep 23 22:13:04 dist-test-slave-s2fj krb5kdc[9697](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.1.135.193@KRBTEST.COM for oryx/127.1.135.254@KRBTEST.COM, Server not found in Kerberos database
Sep 23 22:13:04 dist-test-slave-s2fj krb5kdc[9697](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.1.135.193@KRBTEST.COM for oryx/127.1.135.254@KRBTEST.COM, Server not found in Kerberos database
I20260923 22:13:04.051455 10719 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 22:13:04.047121 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.135.254:38637 (local address 127.1.135.193:37709)
0923 22:13:04.047272 (+ 151us) negotiation.cc:107] Waiting for socket to connect
0923 22:13:04.047282 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0923 22:13:04.047362 (+ 80us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0923 22:13:04.047474 (+ 112us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0923 22:13:04.047474 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0923 22:13:04.047617 (+ 143us) client_negotiation.cc:189] Negotiated authn=SASL
0923 22:13:04.047664 (+ 47us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 22:13:04.047666 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 22:13:04.048287 (+ 621us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 22:13:04.048288 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 22:13:04.048808 (+ 520us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 22:13:04.048809 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 22:13:04.049118 (+ 309us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 22:13:04.049119 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 22:13:04.049156 (+ 37us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 22:13:04.049560 (+ 404us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0923 22:13:04.049565 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0923 22:13:04.051392 (+ 1827us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.1.135.254:38637: Server oryx/127.1.135.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":133,"thread_start_us":80,"threads_started":1}
W20260923 22:13:04.051550 10720 negotiation.cc:337] Failed RPC negotiation. Trace:
0923 22:13:04.047196 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.135.193:37709 (local address 127.1.135.254:38637)
0923 22:13:04.047292 (+ 96us) server_negotiation.cc:206] Beginning negotiation
0923 22:13:04.047294 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0923 22:13:04.047359 (+ 65us) server_negotiation.cc:407] Connection header received
0923 22:13:04.047400 (+ 41us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 22:13:04.047400 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 22:13:04.047410 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 22:13:04.047460 (+ 50us) server_negotiation.cc:226] Negotiated authn=SASL
0923 22:13:04.047720 (+ 260us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 22:13:04.048226 (+ 506us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 22:13:04.048874 (+ 648us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 22:13:04.049058 (+ 184us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 22:13:04.049106 (+ 48us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 22:13:04.051479 (+ 2373us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.1.135.193:37709: BlockingRecv error: recv got EOF from 127.1.135.193:37709 (error 108)
Metrics: {"server-negotiator.queue_time_us":71,"thread_start_us":45,"threads_started":1}
Sep 23 22:13:04 dist-test-slave-s2fj krb5kdc[9697](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.1.135.194@KRBTEST.COM for oryx/127.1.135.254@KRBTEST.COM, Server not found in Kerberos database
Sep 23 22:13:04 dist-test-slave-s2fj krb5kdc[9697](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.1.135.194@KRBTEST.COM for oryx/127.1.135.254@KRBTEST.COM, Server not found in Kerberos database
I20260923 22:13:04.172770 10721 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 22:13:04.167973 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.135.254:38637 (local address 127.1.135.194:33531)
0923 22:13:04.168117 (+ 144us) negotiation.cc:107] Waiting for socket to connect
0923 22:13:04.168131 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0923 22:13:04.168182 (+ 51us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0923 22:13:04.168310 (+ 128us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0923 22:13:04.168311 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0923 22:13:04.168462 (+ 151us) client_negotiation.cc:189] Negotiated authn=SASL
0923 22:13:04.168531 (+ 69us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 22:13:04.168532 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 22:13:04.169145 (+ 613us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 22:13:04.169145 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 22:13:04.169892 (+ 747us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 22:13:04.169895 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 22:13:04.170224 (+ 329us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 22:13:04.170224 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 22:13:04.170260 (+ 36us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 22:13:04.170740 (+ 480us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0923 22:13:04.170744 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0923 22:13:04.172704 (+ 1960us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.1.135.254:38637: Server oryx/127.1.135.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":126,"thread_start_us":77,"threads_started":1}
W20260923 22:13:04.172865 10720 negotiation.cc:337] Failed RPC negotiation. Trace:
0923 22:13:04.168051 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.135.194:33531 (local address 127.1.135.254:38637)
0923 22:13:04.168121 (+ 70us) server_negotiation.cc:206] Beginning negotiation
0923 22:13:04.168122 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0923 22:13:04.168169 (+ 47us) server_negotiation.cc:407] Connection header received
0923 22:13:04.168224 (+ 55us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 22:13:04.168225 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 22:13:04.168234 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 22:13:04.168286 (+ 52us) server_negotiation.cc:226] Negotiated authn=SASL
0923 22:13:04.168593 (+ 307us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 22:13:04.169081 (+ 488us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 22:13:04.169968 (+ 887us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 22:13:04.170162 (+ 194us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 22:13:04.170214 (+ 52us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 22:13:04.172800 (+ 2586us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.1.135.194:33531: BlockingRecv error: recv got EOF from 127.1.135.194:33531 (error 108)
Metrics: {"server-negotiator.queue_time_us":42}
Sep 23 22:13:04 dist-test-slave-s2fj krb5kdc[9697](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.1.135.195@KRBTEST.COM for oryx/127.1.135.254@KRBTEST.COM, Server not found in Kerberos database
Sep 23 22:13:04 dist-test-slave-s2fj krb5kdc[9697](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.1.135.195@KRBTEST.COM for oryx/127.1.135.254@KRBTEST.COM, Server not found in Kerberos database
I20260923 22:13:04.289402 10722 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 22:13:04.284966 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.135.254:38637 (local address 127.1.135.195:47101)
0923 22:13:04.285109 (+ 143us) negotiation.cc:107] Waiting for socket to connect
0923 22:13:04.285119 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0923 22:13:04.285174 (+ 55us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0923 22:13:04.285293 (+ 119us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0923 22:13:04.285293 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0923 22:13:04.285456 (+ 163us) client_negotiation.cc:189] Negotiated authn=SASL
0923 22:13:04.285501 (+ 45us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 22:13:04.285502 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 22:13:04.286089 (+ 587us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 22:13:04.286089 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 22:13:04.286647 (+ 558us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 22:13:04.286648 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 22:13:04.286940 (+ 292us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 22:13:04.286941 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 22:13:04.286976 (+ 35us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 22:13:04.287382 (+ 406us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0923 22:13:04.287386 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0923 22:13:04.289345 (+ 1959us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.1.135.254:38637: Server oryx/127.1.135.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":119,"thread_start_us":67,"threads_started":1}
W20260923 22:13:04.289491 10720 negotiation.cc:337] Failed RPC negotiation. Trace:
0923 22:13:04.285027 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.135.195:47101 (local address 127.1.135.254:38637)
0923 22:13:04.285111 (+ 84us) server_negotiation.cc:206] Beginning negotiation
0923 22:13:04.285113 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0923 22:13:04.285177 (+ 64us) server_negotiation.cc:407] Connection header received
0923 22:13:04.285222 (+ 45us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 22:13:04.285223 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 22:13:04.285233 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 22:13:04.285273 (+ 40us) server_negotiation.cc:226] Negotiated authn=SASL
0923 22:13:04.285558 (+ 285us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 22:13:04.286030 (+ 472us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 22:13:04.286711 (+ 681us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 22:13:04.286885 (+ 174us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 22:13:04.286930 (+ 45us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 22:13:04.289429 (+ 2499us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.1.135.195:47101: BlockingRecv error: recv got EOF from 127.1.135.195:47101 (error 108)
Metrics: {"server-negotiator.queue_time_us":59}
Sep 23 22:13:05 dist-test-slave-s2fj krb5kdc[9697](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.1.135.193@KRBTEST.COM for oryx/127.1.135.254@KRBTEST.COM, Server not found in Kerberos database
Sep 23 22:13:05 dist-test-slave-s2fj krb5kdc[9697](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.1.135.193@KRBTEST.COM for oryx/127.1.135.254@KRBTEST.COM, Server not found in Kerberos database
I20260923 22:13:05.055923 10723 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 22:13:05.051903 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.135.254:38637 (local address 127.1.135.193:51963)
0923 22:13:05.052031 (+ 128us) negotiation.cc:107] Waiting for socket to connect
0923 22:13:05.052041 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0923 22:13:05.052118 (+ 77us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0923 22:13:05.052244 (+ 126us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0923 22:13:05.052244 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0923 22:13:05.052394 (+ 150us) client_negotiation.cc:189] Negotiated authn=SASL
0923 22:13:05.052441 (+ 47us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 22:13:05.052442 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 22:13:05.053038 (+ 596us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 22:13:05.053038 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 22:13:05.053523 (+ 485us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 22:13:05.053524 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 22:13:05.053825 (+ 301us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 22:13:05.053826 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 22:13:05.053863 (+ 37us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 22:13:05.054290 (+ 427us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0923 22:13:05.054294 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0923 22:13:05.055859 (+ 1565us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.1.135.254:38637: Server oryx/127.1.135.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":109,"thread_start_us":71,"threads_started":1}
W20260923 22:13:05.056049 10724 negotiation.cc:337] Failed RPC negotiation. Trace:
0923 22:13:05.051913 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.135.193:51963 (local address 127.1.135.254:38637)
0923 22:13:05.052036 (+ 123us) server_negotiation.cc:206] Beginning negotiation
0923 22:13:05.052038 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0923 22:13:05.052109 (+ 71us) server_negotiation.cc:407] Connection header received
0923 22:13:05.052160 (+ 51us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 22:13:05.052161 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 22:13:05.052170 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 22:13:05.052233 (+ 63us) server_negotiation.cc:226] Negotiated authn=SASL
0923 22:13:05.052505 (+ 272us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 22:13:05.052977 (+ 472us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 22:13:05.053589 (+ 612us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 22:13:05.053766 (+ 177us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 22:13:05.053811 (+ 45us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 22:13:05.055949 (+ 2138us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.1.135.193:51963: BlockingRecv error: recv got EOF from 127.1.135.193:51963 (error 108)
Metrics: {"server-negotiator.queue_time_us":102,"thread_start_us":73,"threads_started":1}
Sep 23 22:13:05 dist-test-slave-s2fj krb5kdc[9697](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.1.135.194@KRBTEST.COM for oryx/127.1.135.254@KRBTEST.COM, Server not found in Kerberos database
Sep 23 22:13:05 dist-test-slave-s2fj krb5kdc[9697](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.1.135.194@KRBTEST.COM for oryx/127.1.135.254@KRBTEST.COM, Server not found in Kerberos database
I20260923 22:13:05.178190 10725 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 22:13:05.173183 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.135.254:38637 (local address 127.1.135.194:56367)
0923 22:13:05.173333 (+ 150us) negotiation.cc:107] Waiting for socket to connect
0923 22:13:05.173347 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0923 22:13:05.173428 (+ 81us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0923 22:13:05.173591 (+ 163us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0923 22:13:05.173592 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0923 22:13:05.173785 (+ 193us) client_negotiation.cc:189] Negotiated authn=SASL
0923 22:13:05.173837 (+ 52us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 22:13:05.173839 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 22:13:05.174510 (+ 671us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 22:13:05.174511 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 22:13:05.175027 (+ 516us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 22:13:05.175029 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 22:13:05.175377 (+ 348us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 22:13:05.175379 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 22:13:05.175432 (+ 53us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 22:13:05.175848 (+ 416us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0923 22:13:05.175852 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0923 22:13:05.178094 (+ 2242us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.1.135.254:38637: Server oryx/127.1.135.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":134,"thread_start_us":76,"threads_started":1}
W20260923 22:13:05.178313 10724 negotiation.cc:337] Failed RPC negotiation. Trace:
0923 22:13:05.173247 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.135.194:56367 (local address 127.1.135.254:38637)
0923 22:13:05.173337 (+ 90us) server_negotiation.cc:206] Beginning negotiation
0923 22:13:05.173339 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0923 22:13:05.173408 (+ 69us) server_negotiation.cc:407] Connection header received
0923 22:13:05.173481 (+ 73us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 22:13:05.173482 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 22:13:05.173495 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 22:13:05.173569 (+ 74us) server_negotiation.cc:226] Negotiated authn=SASL
0923 22:13:05.173912 (+ 343us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 22:13:05.174431 (+ 519us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 22:13:05.175120 (+ 689us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 22:13:05.175311 (+ 191us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 22:13:05.175380 (+ 69us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 22:13:05.178234 (+ 2854us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.1.135.194:56367: BlockingRecv error: recv got EOF from 127.1.135.194:56367 (error 108)
Metrics: {"server-negotiator.queue_time_us":64}
Sep 23 22:13:05 dist-test-slave-s2fj krb5kdc[9697](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.1.135.195@KRBTEST.COM for oryx/127.1.135.254@KRBTEST.COM, Server not found in Kerberos database
Sep 23 22:13:05 dist-test-slave-s2fj krb5kdc[9697](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.1.135.195@KRBTEST.COM for oryx/127.1.135.254@KRBTEST.COM, Server not found in Kerberos database
I20260923 22:13:05.294313 10726 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 22:13:05.289838 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.135.254:38637 (local address 127.1.135.195:38391)
0923 22:13:05.290014 (+ 176us) negotiation.cc:107] Waiting for socket to connect
0923 22:13:05.290026 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0923 22:13:05.290112 (+ 86us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0923 22:13:05.290262 (+ 150us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0923 22:13:05.290263 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0923 22:13:05.290438 (+ 175us) client_negotiation.cc:189] Negotiated authn=SASL
0923 22:13:05.290484 (+ 46us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 22:13:05.290486 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 22:13:05.291146 (+ 660us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 22:13:05.291148 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 22:13:05.291675 (+ 527us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 22:13:05.291677 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 22:13:05.292087 (+ 410us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 22:13:05.292089 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 22:13:05.292155 (+ 66us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 22:13:05.292562 (+ 407us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0923 22:13:05.292566 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0923 22:13:05.294258 (+ 1692us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.1.135.254:38637: Server oryx/127.1.135.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":158,"thread_start_us":99,"threads_started":1}
W20260923 22:13:05.294412 10724 negotiation.cc:337] Failed RPC negotiation. Trace:
0923 22:13:05.289911 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.135.195:38391 (local address 127.1.135.254:38637)
0923 22:13:05.290000 (+ 89us) server_negotiation.cc:206] Beginning negotiation
0923 22:13:05.290003 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0923 22:13:05.290097 (+ 94us) server_negotiation.cc:407] Connection header received
0923 22:13:05.290167 (+ 70us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 22:13:05.290168 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 22:13:05.290183 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 22:13:05.290253 (+ 70us) server_negotiation.cc:226] Negotiated authn=SASL
0923 22:13:05.290566 (+ 313us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 22:13:05.291060 (+ 494us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 22:13:05.291781 (+ 721us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 22:13:05.291984 (+ 203us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 22:13:05.292062 (+ 78us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 22:13:05.294346 (+ 2284us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.1.135.195:38391: BlockingRecv error: recv got EOF from 127.1.135.195:38391 (error 108)
Metrics: {"server-negotiator.queue_time_us":59}
Sep 23 22:13:06 dist-test-slave-s2fj krb5kdc[9697](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.1.135.193@KRBTEST.COM for oryx/127.1.135.254@KRBTEST.COM, Server not found in Kerberos database
Sep 23 22:13:06 dist-test-slave-s2fj krb5kdc[9697](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.1.135.193@KRBTEST.COM for oryx/127.1.135.254@KRBTEST.COM, Server not found in Kerberos database
I20260923 22:13:06.060830 10727 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 22:13:06.056454 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.135.254:38637 (local address 127.1.135.193:50435)
0923 22:13:06.056591 (+ 137us) negotiation.cc:107] Waiting for socket to connect
0923 22:13:06.056601 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0923 22:13:06.056647 (+ 46us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0923 22:13:06.056843 (+ 196us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0923 22:13:06.056844 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0923 22:13:06.056993 (+ 149us) client_negotiation.cc:189] Negotiated authn=SASL
0923 22:13:06.057041 (+ 48us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 22:13:06.057042 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 22:13:06.057665 (+ 623us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 22:13:06.057666 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 22:13:06.058178 (+ 512us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 22:13:06.058179 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 22:13:06.058486 (+ 307us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 22:13:06.058486 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 22:13:06.058522 (+ 36us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 22:13:06.058947 (+ 425us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0923 22:13:06.058952 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0923 22:13:06.060738 (+ 1786us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.1.135.254:38637: Server oryx/127.1.135.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":114,"thread_start_us":54,"threads_started":1}
W20260923 22:13:06.060873 10728 negotiation.cc:337] Failed RPC negotiation. Trace:
0923 22:13:06.056598 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.135.193:50435 (local address 127.1.135.254:38637)
0923 22:13:06.056715 (+ 117us) server_negotiation.cc:206] Beginning negotiation
0923 22:13:06.056717 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0923 22:13:06.056723 (+ 6us) server_negotiation.cc:407] Connection header received
0923 22:13:06.056751 (+ 28us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 22:13:06.056751 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 22:13:06.056761 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 22:13:06.056817 (+ 56us) server_negotiation.cc:226] Negotiated authn=SASL
0923 22:13:06.057123 (+ 306us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 22:13:06.057605 (+ 482us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 22:13:06.058246 (+ 641us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 22:13:06.058426 (+ 180us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 22:13:06.058474 (+ 48us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 22:13:06.060808 (+ 2334us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.1.135.193:50435: BlockingRecv error: recv got EOF from 127.1.135.193:50435 (error 108)
Metrics: {"server-negotiator.queue_time_us":96,"thread_start_us":66,"threads_started":1}
Sep 23 22:13:06 dist-test-slave-s2fj krb5kdc[9697](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.1.135.194@KRBTEST.COM for oryx/127.1.135.254@KRBTEST.COM, Server not found in Kerberos database
Sep 23 22:13:06 dist-test-slave-s2fj krb5kdc[9697](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.1.135.194@KRBTEST.COM for oryx/127.1.135.254@KRBTEST.COM, Server not found in Kerberos database
I20260923 22:13:06.183528 10729 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 22:13:06.178855 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.135.254:38637 (local address 127.1.135.194:49767)
0923 22:13:06.178995 (+ 140us) negotiation.cc:107] Waiting for socket to connect
0923 22:13:06.179005 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0923 22:13:06.179089 (+ 84us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0923 22:13:06.179328 (+ 239us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0923 22:13:06.179329 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0923 22:13:06.179494 (+ 165us) client_negotiation.cc:189] Negotiated authn=SASL
0923 22:13:06.179565 (+ 71us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 22:13:06.179567 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 22:13:06.180397 (+ 830us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 22:13:06.180398 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 22:13:06.180996 (+ 598us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 22:13:06.180998 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 22:13:06.181354 (+ 356us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 22:13:06.181355 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 22:13:06.181408 (+ 53us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 22:13:06.181826 (+ 418us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0923 22:13:06.181830 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0923 22:13:06.183470 (+ 1640us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.1.135.254:38637: Server oryx/127.1.135.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":122,"thread_start_us":87,"threads_started":1}
W20260923 22:13:06.183625 10728 negotiation.cc:337] Failed RPC negotiation. Trace:
0923 22:13:06.178915 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.135.194:49767 (local address 127.1.135.254:38637)
0923 22:13:06.178997 (+ 82us) server_negotiation.cc:206] Beginning negotiation
0923 22:13:06.178999 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0923 22:13:06.179089 (+ 90us) server_negotiation.cc:407] Connection header received
0923 22:13:06.179149 (+ 60us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 22:13:06.179150 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 22:13:06.179162 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 22:13:06.179278 (+ 116us) server_negotiation.cc:226] Negotiated authn=SASL
0923 22:13:06.179632 (+ 354us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 22:13:06.180321 (+ 689us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 22:13:06.181087 (+ 766us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 22:13:06.181276 (+ 189us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 22:13:06.181356 (+ 80us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 22:13:06.183561 (+ 2205us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.1.135.194:49767: BlockingRecv error: recv got EOF from 127.1.135.194:49767 (error 108)
Metrics: {"server-negotiator.queue_time_us":46}
Sep 23 22:13:06 dist-test-slave-s2fj krb5kdc[9697](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.1.135.195@KRBTEST.COM for oryx/127.1.135.254@KRBTEST.COM, Server not found in Kerberos database
Sep 23 22:13:06 dist-test-slave-s2fj krb5kdc[9697](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.1.135.195@KRBTEST.COM for oryx/127.1.135.254@KRBTEST.COM, Server not found in Kerberos database
I20260923 22:13:06.299314 10730 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 22:13:06.294868 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.135.254:38637 (local address 127.1.135.195:41595)
0923 22:13:06.295017 (+ 149us) negotiation.cc:107] Waiting for socket to connect
0923 22:13:06.295029 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0923 22:13:06.295095 (+ 66us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0923 22:13:06.295203 (+ 108us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0923 22:13:06.295204 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0923 22:13:06.295353 (+ 149us) client_negotiation.cc:189] Negotiated authn=SASL
0923 22:13:06.295399 (+ 46us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 22:13:06.295400 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 22:13:06.295985 (+ 585us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 22:13:06.295985 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 22:13:06.296543 (+ 558us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 22:13:06.296545 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 22:13:06.296842 (+ 297us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 22:13:06.296842 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 22:13:06.296878 (+ 36us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 22:13:06.297344 (+ 466us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0923 22:13:06.297348 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0923 22:13:06.299248 (+ 1900us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.1.135.254:38637: Server oryx/127.1.135.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":126,"thread_start_us":58,"threads_started":1}
W20260923 22:13:06.299381 10728 negotiation.cc:337] Failed RPC negotiation. Trace:
0923 22:13:06.295037 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.135.195:41595 (local address 127.1.135.254:38637)
0923 22:13:06.295095 (+ 58us) server_negotiation.cc:206] Beginning negotiation
0923 22:13:06.295097 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0923 22:13:06.295106 (+ 9us) server_negotiation.cc:407] Connection header received
0923 22:13:06.295140 (+ 34us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 22:13:06.295140 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 22:13:06.295151 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 22:13:06.295193 (+ 42us) server_negotiation.cc:226] Negotiated authn=SASL
0923 22:13:06.295453 (+ 260us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 22:13:06.295924 (+ 471us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 22:13:06.296606 (+ 682us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 22:13:06.296783 (+ 177us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 22:13:06.296832 (+ 49us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 22:13:06.299335 (+ 2503us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.1.135.195:41595: BlockingRecv error: recv got EOF from 127.1.135.195:41595 (error 108)
Metrics: {"server-negotiator.queue_time_us":36}
Sep 23 22:13:07 dist-test-slave-s2fj krb5kdc[9697](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.1.135.193@KRBTEST.COM for oryx/127.1.135.254@KRBTEST.COM, Server not found in Kerberos database
Sep 23 22:13:07 dist-test-slave-s2fj krb5kdc[9697](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.1.135.193@KRBTEST.COM for oryx/127.1.135.254@KRBTEST.COM, Server not found in Kerberos database
I20260923 22:13:07.066131 10731 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 22:13:07.061253 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.135.254:38637 (local address 127.1.135.193:53651)
0923 22:13:07.061389 (+ 136us) negotiation.cc:107] Waiting for socket to connect
0923 22:13:07.061399 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0923 22:13:07.061446 (+ 47us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0923 22:13:07.061604 (+ 158us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0923 22:13:07.061605 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0923 22:13:07.061906 (+ 301us) client_negotiation.cc:189] Negotiated authn=SASL
0923 22:13:07.061969 (+ 63us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 22:13:07.061971 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 22:13:07.062710 (+ 739us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 22:13:07.062711 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 22:13:07.063211 (+ 500us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 22:13:07.063212 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 22:13:07.063546 (+ 334us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 22:13:07.063547 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 22:13:07.063583 (+ 36us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 22:13:07.064009 (+ 426us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0923 22:13:07.064013 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0923 22:13:07.066030 (+ 2017us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.1.135.254:38637: Server oryx/127.1.135.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":110,"thread_start_us":82,"threads_started":1}
W20260923 22:13:07.066182 10732 negotiation.cc:337] Failed RPC negotiation. Trace:
0923 22:13:07.061301 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.135.193:53651 (local address 127.1.135.254:38637)
0923 22:13:07.061422 (+ 121us) server_negotiation.cc:206] Beginning negotiation
0923 22:13:07.061424 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0923 22:13:07.061434 (+ 10us) server_negotiation.cc:407] Connection header received
0923 22:13:07.061501 (+ 67us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 22:13:07.061502 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 22:13:07.061511 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 22:13:07.061588 (+ 77us) server_negotiation.cc:226] Negotiated authn=SASL
0923 22:13:07.062046 (+ 458us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 22:13:07.062640 (+ 594us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 22:13:07.063280 (+ 640us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 22:13:07.063487 (+ 207us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 22:13:07.063532 (+ 45us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 22:13:07.066113 (+ 2581us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.1.135.193:53651: BlockingRecv error: recv got EOF from 127.1.135.193:53651 (error 108)
Metrics: {"server-negotiator.queue_time_us":98,"thread_start_us":67,"threads_started":1}
Sep 23 22:13:07 dist-test-slave-s2fj krb5kdc[9697](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.1.135.194@KRBTEST.COM for oryx/127.1.135.254@KRBTEST.COM, Server not found in Kerberos database
Sep 23 22:13:07 dist-test-slave-s2fj krb5kdc[9697](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.1.135.194@KRBTEST.COM for oryx/127.1.135.254@KRBTEST.COM, Server not found in Kerberos database
I20260923 22:13:07.191552 10733 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 22:13:07.183979 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.135.254:38637 (local address 127.1.135.194:39873)
0923 22:13:07.184136 (+ 157us) negotiation.cc:107] Waiting for socket to connect
0923 22:13:07.184147 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0923 22:13:07.184205 (+ 58us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0923 22:13:07.184313 (+ 108us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0923 22:13:07.184313 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0923 22:13:07.184467 (+ 154us) client_negotiation.cc:189] Negotiated authn=SASL
0923 22:13:07.184513 (+ 46us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 22:13:07.184514 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 22:13:07.185116 (+ 602us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 22:13:07.185117 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 22:13:07.185666 (+ 549us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 22:13:07.185667 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 22:13:07.185980 (+ 313us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 22:13:07.185981 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 22:13:07.186020 (+ 39us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 22:13:07.189542 (+ 3522us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0923 22:13:07.189546 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0923 22:13:07.191486 (+ 1940us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.1.135.254:38637: Server oryx/127.1.135.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":131,"thread_start_us":74,"threads_started":1}
W20260923 22:13:07.191663 10732 negotiation.cc:337] Failed RPC negotiation. Trace:
0923 22:13:07.184045 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.135.194:39873 (local address 127.1.135.254:38637)
0923 22:13:07.184116 (+ 71us) server_negotiation.cc:206] Beginning negotiation
0923 22:13:07.184118 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0923 22:13:07.184191 (+ 73us) server_negotiation.cc:407] Connection header received
0923 22:13:07.184247 (+ 56us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 22:13:07.184247 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 22:13:07.184259 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 22:13:07.184307 (+ 48us) server_negotiation.cc:226] Negotiated authn=SASL
0923 22:13:07.184568 (+ 261us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 22:13:07.185051 (+ 483us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 22:13:07.185742 (+ 691us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 22:13:07.185918 (+ 176us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 22:13:07.185965 (+ 47us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 22:13:07.191571 (+ 5606us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.1.135.194:39873: BlockingRecv error: recv got EOF from 127.1.135.194:39873 (error 108)
Metrics: {"server-negotiator.queue_time_us":48}
Sep 23 22:13:07 dist-test-slave-s2fj krb5kdc[9697](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.1.135.195@KRBTEST.COM for oryx/127.1.135.254@KRBTEST.COM, Server not found in Kerberos database
Sep 23 22:13:07 dist-test-slave-s2fj krb5kdc[9697](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.1.135.195@KRBTEST.COM for oryx/127.1.135.254@KRBTEST.COM, Server not found in Kerberos database
I20260923 22:13:07.304039 10734 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 22:13:07.299816 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.135.254:38637 (local address 127.1.135.195:59121)
0923 22:13:07.299972 (+ 156us) negotiation.cc:107] Waiting for socket to connect
0923 22:13:07.299982 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0923 22:13:07.300034 (+ 52us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0923 22:13:07.300246 (+ 212us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0923 22:13:07.300247 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0923 22:13:07.300397 (+ 150us) client_negotiation.cc:189] Negotiated authn=SASL
0923 22:13:07.300444 (+ 47us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 22:13:07.300446 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 22:13:07.301045 (+ 599us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 22:13:07.301045 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 22:13:07.301547 (+ 502us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 22:13:07.301549 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 22:13:07.301841 (+ 292us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 22:13:07.301842 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 22:13:07.301877 (+ 35us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 22:13:07.302292 (+ 415us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0923 22:13:07.302295 (+ 3us) client_negotiation.cc:656] calling sasl_client_start()
0923 22:13:07.303971 (+ 1676us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.1.135.254:38637: Server oryx/127.1.135.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":134,"thread_start_us":87,"threads_started":1}
W20260923 22:13:07.304190 10732 negotiation.cc:337] Failed RPC negotiation. Trace:
0923 22:13:07.299884 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.135.195:59121 (local address 127.1.135.254:38637)
0923 22:13:07.299946 (+ 62us) server_negotiation.cc:206] Beginning negotiation
0923 22:13:07.299948 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0923 22:13:07.300035 (+ 87us) server_negotiation.cc:407] Connection header received
0923 22:13:07.300150 (+ 115us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 22:13:07.300151 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 22:13:07.300170 (+ 19us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 22:13:07.300228 (+ 58us) server_negotiation.cc:226] Negotiated authn=SASL
0923 22:13:07.300503 (+ 275us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 22:13:07.300981 (+ 478us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 22:13:07.301612 (+ 631us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 22:13:07.301783 (+ 171us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 22:13:07.301830 (+ 47us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 22:13:07.304132 (+ 2302us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.1.135.195:59121: BlockingRecv error: recv got EOF from 127.1.135.195:59121 (error 108)
Metrics: {"server-negotiator.queue_time_us":38}
Sep 23 22:13:08 dist-test-slave-s2fj krb5kdc[9697](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.1.135.193@KRBTEST.COM for oryx/127.1.135.254@KRBTEST.COM, Server not found in Kerberos database
Sep 23 22:13:08 dist-test-slave-s2fj krb5kdc[9697](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.1.135.193@KRBTEST.COM for oryx/127.1.135.254@KRBTEST.COM, Server not found in Kerberos database
I20260923 22:13:08.071380 10735 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 22:13:08.066900 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.135.254:38637 (local address 127.1.135.193:54757)
0923 22:13:08.067048 (+ 148us) negotiation.cc:107] Waiting for socket to connect
0923 22:13:08.067058 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0923 22:13:08.067103 (+ 45us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0923 22:13:08.067260 (+ 157us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0923 22:13:08.067261 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0923 22:13:08.067415 (+ 154us) client_negotiation.cc:189] Negotiated authn=SASL
0923 22:13:08.067461 (+ 46us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 22:13:08.067463 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 22:13:08.068058 (+ 595us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 22:13:08.068078 (+ 20us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 22:13:08.068605 (+ 527us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 22:13:08.068606 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 22:13:08.068960 (+ 354us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 22:13:08.068960 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 22:13:08.068997 (+ 37us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 22:13:08.069412 (+ 415us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0923 22:13:08.069417 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0923 22:13:08.071325 (+ 1908us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.1.135.254:38637: Server oryx/127.1.135.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":123,"thread_start_us":83,"threads_started":1}
W20260923 22:13:08.071518 10736 negotiation.cc:337] Failed RPC negotiation. Trace:
0923 22:13:08.066980 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.135.193:54757 (local address 127.1.135.254:38637)
0923 22:13:08.067115 (+ 135us) server_negotiation.cc:206] Beginning negotiation
0923 22:13:08.067117 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0923 22:13:08.067150 (+ 33us) server_negotiation.cc:407] Connection header received
0923 22:13:08.067170 (+ 20us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 22:13:08.067171 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 22:13:08.067181 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 22:13:08.067238 (+ 57us) server_negotiation.cc:226] Negotiated authn=SASL
0923 22:13:08.067524 (+ 286us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 22:13:08.067997 (+ 473us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 22:13:08.068681 (+ 684us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 22:13:08.068891 (+ 210us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 22:13:08.068941 (+ 50us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 22:13:08.071430 (+ 2489us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.1.135.193:54757: BlockingRecv error: recv got EOF from 127.1.135.193:54757 (error 108)
Metrics: {"server-negotiator.queue_time_us":115,"thread_start_us":86,"threads_started":1}
Sep 23 22:13:08 dist-test-slave-s2fj krb5kdc[9697](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.1.135.194@KRBTEST.COM for oryx/127.1.135.254@KRBTEST.COM, Server not found in Kerberos database
Sep 23 22:13:08 dist-test-slave-s2fj krb5kdc[9697](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.1.135.194@KRBTEST.COM for oryx/127.1.135.254@KRBTEST.COM, Server not found in Kerberos database
I20260923 22:13:08.196599 10737 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 22:13:08.192034 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.135.254:38637 (local address 127.1.135.194:46117)
0923 22:13:08.192219 (+ 185us) negotiation.cc:107] Waiting for socket to connect
0923 22:13:08.192231 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0923 22:13:08.192322 (+ 91us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0923 22:13:08.192479 (+ 157us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0923 22:13:08.192480 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0923 22:13:08.192650 (+ 170us) client_negotiation.cc:189] Negotiated authn=SASL
0923 22:13:08.192691 (+ 41us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 22:13:08.192692 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 22:13:08.193343 (+ 651us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 22:13:08.193345 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 22:13:08.193848 (+ 503us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 22:13:08.193850 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 22:13:08.194201 (+ 351us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 22:13:08.194202 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 22:13:08.194256 (+ 54us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 22:13:08.194796 (+ 540us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0923 22:13:08.194800 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0923 22:13:08.196506 (+ 1706us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.1.135.254:38637: Server oryx/127.1.135.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":149,"thread_start_us":94,"threads_started":1}
W20260923 22:13:08.196637 10736 negotiation.cc:337] Failed RPC negotiation. Trace:
0923 22:13:08.192123 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.135.194:46117 (local address 127.1.135.254:38637)
0923 22:13:08.192186 (+ 63us) server_negotiation.cc:206] Beginning negotiation
0923 22:13:08.192188 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0923 22:13:08.192309 (+ 121us) server_negotiation.cc:407] Connection header received
0923 22:13:08.192389 (+ 80us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 22:13:08.192390 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 22:13:08.192402 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 22:13:08.192476 (+ 74us) server_negotiation.cc:226] Negotiated authn=SASL
0923 22:13:08.192772 (+ 296us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 22:13:08.193255 (+ 483us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 22:13:08.193934 (+ 679us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 22:13:08.194126 (+ 192us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 22:13:08.194204 (+ 78us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 22:13:08.196579 (+ 2375us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.1.135.194:46117: BlockingRecv error: recv got EOF from 127.1.135.194:46117 (error 108)
Metrics: {"server-negotiator.queue_time_us":30}
Sep 23 22:13:08 dist-test-slave-s2fj krb5kdc[9697](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.1.135.195@KRBTEST.COM for oryx/127.1.135.254@KRBTEST.COM, Server not found in Kerberos database
Sep 23 22:13:08 dist-test-slave-s2fj krb5kdc[9697](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.1.135.195@KRBTEST.COM for oryx/127.1.135.254@KRBTEST.COM, Server not found in Kerberos database
I20260923 22:13:08.308687 10738 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 22:13:08.304584 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.135.254:38637 (local address 127.1.135.195:53067)
0923 22:13:08.304756 (+ 172us) negotiation.cc:107] Waiting for socket to connect
0923 22:13:08.304766 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0923 22:13:08.304819 (+ 53us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0923 22:13:08.304945 (+ 126us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0923 22:13:08.304945 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0923 22:13:08.305096 (+ 151us) client_negotiation.cc:189] Negotiated authn=SASL
0923 22:13:08.305141 (+ 45us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 22:13:08.305143 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 22:13:08.305740 (+ 597us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 22:13:08.305740 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 22:13:08.306239 (+ 499us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 22:13:08.306240 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 22:13:08.306520 (+ 280us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 22:13:08.306521 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 22:13:08.306554 (+ 33us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 22:13:08.306943 (+ 389us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0923 22:13:08.306947 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0923 22:13:08.308627 (+ 1680us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.1.135.254:38637: Server oryx/127.1.135.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":147,"thread_start_us":115,"threads_started":1}
W20260923 22:13:08.308806 10736 negotiation.cc:337] Failed RPC negotiation. Trace:
0923 22:13:08.304707 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.135.195:53067 (local address 127.1.135.254:38637)
0923 22:13:08.304776 (+ 69us) server_negotiation.cc:206] Beginning negotiation
0923 22:13:08.304778 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0923 22:13:08.304806 (+ 28us) server_negotiation.cc:407] Connection header received
0923 22:13:08.304874 (+ 68us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 22:13:08.304875 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 22:13:08.304886 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 22:13:08.304927 (+ 41us) server_negotiation.cc:226] Negotiated authn=SASL
0923 22:13:08.305215 (+ 288us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 22:13:08.305681 (+ 466us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 22:13:08.306314 (+ 633us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 22:13:08.306466 (+ 152us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 22:13:08.306507 (+ 41us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 22:13:08.308707 (+ 2200us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.1.135.195:53067: BlockingRecv error: recv got EOF from 127.1.135.195:53067 (error 108)
Metrics: {"server-negotiator.queue_time_us":41}
Sep 23 22:13:09 dist-test-slave-s2fj krb5kdc[9697](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.1.135.193@KRBTEST.COM for oryx/127.1.135.254@KRBTEST.COM, Server not found in Kerberos database
Sep 23 22:13:09 dist-test-slave-s2fj krb5kdc[9697](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.1.135.193@KRBTEST.COM for oryx/127.1.135.254@KRBTEST.COM, Server not found in Kerberos database
I20260923 22:13:09.076453 10739 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 22:13:09.071809 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.135.254:38637 (local address 127.1.135.193:38979)
0923 22:13:09.071939 (+ 130us) negotiation.cc:107] Waiting for socket to connect
0923 22:13:09.071956 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0923 22:13:09.072006 (+ 50us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0923 22:13:09.072202 (+ 196us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0923 22:13:09.072208 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0923 22:13:09.072370 (+ 162us) client_negotiation.cc:189] Negotiated authn=SASL
0923 22:13:09.072430 (+ 60us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 22:13:09.072432 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 22:13:09.073099 (+ 667us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 22:13:09.073100 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 22:13:09.073616 (+ 516us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 22:13:09.073618 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 22:13:09.073988 (+ 370us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 22:13:09.073989 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 22:13:09.074027 (+ 38us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 22:13:09.074505 (+ 478us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0923 22:13:09.074509 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0923 22:13:09.076385 (+ 1876us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.1.135.254:38637: Server oryx/127.1.135.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":112,"thread_start_us":81,"threads_started":1}
W20260923 22:13:09.076519 10740 negotiation.cc:337] Failed RPC negotiation. Trace:
0923 22:13:09.071856 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.135.193:38979 (local address 127.1.135.254:38637)
0923 22:13:09.071970 (+ 114us) server_negotiation.cc:206] Beginning negotiation
0923 22:13:09.071972 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0923 22:13:09.072002 (+ 30us) server_negotiation.cc:407] Connection header received
0923 22:13:09.072071 (+ 69us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 22:13:09.072072 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 22:13:09.072081 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 22:13:09.072195 (+ 114us) server_negotiation.cc:226] Negotiated authn=SASL
0923 22:13:09.072483 (+ 288us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 22:13:09.073036 (+ 553us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 22:13:09.073696 (+ 660us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 22:13:09.073924 (+ 228us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 22:13:09.073975 (+ 51us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 22:13:09.076468 (+ 2493us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.1.135.193:38979: BlockingRecv error: recv got EOF from 127.1.135.193:38979 (error 108)
Metrics: {"server-negotiator.queue_time_us":95,"thread_start_us":65,"threads_started":1}
Sep 23 22:13:09 dist-test-slave-s2fj krb5kdc[9697](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.1.135.194@KRBTEST.COM for oryx/127.1.135.254@KRBTEST.COM, Server not found in Kerberos database
Sep 23 22:13:09 dist-test-slave-s2fj krb5kdc[9697](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.1.135.194@KRBTEST.COM for oryx/127.1.135.254@KRBTEST.COM, Server not found in Kerberos database
I20260923 22:13:09.201439 10741 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 22:13:09.197012 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.135.254:38637 (local address 127.1.135.194:38705)
0923 22:13:09.197135 (+ 123us) negotiation.cc:107] Waiting for socket to connect
0923 22:13:09.197144 (+ 9us) client_negotiation.cc:174] Beginning negotiation
0923 22:13:09.197193 (+ 49us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0923 22:13:09.197348 (+ 155us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0923 22:13:09.197349 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0923 22:13:09.197502 (+ 153us) client_negotiation.cc:189] Negotiated authn=SASL
0923 22:13:09.197564 (+ 62us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 22:13:09.197566 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 22:13:09.198232 (+ 666us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 22:13:09.198233 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 22:13:09.198767 (+ 534us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 22:13:09.198768 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 22:13:09.199078 (+ 310us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 22:13:09.199079 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 22:13:09.199117 (+ 38us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 22:13:09.199595 (+ 478us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0923 22:13:09.199599 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0923 22:13:09.201375 (+ 1776us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.1.135.254:38637: Server oryx/127.1.135.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":101,"thread_start_us":73,"threads_started":1}
W20260923 22:13:09.201550 10740 negotiation.cc:337] Failed RPC negotiation. Trace:
0923 22:13:09.197080 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.135.194:38705 (local address 127.1.135.254:38637)
0923 22:13:09.197132 (+ 52us) server_negotiation.cc:206] Beginning negotiation
0923 22:13:09.197134 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0923 22:13:09.197198 (+ 64us) server_negotiation.cc:407] Connection header received
0923 22:13:09.197238 (+ 40us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 22:13:09.197238 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 22:13:09.197248 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 22:13:09.197320 (+ 72us) server_negotiation.cc:226] Negotiated authn=SASL
0923 22:13:09.197668 (+ 348us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 22:13:09.198165 (+ 497us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 22:13:09.198834 (+ 669us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 22:13:09.199016 (+ 182us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 22:13:09.199065 (+ 49us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 22:13:09.201474 (+ 2409us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.1.135.194:38705: BlockingRecv error: recv got EOF from 127.1.135.194:38705 (error 108)
Metrics: {"server-negotiator.queue_time_us":28}
Sep 23 22:13:09 dist-test-slave-s2fj krb5kdc[9697](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.1.135.195@KRBTEST.COM for oryx/127.1.135.254@KRBTEST.COM, Server not found in Kerberos database
Sep 23 22:13:09 dist-test-slave-s2fj krb5kdc[9697](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.1.135.195@KRBTEST.COM for oryx/127.1.135.254@KRBTEST.COM, Server not found in Kerberos database
I20260923 22:13:09.313580 10742 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 22:13:09.309204 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.135.254:38637 (local address 127.1.135.195:53735)
0923 22:13:09.309377 (+ 173us) negotiation.cc:107] Waiting for socket to connect
0923 22:13:09.309394 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0923 22:13:09.309472 (+ 78us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0923 22:13:09.309659 (+ 187us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0923 22:13:09.309660 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0923 22:13:09.309830 (+ 170us) client_negotiation.cc:189] Negotiated authn=SASL
0923 22:13:09.309873 (+ 43us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 22:13:09.309875 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 22:13:09.310596 (+ 721us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 22:13:09.310598 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 22:13:09.311207 (+ 609us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 22:13:09.311210 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 22:13:09.311507 (+ 297us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 22:13:09.311508 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 22:13:09.311543 (+ 35us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 22:13:09.311938 (+ 395us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0923 22:13:09.311942 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0923 22:13:09.313519 (+ 1577us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.1.135.254:38637: Server oryx/127.1.135.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":155,"thread_start_us":101,"threads_started":1}
W20260923 22:13:09.313665 10740 negotiation.cc:337] Failed RPC negotiation. Trace:
0923 22:13:09.309281 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.135.195:53735 (local address 127.1.135.254:38637)
0923 22:13:09.309359 (+ 78us) server_negotiation.cc:206] Beginning negotiation
0923 22:13:09.309361 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0923 22:13:09.309453 (+ 92us) server_negotiation.cc:407] Connection header received
0923 22:13:09.309544 (+ 91us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 22:13:09.309545 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 22:13:09.309568 (+ 23us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 22:13:09.309640 (+ 72us) server_negotiation.cc:226] Negotiated authn=SASL
0923 22:13:09.309961 (+ 321us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 22:13:09.310491 (+ 530us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 22:13:09.311276 (+ 785us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 22:13:09.311441 (+ 165us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 22:13:09.311488 (+ 47us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 22:13:09.313599 (+ 2111us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.1.135.195:53735: BlockingRecv error: recv got EOF from 127.1.135.195:53735 (error 108)
Metrics: {"server-negotiator.queue_time_us":47}
Sep 23 22:13:10 dist-test-slave-s2fj krb5kdc[9697](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.1.135.193@KRBTEST.COM for oryx/127.1.135.254@KRBTEST.COM, Server not found in Kerberos database
Sep 23 22:13:10 dist-test-slave-s2fj krb5kdc[9697](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.1.135.193@KRBTEST.COM for oryx/127.1.135.254@KRBTEST.COM, Server not found in Kerberos database
I20260923 22:13:10.082460 10743 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 22:13:10.076932 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.135.254:38637 (local address 127.1.135.193:34335)
0923 22:13:10.077089 (+ 157us) negotiation.cc:107] Waiting for socket to connect
0923 22:13:10.077101 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0923 22:13:10.077184 (+ 83us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0923 22:13:10.077315 (+ 131us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0923 22:13:10.077317 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0923 22:13:10.077478 (+ 161us) client_negotiation.cc:189] Negotiated authn=SASL
0923 22:13:10.077524 (+ 46us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 22:13:10.077525 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 22:13:10.078139 (+ 614us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 22:13:10.078140 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 22:13:10.078651 (+ 511us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 22:13:10.078652 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 22:13:10.078948 (+ 296us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 22:13:10.078949 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 22:13:10.078986 (+ 37us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 22:13:10.080751 (+ 1765us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0923 22:13:10.080756 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0923 22:13:10.082406 (+ 1650us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.1.135.254:38637: Server oryx/127.1.135.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":136,"thread_start_us":76,"threads_started":1}
W20260923 22:13:10.082576 10744 negotiation.cc:337] Failed RPC negotiation. Trace:
0923 22:13:10.076978 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.135.193:34335 (local address 127.1.135.254:38637)
0923 22:13:10.077130 (+ 152us) server_negotiation.cc:206] Beginning negotiation
0923 22:13:10.077132 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0923 22:13:10.077166 (+ 34us) server_negotiation.cc:407] Connection header received
0923 22:13:10.077227 (+ 61us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 22:13:10.077227 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 22:13:10.077237 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 22:13:10.077295 (+ 58us) server_negotiation.cc:226] Negotiated authn=SASL
0923 22:13:10.077585 (+ 290us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 22:13:10.078073 (+ 488us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 22:13:10.078714 (+ 641us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 22:13:10.078890 (+ 176us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 22:13:10.078935 (+ 45us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 22:13:10.082495 (+ 3560us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.1.135.193:34335: BlockingRecv error: recv got EOF from 127.1.135.193:34335 (error 108)
Metrics: {"server-negotiator.queue_time_us":125,"thread_start_us":80,"threads_started":1}
Sep 23 22:13:10 dist-test-slave-s2fj krb5kdc[9697](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.1.135.194@KRBTEST.COM for oryx/127.1.135.254@KRBTEST.COM, Server not found in Kerberos database
Sep 23 22:13:10 dist-test-slave-s2fj krb5kdc[9697](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.1.135.194@KRBTEST.COM for oryx/127.1.135.254@KRBTEST.COM, Server not found in Kerberos database
I20260923 22:13:10.206513 10745 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 22:13:10.201909 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.135.254:38637 (local address 127.1.135.194:38741)
0923 22:13:10.202048 (+ 139us) negotiation.cc:107] Waiting for socket to connect
0923 22:13:10.202061 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0923 22:13:10.202146 (+ 85us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0923 22:13:10.202301 (+ 155us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0923 22:13:10.202302 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0923 22:13:10.202526 (+ 224us) client_negotiation.cc:189] Negotiated authn=SASL
0923 22:13:10.202568 (+ 42us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 22:13:10.202570 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 22:13:10.203220 (+ 650us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 22:13:10.203221 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 22:13:10.203880 (+ 659us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 22:13:10.203883 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 22:13:10.204350 (+ 467us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 22:13:10.204351 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 22:13:10.204407 (+ 56us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 22:13:10.204830 (+ 423us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0923 22:13:10.204834 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0923 22:13:10.206415 (+ 1581us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.1.135.254:38637: Server oryx/127.1.135.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":101,"thread_start_us":74,"threads_started":1}
W20260923 22:13:10.206550 10744 negotiation.cc:337] Failed RPC negotiation. Trace:
0923 22:13:10.201998 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.135.194:38741 (local address 127.1.135.254:38637)
0923 22:13:10.202064 (+ 66us) server_negotiation.cc:206] Beginning negotiation
0923 22:13:10.202066 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0923 22:13:10.202126 (+ 60us) server_negotiation.cc:407] Connection header received
0923 22:13:10.202217 (+ 91us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 22:13:10.202218 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 22:13:10.202229 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 22:13:10.202298 (+ 69us) server_negotiation.cc:226] Negotiated authn=SASL
0923 22:13:10.202650 (+ 352us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 22:13:10.203129 (+ 479us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 22:13:10.203975 (+ 846us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 22:13:10.204277 (+ 302us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 22:13:10.204353 (+ 76us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 22:13:10.206492 (+ 2139us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.1.135.194:38741: BlockingRecv error: recv got EOF from 127.1.135.194:38741 (error 108)
Metrics: {"server-negotiator.queue_time_us":41}
Sep 23 22:13:10 dist-test-slave-s2fj krb5kdc[9697](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.1.135.195@KRBTEST.COM for oryx/127.1.135.254@KRBTEST.COM, Server not found in Kerberos database
Sep 23 22:13:10 dist-test-slave-s2fj krb5kdc[9697](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.1.135.195@KRBTEST.COM for oryx/127.1.135.254@KRBTEST.COM, Server not found in Kerberos database
I20260923 22:13:10.318395 10746 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 22:13:10.314008 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.135.254:38637 (local address 127.1.135.195:43481)
0923 22:13:10.314165 (+ 157us) negotiation.cc:107] Waiting for socket to connect
0923 22:13:10.314177 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0923 22:13:10.314244 (+ 67us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0923 22:13:10.314383 (+ 139us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0923 22:13:10.314384 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0923 22:13:10.314572 (+ 188us) client_negotiation.cc:189] Negotiated authn=SASL
0923 22:13:10.314614 (+ 42us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 22:13:10.314616 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 22:13:10.315281 (+ 665us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 22:13:10.315283 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 22:13:10.315833 (+ 550us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 22:13:10.315835 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 22:13:10.316231 (+ 396us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 22:13:10.316232 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 22:13:10.316289 (+ 57us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 22:13:10.316698 (+ 409us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0923 22:13:10.316702 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0923 22:13:10.318339 (+ 1637us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.1.135.254:38637: Server oryx/127.1.135.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":139,"thread_start_us":86,"threads_started":1}
W20260923 22:13:10.318511 10744 negotiation.cc:337] Failed RPC negotiation. Trace:
0923 22:13:10.314086 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.135.195:43481 (local address 127.1.135.254:38637)
0923 22:13:10.314160 (+ 74us) server_negotiation.cc:206] Beginning negotiation
0923 22:13:10.314162 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0923 22:13:10.314227 (+ 65us) server_negotiation.cc:407] Connection header received
0923 22:13:10.314296 (+ 69us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 22:13:10.314297 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 22:13:10.314316 (+ 19us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 22:13:10.314378 (+ 62us) server_negotiation.cc:226] Negotiated authn=SASL
0923 22:13:10.314704 (+ 326us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 22:13:10.315201 (+ 497us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 22:13:10.315916 (+ 715us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 22:13:10.316152 (+ 236us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 22:13:10.316231 (+ 79us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 22:13:10.318418 (+ 2187us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.1.135.195:43481: BlockingRecv error: recv got EOF from 127.1.135.195:43481 (error 108)
Metrics: {"server-negotiator.queue_time_us":42}
Sep 23 22:13:11 dist-test-slave-s2fj krb5kdc[9697](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.1.135.193@KRBTEST.COM for oryx/127.1.135.254@KRBTEST.COM, Server not found in Kerberos database
Sep 23 22:13:11 dist-test-slave-s2fj krb5kdc[9697](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.1.135.193@KRBTEST.COM for oryx/127.1.135.254@KRBTEST.COM, Server not found in Kerberos database
I20260923 22:13:11.087777 10747 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 22:13:11.083029 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.135.254:38637 (local address 127.1.135.193:48013)
0923 22:13:11.083180 (+ 151us) negotiation.cc:107] Waiting for socket to connect
0923 22:13:11.083195 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0923 22:13:11.083241 (+ 46us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0923 22:13:11.083372 (+ 131us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0923 22:13:11.083373 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0923 22:13:11.083531 (+ 158us) client_negotiation.cc:189] Negotiated authn=SASL
0923 22:13:11.083574 (+ 43us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 22:13:11.083576 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 22:13:11.084252 (+ 676us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 22:13:11.084253 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 22:13:11.084755 (+ 502us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 22:13:11.084757 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 22:13:11.085064 (+ 307us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 22:13:11.085065 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 22:13:11.085111 (+ 46us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 22:13:11.085722 (+ 611us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0923 22:13:11.085727 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0923 22:13:11.087722 (+ 1995us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.1.135.254:38637: Server oryx/127.1.135.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":129,"thread_start_us":90,"threads_started":1}
W20260923 22:13:11.087936 10748 negotiation.cc:337] Failed RPC negotiation. Trace:
0923 22:13:11.083108 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.135.193:48013 (local address 127.1.135.254:38637)
0923 22:13:11.083239 (+ 131us) server_negotiation.cc:206] Beginning negotiation
0923 22:13:11.083240 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0923 22:13:11.083249 (+ 9us) server_negotiation.cc:407] Connection header received
0923 22:13:11.083290 (+ 41us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 22:13:11.083291 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 22:13:11.083301 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 22:13:11.083352 (+ 51us) server_negotiation.cc:226] Negotiated authn=SASL
0923 22:13:11.083671 (+ 319us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 22:13:11.084192 (+ 521us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 22:13:11.084822 (+ 630us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 22:13:11.085003 (+ 181us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 22:13:11.085051 (+ 48us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 22:13:11.087851 (+ 2800us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.1.135.193:48013: BlockingRecv error: recv got EOF from 127.1.135.193:48013 (error 108)
Metrics: {"server-negotiator.queue_time_us":106,"thread_start_us":72,"threads_started":1}
Sep 23 22:13:11 dist-test-slave-s2fj krb5kdc[9697](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.1.135.194@KRBTEST.COM for oryx/127.1.135.254@KRBTEST.COM, Server not found in Kerberos database
Sep 23 22:13:11 dist-test-slave-s2fj krb5kdc[9697](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.1.135.194@KRBTEST.COM for oryx/127.1.135.254@KRBTEST.COM, Server not found in Kerberos database
I20260923 22:13:11.211683 10749 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 22:13:11.206959 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.135.254:38637 (local address 127.1.135.194:55749)
0923 22:13:11.207068 (+ 109us) negotiation.cc:107] Waiting for socket to connect
0923 22:13:11.207085 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0923 22:13:11.207132 (+ 47us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0923 22:13:11.207256 (+ 124us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0923 22:13:11.207257 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0923 22:13:11.207405 (+ 148us) client_negotiation.cc:189] Negotiated authn=SASL
0923 22:13:11.207449 (+ 44us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 22:13:11.207450 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 22:13:11.208184 (+ 734us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 22:13:11.208185 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 22:13:11.208683 (+ 498us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 22:13:11.208684 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 22:13:11.208997 (+ 313us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 22:13:11.208998 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 22:13:11.209053 (+ 55us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 22:13:11.209646 (+ 593us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0923 22:13:11.209652 (+ 6us) client_negotiation.cc:656] calling sasl_client_start()
0923 22:13:11.211626 (+ 1974us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.1.135.254:38637: Server oryx/127.1.135.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":92,"thread_start_us":67,"threads_started":1}
W20260923 22:13:11.211772 10748 negotiation.cc:337] Failed RPC negotiation. Trace:
0923 22:13:11.207062 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.135.194:55749 (local address 127.1.135.254:38637)
0923 22:13:11.207115 (+ 53us) server_negotiation.cc:206] Beginning negotiation
0923 22:13:11.207117 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0923 22:13:11.207125 (+ 8us) server_negotiation.cc:407] Connection header received
0923 22:13:11.207195 (+ 70us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 22:13:11.207196 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 22:13:11.207205 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 22:13:11.207252 (+ 47us) server_negotiation.cc:226] Negotiated authn=SASL
0923 22:13:11.207533 (+ 281us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 22:13:11.208021 (+ 488us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 22:13:11.208749 (+ 728us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 22:13:11.208939 (+ 190us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 22:13:11.209010 (+ 71us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 22:13:11.211710 (+ 2700us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.1.135.194:55749: BlockingRecv error: recv got EOF from 127.1.135.194:55749 (error 108)
Metrics: {"server-negotiator.queue_time_us":31}
Sep 23 22:13:11 dist-test-slave-s2fj krb5kdc[9697](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.1.135.195@KRBTEST.COM for oryx/127.1.135.254@KRBTEST.COM, Server not found in Kerberos database
Sep 23 22:13:11 dist-test-slave-s2fj krb5kdc[9697](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.1.135.195@KRBTEST.COM for oryx/127.1.135.254@KRBTEST.COM, Server not found in Kerberos database
I20260923 22:13:11.323347 10750 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 22:13:11.318944 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.135.254:38637 (local address 127.1.135.195:48381)
0923 22:13:11.319114 (+ 170us) negotiation.cc:107] Waiting for socket to connect
0923 22:13:11.319127 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0923 22:13:11.319185 (+ 58us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0923 22:13:11.319332 (+ 147us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0923 22:13:11.319333 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0923 22:13:11.319503 (+ 170us) client_negotiation.cc:189] Negotiated authn=SASL
0923 22:13:11.319548 (+ 45us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 22:13:11.319550 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 22:13:11.320174 (+ 624us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 22:13:11.320174 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 22:13:11.320717 (+ 543us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 22:13:11.320719 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 22:13:11.321044 (+ 325us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 22:13:11.321045 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 22:13:11.321083 (+ 38us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 22:13:11.321496 (+ 413us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0923 22:13:11.321500 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0923 22:13:11.323286 (+ 1786us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.1.135.254:38637: Server oryx/127.1.135.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":153,"thread_start_us":122,"threads_started":1}
W20260923 22:13:11.323407 10748 negotiation.cc:337] Failed RPC negotiation. Trace:
0923 22:13:11.319007 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.135.195:48381 (local address 127.1.135.254:38637)
0923 22:13:11.319073 (+ 66us) server_negotiation.cc:206] Beginning negotiation
0923 22:13:11.319074 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0923 22:13:11.319240 (+ 166us) server_negotiation.cc:407] Connection header received
0923 22:13:11.319260 (+ 20us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 22:13:11.319261 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 22:13:11.319273 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 22:13:11.319313 (+ 40us) server_negotiation.cc:226] Negotiated authn=SASL
0923 22:13:11.319606 (+ 293us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 22:13:11.320117 (+ 511us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 22:13:11.320792 (+ 675us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 22:13:11.320967 (+ 175us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 22:13:11.321010 (+ 43us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 22:13:11.323363 (+ 2353us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.1.135.195:48381: BlockingRecv error: recv got EOF from 127.1.135.195:48381 (error 108)
Metrics: {"server-negotiator.queue_time_us":34}
Sep 23 22:13:12 dist-test-slave-s2fj krb5kdc[9697](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.1.135.193@KRBTEST.COM for oryx/127.1.135.254@KRBTEST.COM, Server not found in Kerberos database
Sep 23 22:13:12 dist-test-slave-s2fj krb5kdc[9697](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.1.135.193@KRBTEST.COM for oryx/127.1.135.254@KRBTEST.COM, Server not found in Kerberos database
I20260923 22:13:12.092520 10751 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 22:13:12.088306 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.135.254:38637 (local address 127.1.135.193:37603)
0923 22:13:12.088455 (+ 149us) negotiation.cc:107] Waiting for socket to connect
0923 22:13:12.088467 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0923 22:13:12.088519 (+ 52us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0923 22:13:12.088657 (+ 138us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0923 22:13:12.088658 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0923 22:13:12.088817 (+ 159us) client_negotiation.cc:189] Negotiated authn=SASL
0923 22:13:12.088864 (+ 47us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 22:13:12.088865 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 22:13:12.089484 (+ 619us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 22:13:12.089485 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 22:13:12.089989 (+ 504us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 22:13:12.089990 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 22:13:12.090309 (+ 319us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 22:13:12.090309 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 22:13:12.090347 (+ 38us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 22:13:12.090770 (+ 423us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0923 22:13:12.090774 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0923 22:13:12.092462 (+ 1688us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.1.135.254:38637: Server oryx/127.1.135.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":130,"thread_start_us":91,"threads_started":1}
W20260923 22:13:12.092615 10752 negotiation.cc:337] Failed RPC negotiation. Trace:
0923 22:13:12.088386 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.135.193:37603 (local address 127.1.135.254:38637)
0923 22:13:12.088514 (+ 128us) server_negotiation.cc:206] Beginning negotiation
0923 22:13:12.088516 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0923 22:13:12.088552 (+ 36us) server_negotiation.cc:407] Connection header received
0923 22:13:12.088572 (+ 20us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 22:13:12.088572 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 22:13:12.088583 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 22:13:12.088633 (+ 50us) server_negotiation.cc:226] Negotiated authn=SASL
0923 22:13:12.088929 (+ 296us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 22:13:12.089420 (+ 491us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 22:13:12.090056 (+ 636us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 22:13:12.090245 (+ 189us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 22:13:12.090294 (+ 49us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 22:13:12.092545 (+ 2251us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.1.135.193:37603: BlockingRecv error: recv got EOF from 127.1.135.193:37603 (error 108)
Metrics: {"server-negotiator.queue_time_us":109,"thread_start_us":80,"threads_started":1}
Sep 23 22:13:12 dist-test-slave-s2fj krb5kdc[9697](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.1.135.194@KRBTEST.COM for oryx/127.1.135.254@KRBTEST.COM, Server not found in Kerberos database
Sep 23 22:13:12 dist-test-slave-s2fj krb5kdc[9697](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.1.135.194@KRBTEST.COM for oryx/127.1.135.254@KRBTEST.COM, Server not found in Kerberos database
I20260923 22:13:12.222064 10753 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 22:13:12.212229 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.135.254:38637 (local address 127.1.135.194:57479)
0923 22:13:12.212378 (+ 149us) negotiation.cc:107] Waiting for socket to connect
0923 22:13:12.212388 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0923 22:13:12.212441 (+ 53us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0923 22:13:12.212553 (+ 112us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0923 22:13:12.212554 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0923 22:13:12.212719 (+ 165us) client_negotiation.cc:189] Negotiated authn=SASL
0923 22:13:12.212764 (+ 45us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 22:13:12.212765 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 22:13:12.213358 (+ 593us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 22:13:12.213359 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 22:13:12.213886 (+ 527us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 22:13:12.213887 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 22:13:12.214384 (+ 497us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 22:13:12.214386 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 22:13:12.214448 (+ 62us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 22:13:12.220271 (+ 5823us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0923 22:13:12.220281 (+ 10us) client_negotiation.cc:656] calling sasl_client_start()
0923 22:13:12.222003 (+ 1722us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.1.135.254:38637: Server oryx/127.1.135.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":133,"thread_start_us":78,"threads_started":1}
W20260923 22:13:12.222208 10752 negotiation.cc:337] Failed RPC negotiation. Trace:
0923 22:13:12.212292 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.135.194:57479 (local address 127.1.135.254:38637)
0923 22:13:12.212371 (+ 79us) server_negotiation.cc:206] Beginning negotiation
0923 22:13:12.212373 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0923 22:13:12.212443 (+ 70us) server_negotiation.cc:407] Connection header received
0923 22:13:12.212483 (+ 40us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 22:13:12.212484 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 22:13:12.212495 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 22:13:12.212542 (+ 47us) server_negotiation.cc:226] Negotiated authn=SASL
0923 22:13:12.212818 (+ 276us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 22:13:12.213292 (+ 474us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 22:13:12.213970 (+ 678us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 22:13:12.214303 (+ 333us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 22:13:12.214372 (+ 69us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 22:13:12.222128 (+ 7756us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.1.135.194:57479: BlockingRecv error: recv got EOF from 127.1.135.194:57479 (error 108)
Metrics: {"server-negotiator.queue_time_us":49}
Sep 23 22:13:12 dist-test-slave-s2fj krb5kdc[9697](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.1.135.195@KRBTEST.COM for oryx/127.1.135.254@KRBTEST.COM, Server not found in Kerberos database
Sep 23 22:13:12 dist-test-slave-s2fj krb5kdc[9697](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.1.135.195@KRBTEST.COM for oryx/127.1.135.254@KRBTEST.COM, Server not found in Kerberos database
I20260923 22:13:12.328284 10754 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 22:13:12.323805 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.135.254:38637 (local address 127.1.135.195:54019)
0923 22:13:12.323978 (+ 173us) negotiation.cc:107] Waiting for socket to connect
0923 22:13:12.323988 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0923 22:13:12.324085 (+ 97us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0923 22:13:12.324223 (+ 138us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0923 22:13:12.324224 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0923 22:13:12.324384 (+ 160us) client_negotiation.cc:189] Negotiated authn=SASL
0923 22:13:12.324429 (+ 45us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 22:13:12.324432 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 22:13:12.325048 (+ 616us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 22:13:12.325048 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 22:13:12.325656 (+ 608us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 22:13:12.325659 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 22:13:12.325949 (+ 290us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 22:13:12.325949 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 22:13:12.325987 (+ 38us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 22:13:12.326470 (+ 483us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0923 22:13:12.326475 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0923 22:13:12.328229 (+ 1754us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.1.135.254:38637: Server oryx/127.1.135.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":144,"thread_start_us":95,"threads_started":1}
W20260923 22:13:12.328402 10752 negotiation.cc:337] Failed RPC negotiation. Trace:
0923 22:13:12.323849 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.135.195:54019 (local address 127.1.135.254:38637)
0923 22:13:12.323904 (+ 55us) server_negotiation.cc:206] Beginning negotiation
0923 22:13:12.323906 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0923 22:13:12.324087 (+ 181us) server_negotiation.cc:407] Connection header received
0923 22:13:12.324135 (+ 48us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 22:13:12.324136 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 22:13:12.324150 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 22:13:12.324208 (+ 58us) server_negotiation.cc:226] Negotiated authn=SASL
0923 22:13:12.324488 (+ 280us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 22:13:12.324965 (+ 477us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 22:13:12.325718 (+ 753us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 22:13:12.325888 (+ 170us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 22:13:12.325937 (+ 49us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 22:13:12.328309 (+ 2372us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.1.135.195:54019: BlockingRecv error: recv got EOF from 127.1.135.195:54019 (error 108)
Metrics: {"server-negotiator.queue_time_us":32}
Sep 23 22:13:13 dist-test-slave-s2fj krb5kdc[9697](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.1.135.193@KRBTEST.COM for oryx/127.1.135.254@KRBTEST.COM, Server not found in Kerberos database
Sep 23 22:13:13 dist-test-slave-s2fj krb5kdc[9697](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.1.135.193@KRBTEST.COM for oryx/127.1.135.254@KRBTEST.COM, Server not found in Kerberos database
I20260923 22:13:13.098219 10755 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 22:13:13.093224 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.135.254:38637 (local address 127.1.135.193:43439)
0923 22:13:13.093380 (+ 156us) negotiation.cc:107] Waiting for socket to connect
0923 22:13:13.093390 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0923 22:13:13.093496 (+ 106us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0923 22:13:13.093693 (+ 197us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0923 22:13:13.093695 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0923 22:13:13.093914 (+ 219us) client_negotiation.cc:189] Negotiated authn=SASL
0923 22:13:13.093970 (+ 56us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 22:13:13.093972 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 22:13:13.094953 (+ 981us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 22:13:13.094954 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 22:13:13.095605 (+ 651us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 22:13:13.095607 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 22:13:13.095914 (+ 307us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 22:13:13.095915 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 22:13:13.095951 (+ 36us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 22:13:13.096462 (+ 511us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0923 22:13:13.096472 (+ 10us) client_negotiation.cc:656] calling sasl_client_start()
0923 22:13:13.098164 (+ 1692us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.1.135.254:38637: Server oryx/127.1.135.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":138,"thread_start_us":96,"threads_started":1}
W20260923 22:13:13.098313 10756 negotiation.cc:337] Failed RPC negotiation. Trace:
0923 22:13:13.093317 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.135.193:43439 (local address 127.1.135.254:38637)
0923 22:13:13.093446 (+ 129us) server_negotiation.cc:206] Beginning negotiation
0923 22:13:13.093448 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0923 22:13:13.093479 (+ 31us) server_negotiation.cc:407] Connection header received
0923 22:13:13.093586 (+ 107us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 22:13:13.093587 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 22:13:13.093599 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 22:13:13.093677 (+ 78us) server_negotiation.cc:226] Negotiated authn=SASL
0923 22:13:13.094094 (+ 417us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 22:13:13.094868 (+ 774us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 22:13:13.095672 (+ 804us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 22:13:13.095853 (+ 181us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 22:13:13.095904 (+ 51us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 22:13:13.098241 (+ 2337us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.1.135.193:43439: BlockingRecv error: recv got EOF from 127.1.135.193:43439 (error 108)
Metrics: {"server-negotiator.queue_time_us":109,"thread_start_us":85,"threads_started":1}
Sep 23 22:13:13 dist-test-slave-s2fj krb5kdc[9697](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.1.135.194@KRBTEST.COM for oryx/127.1.135.254@KRBTEST.COM, Server not found in Kerberos database
Sep 23 22:13:13 dist-test-slave-s2fj krb5kdc[9697](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.1.135.194@KRBTEST.COM for oryx/127.1.135.254@KRBTEST.COM, Server not found in Kerberos database
I20260923 22:13:13.228371 10757 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 22:13:13.222521 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.135.254:38637 (local address 127.1.135.194:35943)
0923 22:13:13.222675 (+ 154us) negotiation.cc:107] Waiting for socket to connect
0923 22:13:13.222686 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0923 22:13:13.222758 (+ 72us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0923 22:13:13.222952 (+ 194us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0923 22:13:13.222954 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0923 22:13:13.223177 (+ 223us) client_negotiation.cc:189] Negotiated authn=SASL
0923 22:13:13.223234 (+ 57us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 22:13:13.223236 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 22:13:13.224203 (+ 967us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 22:13:13.224204 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 22:13:13.224910 (+ 706us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 22:13:13.224912 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 22:13:13.225409 (+ 497us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 22:13:13.225410 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 22:13:13.225476 (+ 66us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 22:13:13.225921 (+ 445us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0923 22:13:13.225927 (+ 6us) client_negotiation.cc:656] calling sasl_client_start()
0923 22:13:13.228304 (+ 2377us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.1.135.254:38637: Server oryx/127.1.135.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":137,"thread_start_us":88,"threads_started":1}
W20260923 22:13:13.228487 10756 negotiation.cc:337] Failed RPC negotiation. Trace:
0923 22:13:13.222591 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.135.194:35943 (local address 127.1.135.254:38637)
0923 22:13:13.222663 (+ 72us) server_negotiation.cc:206] Beginning negotiation
0923 22:13:13.222665 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0923 22:13:13.222743 (+ 78us) server_negotiation.cc:407] Connection header received
0923 22:13:13.222834 (+ 91us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 22:13:13.222835 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 22:13:13.222850 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 22:13:13.222936 (+ 86us) server_negotiation.cc:226] Negotiated authn=SASL
0923 22:13:13.223310 (+ 374us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 22:13:13.224111 (+ 801us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 22:13:13.225003 (+ 892us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 22:13:13.225315 (+ 312us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 22:13:13.225375 (+ 60us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 22:13:13.228395 (+ 3020us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.1.135.194:35943: BlockingRecv error: recv got EOF from 127.1.135.194:35943 (error 108)
Metrics: {"server-negotiator.queue_time_us":43}
I20260923 22:13:13.255090 1567 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskRiQe_a/build/release/bin/kudu with pid 10280
I20260923 22:13:13.259572 1567 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskRiQe_a/build/release/bin/kudu with pid 10432
I20260923 22:13:13.263679 1567 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskRiQe_a/build/release/bin/kudu with pid 10566
I20260923 22:13:13.267618 1567 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskRiQe_a/build/release/bin/kudu with pid 10210
2026-09-23T22:13:13Z chronyd exiting
[ OK ] SecurityITest.TestMismatchingPrincipals (17407 ms)
[ RUN ] SecurityITest.TestRequireAuthenticationInsecureCluster
2026-09-23T22:13:13Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-09-23T22:13:13Z Disabled control of system clock
I20260923 22:13:13.294780 1567 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskRiQe_a/build/release/bin/kudu
/tmp/dist-test-taskRiQe_a/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskRiQe_a/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790201539513534-1567-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskRiQe_a/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790201539513534-1567-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskRiQe_a/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790201539513534-1567-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskRiQe_a/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790201539513534-1567-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.1.135.254:43715
--webserver_interface=127.1.135.254
--webserver_port=0
--builtin_ntp_servers=127.1.135.212:40669
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.1.135.254:43715
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {}
W20260923 22:13:13.373831 10761 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260923 22:13:13.374045 10761 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260923 22:13:13.374078 10761 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260923 22:13:13.375173 10761 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260923 22:13:13.375233 10761 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260923 22:13:13.375257 10761 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260923 22:13:13.375285 10761 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260923 22:13:13.375308 10761 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260923 22:13:13.376668 10761 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.1.135.212:40669
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskRiQe_a/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790201539513534-1567-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskRiQe_a/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790201539513534-1567-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.1.135.254:43715
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.1.135.254:43715
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskRiQe_a/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790201539513534-1567-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.1.135.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.10761
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskRiQe_a/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790201539513534-1567-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 23 Sep 2026 20:53:37 UTC on ccycloud.quasar-rkviyx.root.comops.site
build id 4668083
I20260923 22:13:13.376924 10761 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260923 22:13:13.377198 10761 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260923 22:13:13.381959 10766 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260923 22:13:13.381984 10769 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260923 22:13:13.382025 10761 server_base.cc:1034] running on GCE node
W20260923 22:13:13.381959 10767 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260923 22:13:13.382452 10761 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260923 22:13:13.382714 10761 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260923 22:13:13.383847 10761 hybrid_clock.cc:648] HybridClock initialized: now 1790201593383837 us; error 28 us; skew 500 ppm
I20260923 22:13:13.385147 10761 webserver.cc:460] Webserver started at http://127.1.135.254:32921/ using document root <none> and password file <none>
I20260923 22:13:13.385399 10761 fs_manager.cc:359] Metadata directory not provided
I20260923 22:13:13.385445 10761 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260923 22:13:13.385629 10761 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260923 22:13:13.386617 10761 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskRiQe_a/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790201539513534-1567-0/minicluster-data/master-0/data/instance:
uuid: "1226b4db21124d55a14cd6385fe0c00f"
format_stamp: "Formatted at 2026-09-23 22:13:13 on dist-test-slave-s2fj"
server_key: "e7c70c8c90907261f62422ceb436f2b6"
server_key_iv: "2401cef16b16e748f9110a49634e1044"
server_key_version: "encryptionkey@0"
I20260923 22:13:13.386982 10761 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskRiQe_a/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790201539513534-1567-0/minicluster-data/master-0/wal/instance:
uuid: "1226b4db21124d55a14cd6385fe0c00f"
format_stamp: "Formatted at 2026-09-23 22:13:13 on dist-test-slave-s2fj"
server_key: "e7c70c8c90907261f62422ceb436f2b6"
server_key_iv: "2401cef16b16e748f9110a49634e1044"
server_key_version: "encryptionkey@0"
I20260923 22:13:13.388224 10761 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.003s
I20260923 22:13:13.388952 10775 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260923 22:13:13.389125 10761 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260923 22:13:13.389191 10761 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskRiQe_a/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790201539513534-1567-0/minicluster-data/master-0/data,/tmp/dist-test-taskRiQe_a/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790201539513534-1567-0/minicluster-data/master-0/wal
uuid: "1226b4db21124d55a14cd6385fe0c00f"
format_stamp: "Formatted at 2026-09-23 22:13:13 on dist-test-slave-s2fj"
server_key: "e7c70c8c90907261f62422ceb436f2b6"
server_key_iv: "2401cef16b16e748f9110a49634e1044"
server_key_version: "encryptionkey@0"
I20260923 22:13:13.389281 10761 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskRiQe_a/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790201539513534-1567-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskRiQe_a/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790201539513534-1567-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskRiQe_a/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790201539513534-1567-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260923 22:13:13.412907 10761 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260923 22:13:13.413228 10761 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260923 22:13:13.413394 10761 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260923 22:13:13.417500 10761 rpc_server.cc:307] RPC server started. Bound to: 127.1.135.254:43715
I20260923 22:13:13.417565 10827 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.1.135.254:43715 every 8 connection(s)
I20260923 22:13:13.417881 10761 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskRiQe_a/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790201539513534-1567-0/minicluster-data/master-0/data/info.pb
I20260923 22:13:13.418497 10828 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260923 22:13:13.419183 1567 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskRiQe_a/build/release/bin/kudu as pid 10761
I20260923 22:13:13.419282 1567 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskRiQe_a/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790201539513534-1567-0/minicluster-data/master-0/wal/instance
I20260923 22:13:13.419494 1567 external_mini_cluster.cc:1428] Setting key cded26a6baba584bdc0e08e49e1cd89c
I20260923 22:13:13.421725 10828 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 1226b4db21124d55a14cd6385fe0c00f: Bootstrap starting.
I20260923 22:13:13.422433 10828 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 1226b4db21124d55a14cd6385fe0c00f: Neither blocks nor log segments found. Creating new log.
I20260923 22:13:13.422724 10828 log.cc:824] T 00000000000000000000000000000000 P 1226b4db21124d55a14cd6385fe0c00f: Log is configured to *not* fsync() on all Append() calls
I20260923 22:13:13.423390 10828 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 1226b4db21124d55a14cd6385fe0c00f: No bootstrap required, opened a new log
I20260923 22:13:13.425096 10831 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 22:13:13.419996 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60156 (local address 127.1.135.254:43715)
0923 22:13:13.420227 (+ 231us) server_negotiation.cc:206] Beginning negotiation
0923 22:13:13.420234 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0923 22:13:13.420258 (+ 24us) server_negotiation.cc:407] Connection header received
0923 22:13:13.420670 (+ 412us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 22:13:13.420676 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 22:13:13.420733 (+ 57us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 22:13:13.420842 (+ 109us) server_negotiation.cc:226] Negotiated authn=SASL
0923 22:13:13.421065 (+ 223us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 22:13:13.422141 (+ 1076us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 22:13:13.423210 (+ 1069us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 22:13:13.423633 (+ 423us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 22:13:13.423718 (+ 85us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 22:13:13.424450 (+ 732us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0923 22:13:13.424454 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0923 22:13:13.424454 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0923 22:13:13.424617 (+ 163us) server_negotiation.cc:922] calling sasl_server_start()
0923 22:13:13.424648 (+ 31us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0923 22:13:13.424658 (+ 10us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0923 22:13:13.424710 (+ 52us) server_negotiation.cc:1035] Waiting for connection context
0923 22:13:13.424815 (+ 105us) server_negotiation.cc:299] Negotiation successful
0923 22:13:13.424903 (+ 88us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":142,"thread_start_us":116,"threads_started":1}
I20260923 22:13:13.425778 10828 raft_consensus.cc:348] T 00000000000000000000000000000000 P 1226b4db21124d55a14cd6385fe0c00f [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1226b4db21124d55a14cd6385fe0c00f" member_type: VOTER last_known_addr { host: "127.1.135.254" port: 43715 } }
I20260923 22:13:13.425952 10828 raft_consensus.cc:374] T 00000000000000000000000000000000 P 1226b4db21124d55a14cd6385fe0c00f [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260923 22:13:13.426026 10828 raft_consensus.cc:729] T 00000000000000000000000000000000 P 1226b4db21124d55a14cd6385fe0c00f [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 1226b4db21124d55a14cd6385fe0c00f, State: Initialized, Role: FOLLOWER
I20260923 22:13:13.426210 10828 consensus_queue.cc:260] T 00000000000000000000000000000000 P 1226b4db21124d55a14cd6385fe0c00f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1226b4db21124d55a14cd6385fe0c00f" member_type: VOTER last_known_addr { host: "127.1.135.254" port: 43715 } }
I20260923 22:13:13.426296 10828 raft_consensus.cc:388] T 00000000000000000000000000000000 P 1226b4db21124d55a14cd6385fe0c00f [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260923 22:13:13.426339 10828 raft_consensus.cc:482] T 00000000000000000000000000000000 P 1226b4db21124d55a14cd6385fe0c00f [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260923 22:13:13.426388 10828 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 1226b4db21124d55a14cd6385fe0c00f [term 0 FOLLOWER]: Advancing to term 1
I20260923 22:13:13.426906 10828 raft_consensus.cc:504] T 00000000000000000000000000000000 P 1226b4db21124d55a14cd6385fe0c00f [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1226b4db21124d55a14cd6385fe0c00f" member_type: VOTER last_known_addr { host: "127.1.135.254" port: 43715 } }
I20260923 22:13:13.427022 10828 leader_election.cc:302] T 00000000000000000000000000000000 P 1226b4db21124d55a14cd6385fe0c00f [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 1226b4db21124d55a14cd6385fe0c00f; no voters:
I20260923 22:13:13.427201 10828 leader_election.cc:288] T 00000000000000000000000000000000 P 1226b4db21124d55a14cd6385fe0c00f [CANDIDATE]: Term 1 election: Requested vote from peers
I20260923 22:13:13.427369 10834 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 1226b4db21124d55a14cd6385fe0c00f [term 1 FOLLOWER]: Leader election won for term 1
I20260923 22:13:13.427462 10828 sys_catalog.cc:564] T 00000000000000000000000000000000 P 1226b4db21124d55a14cd6385fe0c00f [sys.catalog]: configured and running, proceeding with master startup.
I20260923 22:13:13.427567 10834 raft_consensus.cc:686] T 00000000000000000000000000000000 P 1226b4db21124d55a14cd6385fe0c00f [term 1 LEADER]: Becoming Leader. State: Replica: 1226b4db21124d55a14cd6385fe0c00f, State: Running, Role: LEADER
I20260923 22:13:13.427680 10834 consensus_queue.cc:237] T 00000000000000000000000000000000 P 1226b4db21124d55a14cd6385fe0c00f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1226b4db21124d55a14cd6385fe0c00f" member_type: VOTER last_known_addr { host: "127.1.135.254" port: 43715 } }
I20260923 22:13:13.428052 10833 sys_catalog.cc:455] T 00000000000000000000000000000000 P 1226b4db21124d55a14cd6385fe0c00f [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "1226b4db21124d55a14cd6385fe0c00f" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1226b4db21124d55a14cd6385fe0c00f" member_type: VOTER last_known_addr { host: "127.1.135.254" port: 43715 } } }
I20260923 22:13:13.428164 10833 sys_catalog.cc:458] T 00000000000000000000000000000000 P 1226b4db21124d55a14cd6385fe0c00f [sys.catalog]: This master's current role is: LEADER
I20260923 22:13:13.428359 10835 sys_catalog.cc:455] T 00000000000000000000000000000000 P 1226b4db21124d55a14cd6385fe0c00f [sys.catalog]: SysCatalogTable state changed. Reason: New leader 1226b4db21124d55a14cd6385fe0c00f. Latest consensus state: current_term: 1 leader_uuid: "1226b4db21124d55a14cd6385fe0c00f" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1226b4db21124d55a14cd6385fe0c00f" member_type: VOTER last_known_addr { host: "127.1.135.254" port: 43715 } } }
I20260923 22:13:13.428637 10835 sys_catalog.cc:458] T 00000000000000000000000000000000 P 1226b4db21124d55a14cd6385fe0c00f [sys.catalog]: This master's current role is: LEADER
W20260923 22:13:13.429267 10849 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 1226b4db21124d55a14cd6385fe0c00f: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260923 22:13:13.429333 10849 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260923 22:13:13.429378 10842 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260923 22:13:13.429507 10842 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260923 22:13:13.430878 10842 catalog_manager.cc:1347] Generated new cluster ID: 56ead67653e04b559c624035af6a8f5b
I20260923 22:13:13.430930 10842 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260923 22:13:13.456081 10842 catalog_manager.cc:1370] Generated new certificate authority record
I20260923 22:13:13.456791 10842 catalog_manager.cc:1504] Loading token signing keys...
I20260923 22:13:13.462198 10842 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 1226b4db21124d55a14cd6385fe0c00f: Generated new TSK 0
I20260923 22:13:13.462366 10842 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260923 22:13:13.465837 1567 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskRiQe_a/build/release/bin/kudu
/tmp/dist-test-taskRiQe_a/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskRiQe_a/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790201539513534-1567-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskRiQe_a/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790201539513534-1567-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskRiQe_a/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790201539513534-1567-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskRiQe_a/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790201539513534-1567-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.1.135.193:0
--local_ip_for_outbound_sockets=127.1.135.193
--webserver_interface=127.1.135.193
--webserver_port=0
--tserver_master_addrs=127.1.135.254:43715
--builtin_ntp_servers=127.1.135.212:40669
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {}
W20260923 22:13:13.544452 10852 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260923 22:13:13.544649 10852 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260923 22:13:13.544670 10852 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260923 22:13:13.545811 10852 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260923 22:13:13.545851 10852 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260923 22:13:13.545874 10852 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.1.135.193
I20260923 22:13:13.547132 10852 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.1.135.212:40669
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskRiQe_a/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790201539513534-1567-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskRiQe_a/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790201539513534-1567-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.1.135.193:0
--rpc_server_allow_ephemeral_ports=true
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskRiQe_a/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790201539513534-1567-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.1.135.193
--webserver_port=0
--tserver_master_addrs=127.1.135.254:43715
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.10852
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.1.135.193
--log_dir=/tmp/dist-test-taskRiQe_a/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790201539513534-1567-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 23 Sep 2026 20:53:37 UTC on ccycloud.quasar-rkviyx.root.comops.site
build id 4668083
I20260923 22:13:13.547350 10852 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260923 22:13:13.547648 10852 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260923 22:13:13.553148 10858 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260923 22:13:13.553138 10857 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260923 22:13:13.553138 10860 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260923 22:13:13.553297 10852 server_base.cc:1034] running on GCE node
I20260923 22:13:13.553501 10852 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260923 22:13:13.553748 10852 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260923 22:13:13.554910 10852 hybrid_clock.cc:648] HybridClock initialized: now 1790201593554899 us; error 27 us; skew 500 ppm
I20260923 22:13:13.556190 10852 webserver.cc:460] Webserver started at http://127.1.135.193:46673/ using document root <none> and password file <none>
I20260923 22:13:13.556413 10852 fs_manager.cc:359] Metadata directory not provided
I20260923 22:13:13.556480 10852 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260923 22:13:13.556619 10852 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260923 22:13:13.557578 10852 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskRiQe_a/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790201539513534-1567-0/minicluster-data/ts-0/data/instance:
uuid: "cd854351062b4cd3b0d3dfe8a6597e57"
format_stamp: "Formatted at 2026-09-23 22:13:13 on dist-test-slave-s2fj"
server_key: "a07a1982a23c74009f12d22450a214a9"
server_key_iv: "24a0a616c5c452e05bffa6b274e7f0d8"
server_key_version: "encryptionkey@0"
I20260923 22:13:13.557927 10852 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskRiQe_a/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790201539513534-1567-0/minicluster-data/ts-0/wal/instance:
uuid: "cd854351062b4cd3b0d3dfe8a6597e57"
format_stamp: "Formatted at 2026-09-23 22:13:13 on dist-test-slave-s2fj"
server_key: "a07a1982a23c74009f12d22450a214a9"
server_key_iv: "24a0a616c5c452e05bffa6b274e7f0d8"
server_key_version: "encryptionkey@0"
I20260923 22:13:13.559168 10852 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.003s
I20260923 22:13:13.559821 10866 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260923 22:13:13.559967 10852 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.001s
I20260923 22:13:13.560067 10852 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskRiQe_a/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790201539513534-1567-0/minicluster-data/ts-0/data,/tmp/dist-test-taskRiQe_a/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790201539513534-1567-0/minicluster-data/ts-0/wal
uuid: "cd854351062b4cd3b0d3dfe8a6597e57"
format_stamp: "Formatted at 2026-09-23 22:13:13 on dist-test-slave-s2fj"
server_key: "a07a1982a23c74009f12d22450a214a9"
server_key_iv: "24a0a616c5c452e05bffa6b274e7f0d8"
server_key_version: "encryptionkey@0"
I20260923 22:13:13.560159 10852 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskRiQe_a/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790201539513534-1567-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskRiQe_a/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790201539513534-1567-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskRiQe_a/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790201539513534-1567-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260923 22:13:13.592712 10852 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260923 22:13:13.593004 10852 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260923 22:13:13.593113 10852 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260923 22:13:13.593276 10852 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260923 22:13:13.593600 10852 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260923 22:13:13.593675 10852 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260923 22:13:13.593716 10852 ts_tablet_manager.cc:610] Registered 0 tablets
I20260923 22:13:13.593741 10852 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260923 22:13:13.599625 10852 rpc_server.cc:307] RPC server started. Bound to: 127.1.135.193:44401
I20260923 22:13:13.599663 10979 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.1.135.193:44401 every 8 connection(s)
I20260923 22:13:13.599982 10852 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskRiQe_a/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790201539513534-1567-0/minicluster-data/ts-0/data/info.pb
I20260923 22:13:13.600759 1567 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskRiQe_a/build/release/bin/kudu as pid 10852
I20260923 22:13:13.600848 1567 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskRiQe_a/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790201539513534-1567-0/minicluster-data/ts-0/wal/instance
I20260923 22:13:13.601071 1567 external_mini_cluster.cc:1428] Setting key 8a5033a888165e2ab538f80e7a883e83
I20260923 22:13:13.601974 1567 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskRiQe_a/build/release/bin/kudu
/tmp/dist-test-taskRiQe_a/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskRiQe_a/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790201539513534-1567-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskRiQe_a/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790201539513534-1567-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskRiQe_a/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790201539513534-1567-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskRiQe_a/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790201539513534-1567-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.1.135.194:0
--local_ip_for_outbound_sockets=127.1.135.194
--webserver_interface=127.1.135.194
--webserver_port=0
--tserver_master_addrs=127.1.135.254:43715
--builtin_ntp_servers=127.1.135.212:40669
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {}
I20260923 22:13:13.605510 10831 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 22:13:13.600723 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.135.193:43039 (local address 127.1.135.254:43715)
0923 22:13:13.600773 (+ 50us) server_negotiation.cc:206] Beginning negotiation
0923 22:13:13.600775 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0923 22:13:13.601045 (+ 270us) server_negotiation.cc:407] Connection header received
0923 22:13:13.601690 (+ 645us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 22:13:13.601691 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 22:13:13.601706 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 22:13:13.601768 (+ 62us) server_negotiation.cc:226] Negotiated authn=SASL
0923 22:13:13.602605 (+ 837us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 22:13:13.603072 (+ 467us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 22:13:13.604082 (+ 1010us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 22:13:13.604287 (+ 205us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 22:13:13.604372 (+ 85us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 22:13:13.605222 (+ 850us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0923 22:13:13.605223 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0923 22:13:13.605223 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0923 22:13:13.605228 (+ 5us) server_negotiation.cc:922] calling sasl_server_start()
0923 22:13:13.605241 (+ 13us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0923 22:13:13.605253 (+ 12us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0923 22:13:13.605310 (+ 57us) server_negotiation.cc:1035] Waiting for connection context
0923 22:13:13.605401 (+ 91us) server_negotiation.cc:299] Negotiation successful
0923 22:13:13.605434 (+ 33us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":25}
I20260923 22:13:13.605715 10982 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 22:13:13.600715 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.135.254:43715 (local address 127.1.135.193:43039)
0923 22:13:13.600904 (+ 189us) negotiation.cc:107] Waiting for socket to connect
0923 22:13:13.600948 (+ 44us) client_negotiation.cc:174] Beginning negotiation
0923 22:13:13.601579 (+ 631us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0923 22:13:13.601837 (+ 258us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0923 22:13:13.601841 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0923 22:13:13.601851 (+ 10us) client_negotiation.cc:189] Negotiated authn=SASL
0923 22:13:13.602515 (+ 664us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 22:13:13.602523 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 22:13:13.603168 (+ 645us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 22:13:13.603169 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 22:13:13.603960 (+ 791us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 22:13:13.603962 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 22:13:13.604382 (+ 420us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 22:13:13.604383 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 22:13:13.604514 (+ 131us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 22:13:13.605072 (+ 558us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0923 22:13:13.605073 (+ 1us) client_negotiation.cc:656] calling sasl_client_start()
0923 22:13:13.605099 (+ 26us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0923 22:13:13.605122 (+ 23us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0923 22:13:13.605311 (+ 189us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0923 22:13:13.605317 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0923 22:13:13.605320 (+ 3us) client_negotiation.cc:769] Sending connection context
0923 22:13:13.605403 (+ 83us) client_negotiation.cc:240] Negotiation successful
0923 22:13:13.605497 (+ 94us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":116,"thread_start_us":91,"threads_started":1}
I20260923 22:13:13.606179 10980 heartbeater.cc:347] Connected to a master server at 127.1.135.254:43715
I20260923 22:13:13.606278 10980 heartbeater.cc:464] Registering TS with master...
I20260923 22:13:13.606477 10980 heartbeater.cc:511] Master 127.1.135.254:43715 requested a full tablet report, sending...
I20260923 22:13:13.606980 10792 ts_manager.cc:194] Registered new tserver with Master: cd854351062b4cd3b0d3dfe8a6597e57 (127.1.135.193:44401)
I20260923 22:13:13.607596 10792 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.1.135.193:43039
W20260923 22:13:13.680169 10983 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260923 22:13:13.680346 10983 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260923 22:13:13.680367 10983 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260923 22:13:13.681465 10983 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260923 22:13:13.681502 10983 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260923 22:13:13.681548 10983 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.1.135.194
I20260923 22:13:13.682862 10983 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.1.135.212:40669
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskRiQe_a/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790201539513534-1567-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskRiQe_a/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790201539513534-1567-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.1.135.194:0
--rpc_server_allow_ephemeral_ports=true
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskRiQe_a/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790201539513534-1567-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.1.135.194
--webserver_port=0
--tserver_master_addrs=127.1.135.254:43715
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.10983
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.1.135.194
--log_dir=/tmp/dist-test-taskRiQe_a/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790201539513534-1567-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 23 Sep 2026 20:53:37 UTC on ccycloud.quasar-rkviyx.root.comops.site
build id 4668083
I20260923 22:13:13.683044 10983 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260923 22:13:13.683347 10983 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260923 22:13:13.687935 10988 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260923 22:13:13.687937 10991 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260923 22:13:13.688099 10983 server_base.cc:1034] running on GCE node
W20260923 22:13:13.688165 10989 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260923 22:13:13.688330 10983 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260923 22:13:13.688505 10983 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260923 22:13:13.689673 10983 hybrid_clock.cc:648] HybridClock initialized: now 1790201593689655 us; error 32 us; skew 500 ppm
I20260923 22:13:13.690758 10983 webserver.cc:460] Webserver started at http://127.1.135.194:42257/ using document root <none> and password file <none>
I20260923 22:13:13.690973 10983 fs_manager.cc:359] Metadata directory not provided
I20260923 22:13:13.691015 10983 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260923 22:13:13.691182 10983 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260923 22:13:13.692183 10983 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskRiQe_a/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790201539513534-1567-0/minicluster-data/ts-1/data/instance:
uuid: "a738a0ed43fb4ff0a82adf2160c0cf46"
format_stamp: "Formatted at 2026-09-23 22:13:13 on dist-test-slave-s2fj"
server_key: "26ac6d1e8b62af5b13092e04334fd2de"
server_key_iv: "e801f46a42ec4de5d05684261416050e"
server_key_version: "encryptionkey@0"
I20260923 22:13:13.692502 10983 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskRiQe_a/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790201539513534-1567-0/minicluster-data/ts-1/wal/instance:
uuid: "a738a0ed43fb4ff0a82adf2160c0cf46"
format_stamp: "Formatted at 2026-09-23 22:13:13 on dist-test-slave-s2fj"
server_key: "26ac6d1e8b62af5b13092e04334fd2de"
server_key_iv: "e801f46a42ec4de5d05684261416050e"
server_key_version: "encryptionkey@0"
I20260923 22:13:13.693769 10983 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.001s
I20260923 22:13:13.694394 10997 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260923 22:13:13.694526 10983 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.001s
I20260923 22:13:13.694607 10983 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskRiQe_a/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790201539513534-1567-0/minicluster-data/ts-1/data,/tmp/dist-test-taskRiQe_a/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790201539513534-1567-0/minicluster-data/ts-1/wal
uuid: "a738a0ed43fb4ff0a82adf2160c0cf46"
format_stamp: "Formatted at 2026-09-23 22:13:13 on dist-test-slave-s2fj"
server_key: "26ac6d1e8b62af5b13092e04334fd2de"
server_key_iv: "e801f46a42ec4de5d05684261416050e"
server_key_version: "encryptionkey@0"
I20260923 22:13:13.694653 10983 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskRiQe_a/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790201539513534-1567-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskRiQe_a/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790201539513534-1567-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskRiQe_a/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790201539513534-1567-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260923 22:13:13.722574 10983 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260923 22:13:13.722807 10983 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260923 22:13:13.722904 10983 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260923 22:13:13.723058 10983 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260923 22:13:13.723381 10983 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260923 22:13:13.723418 10983 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260923 22:13:13.723477 10983 ts_tablet_manager.cc:610] Registered 0 tablets
I20260923 22:13:13.723505 10983 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260923 22:13:13.729250 10983 rpc_server.cc:307] RPC server started. Bound to: 127.1.135.194:40203
I20260923 22:13:13.729305 11110 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.1.135.194:40203 every 8 connection(s)
I20260923 22:13:13.729647 10983 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskRiQe_a/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790201539513534-1567-0/minicluster-data/ts-1/data/info.pb
I20260923 22:13:13.734248 10831 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 22:13:13.730328 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.135.194:53489 (local address 127.1.135.254:43715)
0923 22:13:13.730378 (+ 50us) server_negotiation.cc:206] Beginning negotiation
0923 22:13:13.730381 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0923 22:13:13.730617 (+ 236us) server_negotiation.cc:407] Connection header received
0923 22:13:13.731037 (+ 420us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 22:13:13.731038 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 22:13:13.731056 (+ 18us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 22:13:13.731107 (+ 51us) server_negotiation.cc:226] Negotiated authn=SASL
0923 22:13:13.731668 (+ 561us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 22:13:13.732147 (+ 479us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 22:13:13.733054 (+ 907us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 22:13:13.733226 (+ 172us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 22:13:13.733275 (+ 49us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 22:13:13.734059 (+ 784us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0923 22:13:13.734060 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0923 22:13:13.734060 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0923 22:13:13.734064 (+ 4us) server_negotiation.cc:922] calling sasl_server_start()
0923 22:13:13.734073 (+ 9us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0923 22:13:13.734076 (+ 3us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0923 22:13:13.734112 (+ 36us) server_negotiation.cc:1035] Waiting for connection context
0923 22:13:13.734173 (+ 61us) server_negotiation.cc:299] Negotiation successful
0923 22:13:13.734193 (+ 20us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":23}
I20260923 22:13:13.734421 11113 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 22:13:13.730321 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.135.254:43715 (local address 127.1.135.194:53489)
0923 22:13:13.730488 (+ 167us) negotiation.cc:107] Waiting for socket to connect
0923 22:13:13.730519 (+ 31us) client_negotiation.cc:174] Beginning negotiation
0923 22:13:13.730962 (+ 443us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0923 22:13:13.731159 (+ 197us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0923 22:13:13.731162 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0923 22:13:13.731169 (+ 7us) client_negotiation.cc:189] Negotiated authn=SASL
0923 22:13:13.731605 (+ 436us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 22:13:13.731609 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 22:13:13.732224 (+ 615us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 22:13:13.732225 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 22:13:13.732977 (+ 752us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 22:13:13.732979 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 22:13:13.733293 (+ 314us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 22:13:13.733294 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 22:13:13.733394 (+ 100us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 22:13:13.733940 (+ 546us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0923 22:13:13.733940 (+ 0us) client_negotiation.cc:656] calling sasl_client_start()
0923 22:13:13.733966 (+ 26us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0923 22:13:13.733988 (+ 22us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0923 22:13:13.734114 (+ 126us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0923 22:13:13.734116 (+ 2us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0923 22:13:13.734119 (+ 3us) client_negotiation.cc:769] Sending connection context
0923 22:13:13.734162 (+ 43us) client_negotiation.cc:240] Negotiation successful
0923 22:13:13.734232 (+ 70us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":110,"thread_start_us":84,"threads_started":1}
I20260923 22:13:13.734895 11111 heartbeater.cc:347] Connected to a master server at 127.1.135.254:43715
I20260923 22:13:13.734972 11111 heartbeater.cc:464] Registering TS with master...
I20260923 22:13:13.735191 11111 heartbeater.cc:511] Master 127.1.135.254:43715 requested a full tablet report, sending...
I20260923 22:13:13.735551 10792 ts_manager.cc:194] Registered new tserver with Master: a738a0ed43fb4ff0a82adf2160c0cf46 (127.1.135.194:40203)
I20260923 22:13:13.735857 10792 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.1.135.194:53489
I20260923 22:13:13.737100 1567 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskRiQe_a/build/release/bin/kudu as pid 10983
I20260923 22:13:13.737200 1567 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskRiQe_a/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790201539513534-1567-0/minicluster-data/ts-1/wal/instance
I20260923 22:13:13.737377 1567 external_mini_cluster.cc:1428] Setting key 0c864734a14885713923042e1965f8f4
I20260923 22:13:13.738180 1567 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskRiQe_a/build/release/bin/kudu
/tmp/dist-test-taskRiQe_a/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskRiQe_a/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790201539513534-1567-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskRiQe_a/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790201539513534-1567-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskRiQe_a/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790201539513534-1567-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskRiQe_a/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790201539513534-1567-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.1.135.195:0
--local_ip_for_outbound_sockets=127.1.135.195
--webserver_interface=127.1.135.195
--webserver_port=0
--tserver_master_addrs=127.1.135.254:43715
--builtin_ntp_servers=127.1.135.212:40669
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {}
W20260923 22:13:13.815801 11114 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260923 22:13:13.815975 11114 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260923 22:13:13.815995 11114 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260923 22:13:13.817137 11114 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260923 22:13:13.817174 11114 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260923 22:13:13.817198 11114 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.1.135.195
I20260923 22:13:13.818445 11114 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.1.135.212:40669
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskRiQe_a/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790201539513534-1567-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskRiQe_a/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790201539513534-1567-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.1.135.195:0
--rpc_server_allow_ephemeral_ports=true
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskRiQe_a/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790201539513534-1567-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.1.135.195
--webserver_port=0
--tserver_master_addrs=127.1.135.254:43715
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.11114
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.1.135.195
--log_dir=/tmp/dist-test-taskRiQe_a/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790201539513534-1567-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 23 Sep 2026 20:53:37 UTC on ccycloud.quasar-rkviyx.root.comops.site
build id 4668083
I20260923 22:13:13.818671 11114 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260923 22:13:13.818933 11114 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260923 22:13:13.823557 11120 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260923 22:13:13.823585 11119 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260923 22:13:13.823613 11114 server_base.cc:1034] running on GCE node
W20260923 22:13:13.823576 11122 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260923 22:13:13.823899 11114 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260923 22:13:13.824229 11114 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260923 22:13:13.825385 11114 hybrid_clock.cc:648] HybridClock initialized: now 1790201593825378 us; error 38 us; skew 500 ppm
I20260923 22:13:13.826388 11114 webserver.cc:460] Webserver started at http://127.1.135.195:41675/ using document root <none> and password file <none>
I20260923 22:13:13.826598 11114 fs_manager.cc:359] Metadata directory not provided
I20260923 22:13:13.826658 11114 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260923 22:13:13.826792 11114 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260923 22:13:13.827901 11114 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskRiQe_a/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790201539513534-1567-0/minicluster-data/ts-2/data/instance:
uuid: "5a4c9f58ceac4d08bca8d506e99b74e7"
format_stamp: "Formatted at 2026-09-23 22:13:13 on dist-test-slave-s2fj"
server_key: "f96b5a93869125cc94cf3e7e14d3ceeb"
server_key_iv: "2b6a2ea876b08d853ec5c9f80f7d0661"
server_key_version: "encryptionkey@0"
I20260923 22:13:13.828279 11114 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskRiQe_a/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790201539513534-1567-0/minicluster-data/ts-2/wal/instance:
uuid: "5a4c9f58ceac4d08bca8d506e99b74e7"
format_stamp: "Formatted at 2026-09-23 22:13:13 on dist-test-slave-s2fj"
server_key: "f96b5a93869125cc94cf3e7e14d3ceeb"
server_key_iv: "2b6a2ea876b08d853ec5c9f80f7d0661"
server_key_version: "encryptionkey@0"
I20260923 22:13:13.829604 11114 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.001s
I20260923 22:13:13.830262 11128 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260923 22:13:13.830435 11114 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.001s
I20260923 22:13:13.830525 11114 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskRiQe_a/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790201539513534-1567-0/minicluster-data/ts-2/data,/tmp/dist-test-taskRiQe_a/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790201539513534-1567-0/minicluster-data/ts-2/wal
uuid: "5a4c9f58ceac4d08bca8d506e99b74e7"
format_stamp: "Formatted at 2026-09-23 22:13:13 on dist-test-slave-s2fj"
server_key: "f96b5a93869125cc94cf3e7e14d3ceeb"
server_key_iv: "2b6a2ea876b08d853ec5c9f80f7d0661"
server_key_version: "encryptionkey@0"
I20260923 22:13:13.830608 11114 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskRiQe_a/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790201539513534-1567-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskRiQe_a/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790201539513534-1567-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskRiQe_a/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790201539513534-1567-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260923 22:13:13.853123 11114 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260923 22:13:13.853410 11114 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260923 22:13:13.853559 11114 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260923 22:13:13.853760 11114 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260923 22:13:13.854075 11114 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260923 22:13:13.854128 11114 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260923 22:13:13.854163 11114 ts_tablet_manager.cc:610] Registered 0 tablets
I20260923 22:13:13.854202 11114 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260923 22:13:13.859652 11114 rpc_server.cc:307] RPC server started. Bound to: 127.1.135.195:40881
I20260923 22:13:13.859696 11241 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.1.135.195:40881 every 8 connection(s)
I20260923 22:13:13.859993 11114 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskRiQe_a/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790201539513534-1567-0/minicluster-data/ts-2/data/info.pb
I20260923 22:13:13.862927 1567 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskRiQe_a/build/release/bin/kudu as pid 11114
I20260923 22:13:13.863019 1567 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskRiQe_a/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790201539513534-1567-0/minicluster-data/ts-2/wal/instance
I20260923 22:13:13.863214 1567 external_mini_cluster.cc:1428] Setting key d34170b9acbb0fe6bee514543ef9e4c1
I20260923 22:13:13.865024 10831 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 22:13:13.860686 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.135.195:37983 (local address 127.1.135.254:43715)
0923 22:13:13.860725 (+ 39us) server_negotiation.cc:206] Beginning negotiation
0923 22:13:13.860727 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0923 22:13:13.860982 (+ 255us) server_negotiation.cc:407] Connection header received
0923 22:13:13.861420 (+ 438us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 22:13:13.861421 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 22:13:13.861430 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 22:13:13.861480 (+ 50us) server_negotiation.cc:226] Negotiated authn=SASL
0923 22:13:13.862056 (+ 576us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 22:13:13.862595 (+ 539us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 22:13:13.863785 (+ 1190us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 22:13:13.864093 (+ 308us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 22:13:13.864155 (+ 62us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 22:13:13.864837 (+ 682us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0923 22:13:13.864838 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0923 22:13:13.864838 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0923 22:13:13.864842 (+ 4us) server_negotiation.cc:922] calling sasl_server_start()
0923 22:13:13.864855 (+ 13us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0923 22:13:13.864859 (+ 4us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0923 22:13:13.864904 (+ 45us) server_negotiation.cc:1035] Waiting for connection context
0923 22:13:13.864953 (+ 49us) server_negotiation.cc:299] Negotiation successful
0923 22:13:13.864971 (+ 18us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":17}
I20260923 22:13:13.865280 11244 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 22:13:13.860685 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.135.254:43715 (local address 127.1.135.195:37983)
0923 22:13:13.860849 (+ 164us) negotiation.cc:107] Waiting for socket to connect
0923 22:13:13.860898 (+ 49us) client_negotiation.cc:174] Beginning negotiation
0923 22:13:13.861340 (+ 442us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0923 22:13:13.861531 (+ 191us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0923 22:13:13.861534 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0923 22:13:13.861541 (+ 7us) client_negotiation.cc:189] Negotiated authn=SASL
0923 22:13:13.861980 (+ 439us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 22:13:13.861983 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 22:13:13.862664 (+ 681us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 22:13:13.862665 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 22:13:13.863713 (+ 1048us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 22:13:13.863716 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 22:13:13.864163 (+ 447us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 22:13:13.864164 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 22:13:13.864307 (+ 143us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 22:13:13.864721 (+ 414us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0923 22:13:13.864722 (+ 1us) client_negotiation.cc:656] calling sasl_client_start()
0923 22:13:13.864746 (+ 24us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0923 22:13:13.864766 (+ 20us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0923 22:13:13.864907 (+ 141us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0923 22:13:13.864909 (+ 2us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0923 22:13:13.864911 (+ 2us) client_negotiation.cc:769] Sending connection context
0923 22:13:13.864955 (+ 44us) client_negotiation.cc:240] Negotiation successful
0923 22:13:13.865023 (+ 68us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":110,"thread_start_us":87,"threads_started":1}
I20260923 22:13:13.865733 11242 heartbeater.cc:347] Connected to a master server at 127.1.135.254:43715
I20260923 22:13:13.865816 11242 heartbeater.cc:464] Registering TS with master...
I20260923 22:13:13.865993 11242 heartbeater.cc:511] Master 127.1.135.254:43715 requested a full tablet report, sending...
I20260923 22:13:13.866320 10792 ts_manager.cc:194] Registered new tserver with Master: 5a4c9f58ceac4d08bca8d506e99b74e7 (127.1.135.195:40881)
I20260923 22:13:13.866694 10792 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.1.135.195:37983
I20260923 22:13:13.875187 1567 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
W20260923 22:13:13.877079 10831 negotiation.cc:337] Failed RPC negotiation. Trace:
0923 22:13:13.876519 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60162 (local address 127.1.135.254:43715)
0923 22:13:13.876601 (+ 82us) server_negotiation.cc:206] Beginning negotiation
0923 22:13:13.876603 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0923 22:13:13.876695 (+ 92us) server_negotiation.cc:407] Connection header received
0923 22:13:13.876800 (+ 105us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 22:13:13.876801 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 22:13:13.876810 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 22:13:13.876878 (+ 68us) server_negotiation.cc:226] Negotiated authn=SASL
0923 22:13:13.877022 (+ 144us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:60162: BlockingRecv error: recv got EOF from 127.0.0.1:60162 (error 108)
Metrics: {"server-negotiator.queue_time_us":50}
I20260923 22:13:13.877151 1567 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskRiQe_a/build/release/bin/kudu with pid 10852
I20260923 22:13:13.881745 1567 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskRiQe_a/build/release/bin/kudu with pid 10983
I20260923 22:13:13.885941 1567 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskRiQe_a/build/release/bin/kudu with pid 11114
I20260923 22:13:13.889981 1567 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskRiQe_a/build/release/bin/kudu with pid 10761
2026-09-23T22:13:13Z chronyd exiting
[ OK ] SecurityITest.TestRequireAuthenticationInsecureCluster (621 ms)
[ RUN ] SecurityITest.TestRequireEncryptionInsecureCluster
2026-09-23T22:13:13Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-09-23T22:13:13Z Disabled control of system clock
I20260923 22:13:13.915541 1567 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskRiQe_a/build/release/bin/kudu
/tmp/dist-test-taskRiQe_a/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskRiQe_a/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790201539513534-1567-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskRiQe_a/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790201539513534-1567-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskRiQe_a/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790201539513534-1567-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskRiQe_a/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790201539513534-1567-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.1.135.254:38971
--webserver_interface=127.1.135.254
--webserver_port=0
--builtin_ntp_servers=127.1.135.212:39981
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.1.135.254:38971
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--rpc_encryption=disabled
--rpc_authentication=disabled with env {}
W20260923 22:13:13.993394 11253 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260923 22:13:13.993585 11253 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260923 22:13:13.993604 11253 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260923 22:13:13.994611 11253 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260923 22:13:13.994642 11253 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260923 22:13:13.994654 11253 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260923 22:13:13.994666 11253 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260923 22:13:13.994678 11253 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260923 22:13:13.995858 11253 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.1.135.212:39981
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskRiQe_a/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790201539513534-1567-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskRiQe_a/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790201539513534-1567-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.1.135.254:38971
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.1.135.254:38971
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskRiQe_a/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790201539513534-1567-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.1.135.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.11253
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskRiQe_a/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790201539513534-1567-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 23 Sep 2026 20:53:37 UTC on ccycloud.quasar-rkviyx.root.comops.site
build id 4668083
I20260923 22:13:13.996032 11253 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260923 22:13:13.996308 11253 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260923 22:13:14.000955 11261 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260923 22:13:14.001016 11258 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260923 22:13:14.001024 11259 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260923 22:13:14.001133 11253 server_base.cc:1034] running on GCE node
I20260923 22:13:14.001392 11253 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260923 22:13:14.001689 11253 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260923 22:13:14.002851 11253 hybrid_clock.cc:648] HybridClock initialized: now 1790201594002829 us; error 33 us; skew 500 ppm
I20260923 22:13:14.004154 11253 webserver.cc:460] Webserver started at http://127.1.135.254:34819/ using document root <none> and password file <none>
I20260923 22:13:14.004390 11253 fs_manager.cc:359] Metadata directory not provided
I20260923 22:13:14.004474 11253 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260923 22:13:14.004630 11253 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260923 22:13:14.005602 11253 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskRiQe_a/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790201539513534-1567-0/minicluster-data/master-0/data/instance:
uuid: "d1e69baad6ef460da643d860559fdf1d"
format_stamp: "Formatted at 2026-09-23 22:13:14 on dist-test-slave-s2fj"
server_key: "c078580e2a268168f84a8f3893c1e26b"
server_key_iv: "42dcd9eccfe7d10a98f5eb474e89a65c"
server_key_version: "encryptionkey@0"
I20260923 22:13:14.005923 11253 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskRiQe_a/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790201539513534-1567-0/minicluster-data/master-0/wal/instance:
uuid: "d1e69baad6ef460da643d860559fdf1d"
format_stamp: "Formatted at 2026-09-23 22:13:14 on dist-test-slave-s2fj"
server_key: "c078580e2a268168f84a8f3893c1e26b"
server_key_iv: "42dcd9eccfe7d10a98f5eb474e89a65c"
server_key_version: "encryptionkey@0"
I20260923 22:13:14.007088 11253 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.003s
I20260923 22:13:14.007730 11267 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260923 22:13:14.007876 11253 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.001s
I20260923 22:13:14.007961 11253 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskRiQe_a/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790201539513534-1567-0/minicluster-data/master-0/data,/tmp/dist-test-taskRiQe_a/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790201539513534-1567-0/minicluster-data/master-0/wal
uuid: "d1e69baad6ef460da643d860559fdf1d"
format_stamp: "Formatted at 2026-09-23 22:13:14 on dist-test-slave-s2fj"
server_key: "c078580e2a268168f84a8f3893c1e26b"
server_key_iv: "42dcd9eccfe7d10a98f5eb474e89a65c"
server_key_version: "encryptionkey@0"
I20260923 22:13:14.008042 11253 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskRiQe_a/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790201539513534-1567-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskRiQe_a/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790201539513534-1567-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskRiQe_a/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790201539513534-1567-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260923 22:13:14.011235 11253 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260923 22:13:14.011482 11253 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260923 22:13:14.011613 11253 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260923 22:13:14.015254 11253 rpc_server.cc:307] RPC server started. Bound to: 127.1.135.254:38971
I20260923 22:13:14.015280 11319 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.1.135.254:38971 every 8 connection(s)
I20260923 22:13:14.015580 11253 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskRiQe_a/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790201539513534-1567-0/minicluster-data/master-0/data/info.pb
I20260923 22:13:14.016258 11320 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260923 22:13:14.018734 11320 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P d1e69baad6ef460da643d860559fdf1d: Bootstrap starting.
I20260923 22:13:14.019322 11320 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P d1e69baad6ef460da643d860559fdf1d: Neither blocks nor log segments found. Creating new log.
I20260923 22:13:14.019317 1567 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskRiQe_a/build/release/bin/kudu as pid 11253
I20260923 22:13:14.019402 1567 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskRiQe_a/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790201539513534-1567-0/minicluster-data/master-0/wal/instance
I20260923 22:13:14.019639 1567 external_mini_cluster.cc:1428] Setting key ea527224000cab42d260a512b9ebc841
I20260923 22:13:14.019694 11320 log.cc:824] T 00000000000000000000000000000000 P d1e69baad6ef460da643d860559fdf1d: Log is configured to *not* fsync() on all Append() calls
I20260923 22:13:14.020463 11320 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P d1e69baad6ef460da643d860559fdf1d: No bootstrap required, opened a new log
I20260923 22:13:14.022037 11324 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 22:13:14.020133 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44880 (local address 127.1.135.254:38971)
0923 22:13:14.020338 (+ 205us) server_negotiation.cc:206] Beginning negotiation
0923 22:13:14.020346 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0923 22:13:14.020381 (+ 35us) server_negotiation.cc:407] Connection header received
0923 22:13:14.020819 (+ 438us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 22:13:14.020825 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 22:13:14.020862 (+ 37us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 22:13:14.021007 (+ 145us) server_negotiation.cc:226] Negotiated authn=SASL
0923 22:13:14.021601 (+ 594us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0923 22:13:14.021604 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0923 22:13:14.021604 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0923 22:13:14.021608 (+ 4us) server_negotiation.cc:922] calling sasl_server_start()
0923 22:13:14.021648 (+ 40us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0923 22:13:14.021662 (+ 14us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0923 22:13:14.021741 (+ 79us) server_negotiation.cc:1035] Waiting for connection context
0923 22:13:14.021818 (+ 77us) server_negotiation.cc:299] Negotiation successful
0923 22:13:14.021869 (+ 51us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":132,"thread_start_us":73,"threads_started":1}
I20260923 22:13:14.022226 11320 raft_consensus.cc:348] T 00000000000000000000000000000000 P d1e69baad6ef460da643d860559fdf1d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d1e69baad6ef460da643d860559fdf1d" member_type: VOTER last_known_addr { host: "127.1.135.254" port: 38971 } }
I20260923 22:13:14.022387 11320 raft_consensus.cc:374] T 00000000000000000000000000000000 P d1e69baad6ef460da643d860559fdf1d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260923 22:13:14.022421 11320 raft_consensus.cc:729] T 00000000000000000000000000000000 P d1e69baad6ef460da643d860559fdf1d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d1e69baad6ef460da643d860559fdf1d, State: Initialized, Role: FOLLOWER
I20260923 22:13:14.022547 11320 consensus_queue.cc:260] T 00000000000000000000000000000000 P d1e69baad6ef460da643d860559fdf1d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d1e69baad6ef460da643d860559fdf1d" member_type: VOTER last_known_addr { host: "127.1.135.254" port: 38971 } }
I20260923 22:13:14.022655 11320 raft_consensus.cc:388] T 00000000000000000000000000000000 P d1e69baad6ef460da643d860559fdf1d [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260923 22:13:14.022697 11320 raft_consensus.cc:482] T 00000000000000000000000000000000 P d1e69baad6ef460da643d860559fdf1d [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260923 22:13:14.022747 11320 raft_consensus.cc:3037] T 00000000000000000000000000000000 P d1e69baad6ef460da643d860559fdf1d [term 0 FOLLOWER]: Advancing to term 1
I20260923 22:13:14.023439 11320 raft_consensus.cc:504] T 00000000000000000000000000000000 P d1e69baad6ef460da643d860559fdf1d [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d1e69baad6ef460da643d860559fdf1d" member_type: VOTER last_known_addr { host: "127.1.135.254" port: 38971 } }
I20260923 22:13:14.023546 11320 leader_election.cc:302] T 00000000000000000000000000000000 P d1e69baad6ef460da643d860559fdf1d [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: d1e69baad6ef460da643d860559fdf1d; no voters:
I20260923 22:13:14.023685 11320 leader_election.cc:288] T 00000000000000000000000000000000 P d1e69baad6ef460da643d860559fdf1d [CANDIDATE]: Term 1 election: Requested vote from peers
I20260923 22:13:14.023713 11325 raft_consensus.cc:2781] T 00000000000000000000000000000000 P d1e69baad6ef460da643d860559fdf1d [term 1 FOLLOWER]: Leader election won for term 1
I20260923 22:13:14.023855 11325 raft_consensus.cc:686] T 00000000000000000000000000000000 P d1e69baad6ef460da643d860559fdf1d [term 1 LEADER]: Becoming Leader. State: Replica: d1e69baad6ef460da643d860559fdf1d, State: Running, Role: LEADER
I20260923 22:13:14.023945 11320 sys_catalog.cc:564] T 00000000000000000000000000000000 P d1e69baad6ef460da643d860559fdf1d [sys.catalog]: configured and running, proceeding with master startup.
I20260923 22:13:14.023957 11325 consensus_queue.cc:237] T 00000000000000000000000000000000 P d1e69baad6ef460da643d860559fdf1d [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d1e69baad6ef460da643d860559fdf1d" member_type: VOTER last_known_addr { host: "127.1.135.254" port: 38971 } }
I20260923 22:13:14.024439 11326 sys_catalog.cc:455] T 00000000000000000000000000000000 P d1e69baad6ef460da643d860559fdf1d [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "d1e69baad6ef460da643d860559fdf1d" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d1e69baad6ef460da643d860559fdf1d" member_type: VOTER last_known_addr { host: "127.1.135.254" port: 38971 } } }
I20260923 22:13:14.024461 11327 sys_catalog.cc:455] T 00000000000000000000000000000000 P d1e69baad6ef460da643d860559fdf1d [sys.catalog]: SysCatalogTable state changed. Reason: New leader d1e69baad6ef460da643d860559fdf1d. Latest consensus state: current_term: 1 leader_uuid: "d1e69baad6ef460da643d860559fdf1d" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d1e69baad6ef460da643d860559fdf1d" member_type: VOTER last_known_addr { host: "127.1.135.254" port: 38971 } } }
I20260923 22:13:14.024629 11326 sys_catalog.cc:458] T 00000000000000000000000000000000 P d1e69baad6ef460da643d860559fdf1d [sys.catalog]: This master's current role is: LEADER
I20260923 22:13:14.024646 11327 sys_catalog.cc:458] T 00000000000000000000000000000000 P d1e69baad6ef460da643d860559fdf1d [sys.catalog]: This master's current role is: LEADER
I20260923 22:13:14.025207 11334 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260923 22:13:14.025595 11334 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260923 22:13:14.026912 11334 catalog_manager.cc:1347] Generated new cluster ID: d816ca4e234b4e53bfd103ecbf52dcad
I20260923 22:13:14.026959 11334 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260923 22:13:14.035959 11334 catalog_manager.cc:1370] Generated new certificate authority record
I20260923 22:13:14.036139 11334 catalog_manager.cc:1504] Loading token signing keys...
I20260923 22:13:14.043440 11334 catalog_manager.cc:5868] T 00000000000000000000000000000000 P d1e69baad6ef460da643d860559fdf1d: Generated new TSK 0
I20260923 22:13:14.043640 11334 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260923 22:13:14.045496 1567 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskRiQe_a/build/release/bin/kudu
/tmp/dist-test-taskRiQe_a/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskRiQe_a/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790201539513534-1567-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskRiQe_a/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790201539513534-1567-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskRiQe_a/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790201539513534-1567-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskRiQe_a/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790201539513534-1567-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.1.135.193:0
--local_ip_for_outbound_sockets=127.1.135.193
--webserver_interface=127.1.135.193
--webserver_port=0
--tserver_master_addrs=127.1.135.254:38971
--builtin_ntp_servers=127.1.135.212:39981
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_encryption=disabled
--rpc_authentication=disabled with env {}
W20260923 22:13:14.125561 11344 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260923 22:13:14.125765 11344 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260923 22:13:14.125799 11344 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260923 22:13:14.126969 11344 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260923 22:13:14.127008 11344 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260923 22:13:14.127032 11344 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.1.135.193
I20260923 22:13:14.128355 11344 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.1.135.212:39981
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskRiQe_a/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790201539513534-1567-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskRiQe_a/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790201539513534-1567-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.1.135.193:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskRiQe_a/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790201539513534-1567-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.1.135.193
--webserver_port=0
--tserver_master_addrs=127.1.135.254:38971
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.11344
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.1.135.193
--log_dir=/tmp/dist-test-taskRiQe_a/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790201539513534-1567-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 23 Sep 2026 20:53:37 UTC on ccycloud.quasar-rkviyx.root.comops.site
build id 4668083
I20260923 22:13:14.128585 11344 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260923 22:13:14.128870 11344 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260923 22:13:14.134372 11350 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260923 22:13:14.134378 11349 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260923 22:13:14.134547 11352 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260923 22:13:14.134469 11344 server_base.cc:1034] running on GCE node
I20260923 22:13:14.134796 11344 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260923 22:13:14.135054 11344 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260923 22:13:14.136205 11344 hybrid_clock.cc:648] HybridClock initialized: now 1790201594136192 us; error 30 us; skew 500 ppm
I20260923 22:13:14.137485 11344 webserver.cc:460] Webserver started at http://127.1.135.193:34115/ using document root <none> and password file <none>
I20260923 22:13:14.137732 11344 fs_manager.cc:359] Metadata directory not provided
I20260923 22:13:14.137776 11344 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260923 22:13:14.137915 11344 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260923 22:13:14.138911 11344 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskRiQe_a/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790201539513534-1567-0/minicluster-data/ts-0/data/instance:
uuid: "8795f6f4473041658c645c6da61bdeab"
format_stamp: "Formatted at 2026-09-23 22:13:14 on dist-test-slave-s2fj"
server_key: "5646c55c485a58b7d5ba3e8aafadd2f2"
server_key_iv: "8fb82dce49f66a8d50fe4bee1a7c7ce9"
server_key_version: "encryptionkey@0"
I20260923 22:13:14.139251 11344 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskRiQe_a/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790201539513534-1567-0/minicluster-data/ts-0/wal/instance:
uuid: "8795f6f4473041658c645c6da61bdeab"
format_stamp: "Formatted at 2026-09-23 22:13:14 on dist-test-slave-s2fj"
server_key: "5646c55c485a58b7d5ba3e8aafadd2f2"
server_key_iv: "8fb82dce49f66a8d50fe4bee1a7c7ce9"
server_key_version: "encryptionkey@0"
I20260923 22:13:14.140676 11344 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.003s
I20260923 22:13:14.141453 11358 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260923 22:13:14.141597 11344 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.001s
I20260923 22:13:14.141695 11344 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskRiQe_a/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790201539513534-1567-0/minicluster-data/ts-0/data,/tmp/dist-test-taskRiQe_a/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790201539513534-1567-0/minicluster-data/ts-0/wal
uuid: "8795f6f4473041658c645c6da61bdeab"
format_stamp: "Formatted at 2026-09-23 22:13:14 on dist-test-slave-s2fj"
server_key: "5646c55c485a58b7d5ba3e8aafadd2f2"
server_key_iv: "8fb82dce49f66a8d50fe4bee1a7c7ce9"
server_key_version: "encryptionkey@0"
I20260923 22:13:14.141760 11344 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskRiQe_a/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790201539513534-1567-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskRiQe_a/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790201539513534-1567-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskRiQe_a/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790201539513534-1567-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260923 22:13:14.145025 11344 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260923 22:13:14.145289 11344 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260923 22:13:14.145413 11344 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260923 22:13:14.145644 11344 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260923 22:13:14.145979 11344 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260923 22:13:14.146014 11344 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260923 22:13:14.146034 11344 ts_tablet_manager.cc:610] Registered 0 tablets
I20260923 22:13:14.146046 11344 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260923 22:13:14.151636 11344 rpc_server.cc:307] RPC server started. Bound to: 127.1.135.193:45401
I20260923 22:13:14.151695 11471 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.1.135.193:45401 every 8 connection(s)
I20260923 22:13:14.151980 11344 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskRiQe_a/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790201539513534-1567-0/minicluster-data/ts-0/data/info.pb
I20260923 22:13:14.154533 11324 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 22:13:14.152669 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.135.193:51029 (local address 127.1.135.254:38971)
0923 22:13:14.152744 (+ 75us) server_negotiation.cc:206] Beginning negotiation
0923 22:13:14.152746 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0923 22:13:14.152945 (+ 199us) server_negotiation.cc:407] Connection header received
0923 22:13:14.153325 (+ 380us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 22:13:14.153326 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 22:13:14.153330 (+ 4us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 22:13:14.153380 (+ 50us) server_negotiation.cc:226] Negotiated authn=SASL
0923 22:13:14.154354 (+ 974us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0923 22:13:14.154355 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0923 22:13:14.154355 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0923 22:13:14.154359 (+ 4us) server_negotiation.cc:922] calling sasl_server_start()
0923 22:13:14.154371 (+ 12us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0923 22:13:14.154375 (+ 4us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0923 22:13:14.154425 (+ 50us) server_negotiation.cc:1035] Waiting for connection context
0923 22:13:14.154472 (+ 47us) server_negotiation.cc:299] Negotiation successful
0923 22:13:14.154482 (+ 10us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":45}
I20260923 22:13:14.154729 11474 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 22:13:14.152637 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.135.254:38971 (local address 127.1.135.193:51029)
0923 22:13:14.152821 (+ 184us) negotiation.cc:107] Waiting for socket to connect
0923 22:13:14.152852 (+ 31us) client_negotiation.cc:174] Beginning negotiation
0923 22:13:14.153251 (+ 399us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0923 22:13:14.153426 (+ 175us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0923 22:13:14.153428 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0923 22:13:14.153435 (+ 7us) client_negotiation.cc:189] Negotiated authn=SASL
0923 22:13:14.154231 (+ 796us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0923 22:13:14.154231 (+ 0us) client_negotiation.cc:656] calling sasl_client_start()
0923 22:13:14.154256 (+ 25us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0923 22:13:14.154279 (+ 23us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0923 22:13:14.154425 (+ 146us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0923 22:13:14.154427 (+ 2us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0923 22:13:14.154429 (+ 2us) client_negotiation.cc:769] Sending connection context
0923 22:13:14.154477 (+ 48us) client_negotiation.cc:240] Negotiation successful
0923 22:13:14.154519 (+ 42us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":132,"thread_start_us":111,"threads_started":1}
I20260923 22:13:14.155208 11472 heartbeater.cc:347] Connected to a master server at 127.1.135.254:38971
I20260923 22:13:14.155311 11472 heartbeater.cc:464] Registering TS with master...
I20260923 22:13:14.155467 11472 heartbeater.cc:511] Master 127.1.135.254:38971 requested a full tablet report, sending...
I20260923 22:13:14.156008 11284 ts_manager.cc:194] Registered new tserver with Master: 8795f6f4473041658c645c6da61bdeab (127.1.135.193:45401)
I20260923 22:13:14.159894 1567 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskRiQe_a/build/release/bin/kudu as pid 11344
I20260923 22:13:14.159973 1567 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskRiQe_a/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790201539513534-1567-0/minicluster-data/ts-0/wal/instance
I20260923 22:13:14.160204 1567 external_mini_cluster.cc:1428] Setting key 7c6cef766270729dff9014a08587f8d8
I20260923 22:13:14.161015 1567 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskRiQe_a/build/release/bin/kudu
/tmp/dist-test-taskRiQe_a/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskRiQe_a/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790201539513534-1567-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskRiQe_a/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790201539513534-1567-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskRiQe_a/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790201539513534-1567-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskRiQe_a/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790201539513534-1567-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.1.135.194:0
--local_ip_for_outbound_sockets=127.1.135.194
--webserver_interface=127.1.135.194
--webserver_port=0
--tserver_master_addrs=127.1.135.254:38971
--builtin_ntp_servers=127.1.135.212:39981
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_encryption=disabled
--rpc_authentication=disabled with env {}
W20260923 22:13:14.239972 11475 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260923 22:13:14.240281 11475 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260923 22:13:14.240319 11475 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260923 22:13:14.241436 11475 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260923 22:13:14.241503 11475 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260923 22:13:14.241542 11475 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.1.135.194
I20260923 22:13:14.242868 11475 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.1.135.212:39981
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskRiQe_a/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790201539513534-1567-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskRiQe_a/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790201539513534-1567-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.1.135.194:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskRiQe_a/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790201539513534-1567-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.1.135.194
--webserver_port=0
--tserver_master_addrs=127.1.135.254:38971
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.11475
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.1.135.194
--log_dir=/tmp/dist-test-taskRiQe_a/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790201539513534-1567-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 23 Sep 2026 20:53:37 UTC on ccycloud.quasar-rkviyx.root.comops.site
build id 4668083
I20260923 22:13:14.243144 11475 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260923 22:13:14.243413 11475 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260923 22:13:14.248301 11481 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260923 22:13:14.248338 11475 server_base.cc:1034] running on GCE node
W20260923 22:13:14.248364 11483 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260923 22:13:14.248307 11480 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260923 22:13:14.248616 11475 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260923 22:13:14.248800 11475 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260923 22:13:14.249933 11475 hybrid_clock.cc:648] HybridClock initialized: now 1790201594249914 us; error 35 us; skew 500 ppm
I20260923 22:13:14.251071 11475 webserver.cc:460] Webserver started at http://127.1.135.194:36927/ using document root <none> and password file <none>
I20260923 22:13:14.251255 11475 fs_manager.cc:359] Metadata directory not provided
I20260923 22:13:14.251305 11475 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260923 22:13:14.251386 11475 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260923 22:13:14.252424 11475 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskRiQe_a/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790201539513534-1567-0/minicluster-data/ts-1/data/instance:
uuid: "0339c9275dd64db991a0f9ad0922fb00"
format_stamp: "Formatted at 2026-09-23 22:13:14 on dist-test-slave-s2fj"
server_key: "96a2f6fea3146a3638129d4a05898c20"
server_key_iv: "8a57d9e17e31cd267b89f729912effa0"
server_key_version: "encryptionkey@0"
I20260923 22:13:14.252727 11475 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskRiQe_a/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790201539513534-1567-0/minicluster-data/ts-1/wal/instance:
uuid: "0339c9275dd64db991a0f9ad0922fb00"
format_stamp: "Formatted at 2026-09-23 22:13:14 on dist-test-slave-s2fj"
server_key: "96a2f6fea3146a3638129d4a05898c20"
server_key_iv: "8a57d9e17e31cd267b89f729912effa0"
server_key_version: "encryptionkey@0"
I20260923 22:13:14.253849 11475 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.000s
I20260923 22:13:14.254441 11489 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260923 22:13:14.254592 11475 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260923 22:13:14.254668 11475 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskRiQe_a/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790201539513534-1567-0/minicluster-data/ts-1/data,/tmp/dist-test-taskRiQe_a/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790201539513534-1567-0/minicluster-data/ts-1/wal
uuid: "0339c9275dd64db991a0f9ad0922fb00"
format_stamp: "Formatted at 2026-09-23 22:13:14 on dist-test-slave-s2fj"
server_key: "96a2f6fea3146a3638129d4a05898c20"
server_key_iv: "8a57d9e17e31cd267b89f729912effa0"
server_key_version: "encryptionkey@0"
I20260923 22:13:14.254717 11475 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskRiQe_a/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790201539513534-1567-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskRiQe_a/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790201539513534-1567-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskRiQe_a/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790201539513534-1567-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260923 22:13:14.257920 11475 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260923 22:13:14.258112 11475 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260923 22:13:14.258203 11475 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260923 22:13:14.258347 11475 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260923 22:13:14.258628 11475 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260923 22:13:14.258659 11475 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260923 22:13:14.258723 11475 ts_tablet_manager.cc:610] Registered 0 tablets
I20260923 22:13:14.258757 11475 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260923 22:13:14.264148 11475 rpc_server.cc:307] RPC server started. Bound to: 127.1.135.194:45741
I20260923 22:13:14.264195 11602 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.1.135.194:45741 every 8 connection(s)
I20260923 22:13:14.264464 11475 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskRiQe_a/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790201539513534-1567-0/minicluster-data/ts-1/data/info.pb
I20260923 22:13:14.265343 1567 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskRiQe_a/build/release/bin/kudu as pid 11475
I20260923 22:13:14.265471 1567 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskRiQe_a/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790201539513534-1567-0/minicluster-data/ts-1/wal/instance
I20260923 22:13:14.265662 1567 external_mini_cluster.cc:1428] Setting key bc88dcd4893e401c1238b7602fa3a60a
I20260923 22:13:14.266659 1567 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskRiQe_a/build/release/bin/kudu
/tmp/dist-test-taskRiQe_a/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskRiQe_a/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790201539513534-1567-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskRiQe_a/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790201539513534-1567-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskRiQe_a/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790201539513534-1567-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskRiQe_a/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790201539513534-1567-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.1.135.195:0
--local_ip_for_outbound_sockets=127.1.135.195
--webserver_interface=127.1.135.195
--webserver_port=0
--tserver_master_addrs=127.1.135.254:38971
--builtin_ntp_servers=127.1.135.212:39981
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_encryption=disabled
--rpc_authentication=disabled with env {}
I20260923 22:13:14.267457 11324 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 22:13:14.265123 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.135.194:48099 (local address 127.1.135.254:38971)
0923 22:13:14.265202 (+ 79us) server_negotiation.cc:206] Beginning negotiation
0923 22:13:14.265205 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0923 22:13:14.265384 (+ 179us) server_negotiation.cc:407] Connection header received
0923 22:13:14.265970 (+ 586us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 22:13:14.265972 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 22:13:14.265978 (+ 6us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 22:13:14.266057 (+ 79us) server_negotiation.cc:226] Negotiated authn=SASL
0923 22:13:14.267206 (+ 1149us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0923 22:13:14.267208 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0923 22:13:14.267208 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0923 22:13:14.267213 (+ 5us) server_negotiation.cc:922] calling sasl_server_start()
0923 22:13:14.267229 (+ 16us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0923 22:13:14.267235 (+ 6us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0923 22:13:14.267299 (+ 64us) server_negotiation.cc:1035] Waiting for connection context
0923 22:13:14.267347 (+ 48us) server_negotiation.cc:299] Negotiation successful
0923 22:13:14.267361 (+ 14us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":53}
I20260923 22:13:14.267629 11605 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 22:13:14.265091 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.135.254:38971 (local address 127.1.135.194:48099)
0923 22:13:14.265276 (+ 185us) negotiation.cc:107] Waiting for socket to connect
0923 22:13:14.265303 (+ 27us) client_negotiation.cc:174] Beginning negotiation
0923 22:13:14.265861 (+ 558us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0923 22:13:14.266107 (+ 246us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0923 22:13:14.266110 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0923 22:13:14.266117 (+ 7us) client_negotiation.cc:189] Negotiated authn=SASL
0923 22:13:14.267066 (+ 949us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0923 22:13:14.267066 (+ 0us) client_negotiation.cc:656] calling sasl_client_start()
0923 22:13:14.267091 (+ 25us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0923 22:13:14.267116 (+ 25us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0923 22:13:14.267298 (+ 182us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0923 22:13:14.267300 (+ 2us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0923 22:13:14.267302 (+ 2us) client_negotiation.cc:769] Sending connection context
0923 22:13:14.267350 (+ 48us) client_negotiation.cc:240] Negotiation successful
0923 22:13:14.267392 (+ 42us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":130,"thread_start_us":102,"threads_started":1}
I20260923 22:13:14.268100 11603 heartbeater.cc:347] Connected to a master server at 127.1.135.254:38971
I20260923 22:13:14.268201 11603 heartbeater.cc:464] Registering TS with master...
I20260923 22:13:14.268335 11603 heartbeater.cc:511] Master 127.1.135.254:38971 requested a full tablet report, sending...
I20260923 22:13:14.268707 11284 ts_manager.cc:194] Registered new tserver with Master: 0339c9275dd64db991a0f9ad0922fb00 (127.1.135.194:45741)
W20260923 22:13:14.345115 11606 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260923 22:13:14.345335 11606 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260923 22:13:14.345368 11606 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260923 22:13:14.346501 11606 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260923 22:13:14.346565 11606 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260923 22:13:14.346633 11606 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.1.135.195
I20260923 22:13:14.347930 11606 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.1.135.212:39981
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskRiQe_a/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790201539513534-1567-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskRiQe_a/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790201539513534-1567-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.1.135.195:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskRiQe_a/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790201539513534-1567-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.1.135.195
--webserver_port=0
--tserver_master_addrs=127.1.135.254:38971
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.11606
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.1.135.195
--log_dir=/tmp/dist-test-taskRiQe_a/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790201539513534-1567-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 23 Sep 2026 20:53:37 UTC on ccycloud.quasar-rkviyx.root.comops.site
build id 4668083
I20260923 22:13:14.348210 11606 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260923 22:13:14.348487 11606 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260923 22:13:14.353150 11611 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260923 22:13:14.353178 11612 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260923 22:13:14.353191 11614 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260923 22:13:14.353193 11606 server_base.cc:1034] running on GCE node
I20260923 22:13:14.353579 11606 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260923 22:13:14.353817 11606 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260923 22:13:14.354974 11606 hybrid_clock.cc:648] HybridClock initialized: now 1790201594354970 us; error 34 us; skew 500 ppm
I20260923 22:13:14.356252 11606 webserver.cc:460] Webserver started at http://127.1.135.195:42579/ using document root <none> and password file <none>
I20260923 22:13:14.356508 11606 fs_manager.cc:359] Metadata directory not provided
I20260923 22:13:14.356570 11606 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260923 22:13:14.356701 11606 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260923 22:13:14.357752 11606 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskRiQe_a/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790201539513534-1567-0/minicluster-data/ts-2/data/instance:
uuid: "af977fb5a53944168d9b22cc19f687fe"
format_stamp: "Formatted at 2026-09-23 22:13:14 on dist-test-slave-s2fj"
server_key: "4d9babd36a338b08f0a673182961b69f"
server_key_iv: "c95c282de2634ef61eebe1baeacf3d59"
server_key_version: "encryptionkey@0"
I20260923 22:13:14.358134 11606 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskRiQe_a/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790201539513534-1567-0/minicluster-data/ts-2/wal/instance:
uuid: "af977fb5a53944168d9b22cc19f687fe"
format_stamp: "Formatted at 2026-09-23 22:13:14 on dist-test-slave-s2fj"
server_key: "4d9babd36a338b08f0a673182961b69f"
server_key_iv: "c95c282de2634ef61eebe1baeacf3d59"
server_key_version: "encryptionkey@0"
I20260923 22:13:14.359373 11606 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.001s sys 0.000s
I20260923 22:13:14.360028 11620 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260923 22:13:14.360213 11606 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260923 22:13:14.360306 11606 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskRiQe_a/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790201539513534-1567-0/minicluster-data/ts-2/data,/tmp/dist-test-taskRiQe_a/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790201539513534-1567-0/minicluster-data/ts-2/wal
uuid: "af977fb5a53944168d9b22cc19f687fe"
format_stamp: "Formatted at 2026-09-23 22:13:14 on dist-test-slave-s2fj"
server_key: "4d9babd36a338b08f0a673182961b69f"
server_key_iv: "c95c282de2634ef61eebe1baeacf3d59"
server_key_version: "encryptionkey@0"
I20260923 22:13:14.360381 11606 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskRiQe_a/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790201539513534-1567-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskRiQe_a/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790201539513534-1567-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskRiQe_a/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790201539513534-1567-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260923 22:13:14.363550 11606 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260923 22:13:14.363796 11606 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260923 22:13:14.363948 11606 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260923 22:13:14.364168 11606 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260923 22:13:14.364460 11606 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260923 22:13:14.364514 11606 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260923 22:13:14.364553 11606 ts_tablet_manager.cc:610] Registered 0 tablets
I20260923 22:13:14.364580 11606 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260923 22:13:14.370108 11606 rpc_server.cc:307] RPC server started. Bound to: 127.1.135.195:41777
I20260923 22:13:14.370143 11733 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.1.135.195:41777 every 8 connection(s)
I20260923 22:13:14.370446 11606 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskRiQe_a/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790201539513534-1567-0/minicluster-data/ts-2/data/info.pb
I20260923 22:13:14.371549 1567 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskRiQe_a/build/release/bin/kudu as pid 11606
I20260923 22:13:14.371649 1567 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskRiQe_a/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790201539513534-1567-0/minicluster-data/ts-2/wal/instance
I20260923 22:13:14.371851 1567 external_mini_cluster.cc:1428] Setting key 67b181f94019a122da8c5932034b9cb5
I20260923 22:13:14.373565 11324 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 22:13:14.371124 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.135.195:55211 (local address 127.1.135.254:38971)
0923 22:13:14.371191 (+ 67us) server_negotiation.cc:206] Beginning negotiation
0923 22:13:14.371193 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0923 22:13:14.371391 (+ 198us) server_negotiation.cc:407] Connection header received
0923 22:13:14.372110 (+ 719us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 22:13:14.372112 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 22:13:14.372120 (+ 8us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 22:13:14.372171 (+ 51us) server_negotiation.cc:226] Negotiated authn=SASL
0923 22:13:14.373355 (+ 1184us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0923 22:13:14.373356 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0923 22:13:14.373357 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0923 22:13:14.373361 (+ 4us) server_negotiation.cc:922] calling sasl_server_start()
0923 22:13:14.373375 (+ 14us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0923 22:13:14.373378 (+ 3us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0923 22:13:14.373429 (+ 51us) server_negotiation.cc:1035] Waiting for connection context
0923 22:13:14.373495 (+ 66us) server_negotiation.cc:299] Negotiation successful
0923 22:13:14.373508 (+ 13us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":38}
I20260923 22:13:14.373728 11736 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 22:13:14.371092 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.135.254:38971 (local address 127.1.135.195:55211)
0923 22:13:14.371277 (+ 185us) negotiation.cc:107] Waiting for socket to connect
0923 22:13:14.371306 (+ 29us) client_negotiation.cc:174] Beginning negotiation
0923 22:13:14.371981 (+ 675us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0923 22:13:14.372256 (+ 275us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0923 22:13:14.372277 (+ 21us) client_negotiation.cc:394] Received NEGOTIATE response from server
0923 22:13:14.372286 (+ 9us) client_negotiation.cc:189] Negotiated authn=SASL
0923 22:13:14.373207 (+ 921us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0923 22:13:14.373207 (+ 0us) client_negotiation.cc:656] calling sasl_client_start()
0923 22:13:14.373252 (+ 45us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0923 22:13:14.373277 (+ 25us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0923 22:13:14.373430 (+ 153us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0923 22:13:14.373432 (+ 2us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0923 22:13:14.373435 (+ 3us) client_negotiation.cc:769] Sending connection context
0923 22:13:14.373491 (+ 56us) client_negotiation.cc:240] Negotiation successful
0923 22:13:14.373534 (+ 43us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":135,"thread_start_us":60,"threads_started":1}
I20260923 22:13:14.374163 11734 heartbeater.cc:347] Connected to a master server at 127.1.135.254:38971
I20260923 22:13:14.374233 11734 heartbeater.cc:464] Registering TS with master...
I20260923 22:13:14.374382 11734 heartbeater.cc:511] Master 127.1.135.254:38971 requested a full tablet report, sending...
I20260923 22:13:14.374727 11284 ts_manager.cc:194] Registered new tserver with Master: af977fb5a53944168d9b22cc19f687fe (127.1.135.195:41777)
I20260923 22:13:14.383683 1567 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
W20260923 22:13:14.385550 11324 negotiation.cc:337] Failed RPC negotiation. Trace:
0923 22:13:14.385049 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59734 (local address 127.1.135.254:38971)
0923 22:13:14.385149 (+ 100us) server_negotiation.cc:206] Beginning negotiation
0923 22:13:14.385150 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0923 22:13:14.385230 (+ 80us) server_negotiation.cc:407] Connection header received
0923 22:13:14.385288 (+ 58us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 22:13:14.385289 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 22:13:14.385294 (+ 5us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 22:13:14.385364 (+ 70us) server_negotiation.cc:226] Negotiated authn=SASL
0923 22:13:14.385484 (+ 120us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:59734: BlockingRecv error: recv got EOF from 127.0.0.1:59734 (error 108)
Metrics: {"server-negotiator.queue_time_us":64}
I20260923 22:13:14.385656 1567 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskRiQe_a/build/release/bin/kudu with pid 11344
I20260923 22:13:14.390759 1567 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskRiQe_a/build/release/bin/kudu with pid 11475
I20260923 22:13:14.395689 1567 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskRiQe_a/build/release/bin/kudu with pid 11606
I20260923 22:13:14.400807 1567 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskRiQe_a/build/release/bin/kudu with pid 11253
2026-09-23T22:13:14Z chronyd exiting
[ OK ] SecurityITest.TestRequireEncryptionInsecureCluster (511 ms)
[ RUN ] SecurityITest.TestRequireAuthenticationSecureCluster
Loading random data
Initializing database '/tmp/dist-test-taskRiQe_a/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790201539513534-1567-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Sep 23 22:13:14 dist-test-slave-s2fj krb5kdc[11744](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Sep 23 22:13:14 dist-test-slave-s2fj krb5kdc[11744](info): set up 2 sockets
Sep 23 22:13:14 dist-test-slave-s2fj krb5kdc[11744](info): commencing operation
krb5kdc: starting...
W20260923 22:13:15.429260 1567 mini_kdc.cc:121] Time spent starting KDC: real 1.009s user 0.002s sys 0.003s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskRiQe_a/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790201539513534-1567-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskRiQe_a/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790201539513534-1567-0/krb5kdc/test-user.keytab.
Sep 23 22:13:15 dist-test-slave-s2fj krb5kdc[11744](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790201595, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-09-23T22:13:15Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-09-23T22:13:15Z Disabled control of system clock
WARNING: no policy specified for kudu/127.1.135.254@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.1.135.254@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.1.135.254 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskRiQe_a/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790201539513534-1567-0/krb5kdc/kudu_127.1.135.254.keytab.
Entry for principal kudu/127.1.135.254 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskRiQe_a/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790201539513534-1567-0/krb5kdc/kudu_127.1.135.254.keytab.
I20260923 22:13:15.540436 1567 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskRiQe_a/build/release/bin/kudu
/tmp/dist-test-taskRiQe_a/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskRiQe_a/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790201539513534-1567-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskRiQe_a/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790201539513534-1567-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskRiQe_a/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790201539513534-1567-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskRiQe_a/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790201539513534-1567-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.1.135.254:45027
--webserver_interface=127.1.135.254
--webserver_port=0
--keytab_file=/tmp/dist-test-taskRiQe_a/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790201539513534-1567-0/krb5kdc/kudu_127.1.135.254.keytab
--principal=kudu/127.1.135.254
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.1.135.212:35883
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.1.135.254:45027
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskRiQe_a/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790201539513534-1567-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskRiQe_a/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790201539513534-1567-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskRiQe_a/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790201539513534-1567-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260923 22:13:15.617280 11757 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260923 22:13:15.617460 11757 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260923 22:13:15.617480 11757 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260923 22:13:15.618505 11757 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260923 22:13:15.618541 11757 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260923 22:13:15.618556 11757 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260923 22:13:15.618567 11757 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260923 22:13:15.618579 11757 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260923 22:13:15.619982 11757 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.1.135.212:35883
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskRiQe_a/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790201539513534-1567-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskRiQe_a/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790201539513534-1567-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.1.135.254:45027
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskRiQe_a/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790201539513534-1567-0/krb5kdc/kudu_127.1.135.254.keytab
--principal=kudu/127.1.135.254
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.1.135.254:45027
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskRiQe_a/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790201539513534-1567-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.1.135.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.11757
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskRiQe_a/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790201539513534-1567-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 23 Sep 2026 20:53:37 UTC on ccycloud.quasar-rkviyx.root.comops.site
build id 4668083
I20260923 22:13:15.620191 11757 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260923 22:13:15.620349 11757 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260923 22:13:15.624696 11763 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260923 22:13:15.624707 11762 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260923 22:13:15.624821 11765 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260923 22:13:15.624998 11757 server_base.cc:1034] running on GCE node
I20260923 22:13:15.625232 11757 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260923 22:13:15.625524 11757 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260923 22:13:15.626660 11757 hybrid_clock.cc:648] HybridClock initialized: now 1790201595626655 us; error 40 us; skew 500 ppm
Sep 23 22:13:15 dist-test-slave-s2fj krb5kdc[11744](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790201595, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.135.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260923 22:13:15.628924 11757 init.cc:377] Logged in from keytab as kudu/127.1.135.254@KRBTEST.COM (short username kudu)
I20260923 22:13:15.629374 11757 webserver.cc:460] Webserver started at http://127.1.135.254:42473/ using document root <none> and password file <none>
I20260923 22:13:15.629587 11757 fs_manager.cc:359] Metadata directory not provided
I20260923 22:13:15.629673 11757 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260923 22:13:15.629818 11757 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260923 22:13:15.630795 11757 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskRiQe_a/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790201539513534-1567-0/minicluster-data/master-0/data/instance:
uuid: "352c40227f674e1cb876aaa945ed8166"
format_stamp: "Formatted at 2026-09-23 22:13:15 on dist-test-slave-s2fj"
server_key: "f6e2dc63f1c90a4202630499d38f4b37"
server_key_iv: "24085063ced1455e4c79823109b5d391"
server_key_version: "encryptionkey@0"
I20260923 22:13:15.631141 11757 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskRiQe_a/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790201539513534-1567-0/minicluster-data/master-0/wal/instance:
uuid: "352c40227f674e1cb876aaa945ed8166"
format_stamp: "Formatted at 2026-09-23 22:13:15 on dist-test-slave-s2fj"
server_key: "f6e2dc63f1c90a4202630499d38f4b37"
server_key_iv: "24085063ced1455e4c79823109b5d391"
server_key_version: "encryptionkey@0"
I20260923 22:13:15.632455 11757 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260923 22:13:15.633167 11772 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260923 22:13:15.633328 11757 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260923 22:13:15.633431 11757 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskRiQe_a/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790201539513534-1567-0/minicluster-data/master-0/data,/tmp/dist-test-taskRiQe_a/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790201539513534-1567-0/minicluster-data/master-0/wal
uuid: "352c40227f674e1cb876aaa945ed8166"
format_stamp: "Formatted at 2026-09-23 22:13:15 on dist-test-slave-s2fj"
server_key: "f6e2dc63f1c90a4202630499d38f4b37"
server_key_iv: "24085063ced1455e4c79823109b5d391"
server_key_version: "encryptionkey@0"
I20260923 22:13:15.633503 11757 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskRiQe_a/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790201539513534-1567-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskRiQe_a/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790201539513534-1567-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskRiQe_a/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790201539513534-1567-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260923 22:13:15.644767 11757 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260923 22:13:15.646448 11757 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260923 22:13:15.646610 11757 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260923 22:13:15.650619 11757 rpc_server.cc:307] RPC server started. Bound to: 127.1.135.254:45027
I20260923 22:13:15.650650 11824 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.1.135.254:45027 every 8 connection(s)
I20260923 22:13:15.650967 11757 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskRiQe_a/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790201539513534-1567-0/minicluster-data/master-0/data/info.pb
I20260923 22:13:15.651889 11825 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260923 22:13:15.654047 11825 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 352c40227f674e1cb876aaa945ed8166: Bootstrap starting.
I20260923 22:13:15.654579 1567 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskRiQe_a/build/release/bin/kudu as pid 11757
I20260923 22:13:15.654605 11825 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 352c40227f674e1cb876aaa945ed8166: Neither blocks nor log segments found. Creating new log.
I20260923 22:13:15.654665 1567 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskRiQe_a/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790201539513534-1567-0/minicluster-data/master-0/wal/instance
I20260923 22:13:15.654855 1567 external_mini_cluster.cc:1428] Setting key dcc8f649dbe3206828492eb3f9a5611d
I20260923 22:13:15.654917 11825 log.cc:824] T 00000000000000000000000000000000 P 352c40227f674e1cb876aaa945ed8166: Log is configured to *not* fsync() on all Append() calls
I20260923 22:13:15.655680 11825 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 352c40227f674e1cb876aaa945ed8166: No bootstrap required, opened a new log
I20260923 22:13:15.657188 11825 raft_consensus.cc:348] T 00000000000000000000000000000000 P 352c40227f674e1cb876aaa945ed8166 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "352c40227f674e1cb876aaa945ed8166" member_type: VOTER last_known_addr { host: "127.1.135.254" port: 45027 } }
I20260923 22:13:15.657313 11825 raft_consensus.cc:374] T 00000000000000000000000000000000 P 352c40227f674e1cb876aaa945ed8166 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260923 22:13:15.657336 11825 raft_consensus.cc:729] T 00000000000000000000000000000000 P 352c40227f674e1cb876aaa945ed8166 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 352c40227f674e1cb876aaa945ed8166, State: Initialized, Role: FOLLOWER
I20260923 22:13:15.657473 11825 consensus_queue.cc:260] T 00000000000000000000000000000000 P 352c40227f674e1cb876aaa945ed8166 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "352c40227f674e1cb876aaa945ed8166" member_type: VOTER last_known_addr { host: "127.1.135.254" port: 45027 } }
I20260923 22:13:15.657552 11825 raft_consensus.cc:388] T 00000000000000000000000000000000 P 352c40227f674e1cb876aaa945ed8166 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260923 22:13:15.657575 11825 raft_consensus.cc:482] T 00000000000000000000000000000000 P 352c40227f674e1cb876aaa945ed8166 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260923 22:13:15.657644 11825 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 352c40227f674e1cb876aaa945ed8166 [term 0 FOLLOWER]: Advancing to term 1
I20260923 22:13:15.658131 11825 raft_consensus.cc:504] T 00000000000000000000000000000000 P 352c40227f674e1cb876aaa945ed8166 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "352c40227f674e1cb876aaa945ed8166" member_type: VOTER last_known_addr { host: "127.1.135.254" port: 45027 } }
I20260923 22:13:15.658213 11825 leader_election.cc:302] T 00000000000000000000000000000000 P 352c40227f674e1cb876aaa945ed8166 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 352c40227f674e1cb876aaa945ed8166; no voters:
I20260923 22:13:15.658468 11825 leader_election.cc:288] T 00000000000000000000000000000000 P 352c40227f674e1cb876aaa945ed8166 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260923 22:13:15.658774 11825 sys_catalog.cc:564] T 00000000000000000000000000000000 P 352c40227f674e1cb876aaa945ed8166 [sys.catalog]: configured and running, proceeding with master startup.
I20260923 22:13:15.658490 11830 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 352c40227f674e1cb876aaa945ed8166 [term 1 FOLLOWER]: Leader election won for term 1
I20260923 22:13:15.659117 11830 raft_consensus.cc:686] T 00000000000000000000000000000000 P 352c40227f674e1cb876aaa945ed8166 [term 1 LEADER]: Becoming Leader. State: Replica: 352c40227f674e1cb876aaa945ed8166, State: Running, Role: LEADER
I20260923 22:13:15.659322 11830 consensus_queue.cc:237] T 00000000000000000000000000000000 P 352c40227f674e1cb876aaa945ed8166 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "352c40227f674e1cb876aaa945ed8166" member_type: VOTER last_known_addr { host: "127.1.135.254" port: 45027 } }
I20260923 22:13:15.659718 11832 sys_catalog.cc:455] T 00000000000000000000000000000000 P 352c40227f674e1cb876aaa945ed8166 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 352c40227f674e1cb876aaa945ed8166. Latest consensus state: current_term: 1 leader_uuid: "352c40227f674e1cb876aaa945ed8166" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "352c40227f674e1cb876aaa945ed8166" member_type: VOTER last_known_addr { host: "127.1.135.254" port: 45027 } } }
I20260923 22:13:15.659792 11832 sys_catalog.cc:458] T 00000000000000000000000000000000 P 352c40227f674e1cb876aaa945ed8166 [sys.catalog]: This master's current role is: LEADER
I20260923 22:13:15.660194 11831 sys_catalog.cc:455] T 00000000000000000000000000000000 P 352c40227f674e1cb876aaa945ed8166 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "352c40227f674e1cb876aaa945ed8166" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "352c40227f674e1cb876aaa945ed8166" member_type: VOTER last_known_addr { host: "127.1.135.254" port: 45027 } } }
I20260923 22:13:15.660702 11831 sys_catalog.cc:458] T 00000000000000000000000000000000 P 352c40227f674e1cb876aaa945ed8166 [sys.catalog]: This master's current role is: LEADER
W20260923 22:13:15.660737 11845 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 352c40227f674e1cb876aaa945ed8166: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260923 22:13:15.660861 11845 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260923 22:13:15.660959 11842 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260923 22:13:15.661142 11842 catalog_manager.cc:1484] Initializing Kudu cluster ID...
Sep 23 22:13:15 dist-test-slave-s2fj krb5kdc[11744](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790201595, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.1.135.254@KRBTEST.COM
I20260923 22:13:15.662602 11842 catalog_manager.cc:1347] Generated new cluster ID: ccc0d6c7327042389af4733a4bcc4e74
I20260923 22:13:15.662654 11842 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260923 22:13:15.666065 11829 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 22:13:15.655353 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53220 (local address 127.1.135.254:45027)
0923 22:13:15.655538 (+ 185us) server_negotiation.cc:206] Beginning negotiation
0923 22:13:15.655548 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0923 22:13:15.655577 (+ 29us) server_negotiation.cc:407] Connection header received
0923 22:13:15.656028 (+ 451us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 22:13:15.656035 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 22:13:15.656127 (+ 92us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 22:13:15.656271 (+ 144us) server_negotiation.cc:226] Negotiated authn=SASL
0923 22:13:15.656954 (+ 683us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 22:13:15.658225 (+ 1271us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 22:13:15.658963 (+ 738us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 22:13:15.659631 (+ 668us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 22:13:15.660092 (+ 461us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 22:13:15.662782 (+ 2690us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0923 22:13:15.662786 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0923 22:13:15.662786 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0923 22:13:15.662797 (+ 11us) server_negotiation.cc:922] calling sasl_server_start()
0923 22:13:15.664521 (+ 1724us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0923 22:13:15.664897 (+ 376us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0923 22:13:15.664900 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0923 22:13:15.664900 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0923 22:13:15.664986 (+ 86us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0923 22:13:15.665197 (+ 211us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0923 22:13:15.665198 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0923 22:13:15.665199 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0923 22:13:15.665406 (+ 207us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0923 22:13:15.665586 (+ 180us) server_negotiation.cc:1035] Waiting for connection context
0923 22:13:15.665648 (+ 62us) server_negotiation.cc:299] Negotiation successful
0923 22:13:15.665776 (+ 128us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":140,"thread_start_us":93,"threads_started":1}
I20260923 22:13:15.668923 11842 catalog_manager.cc:1370] Generated new certificate authority record
I20260923 22:13:15.669415 11842 catalog_manager.cc:1504] Loading token signing keys...
I20260923 22:13:15.678218 11842 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 352c40227f674e1cb876aaa945ed8166: Generated new TSK 0
I20260923 22:13:15.678382 11842 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.1.135.193@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.1.135.193@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.1.135.193 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskRiQe_a/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790201539513534-1567-0/krb5kdc/kudu_127.1.135.193.keytab.
Entry for principal kudu/127.1.135.193 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskRiQe_a/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790201539513534-1567-0/krb5kdc/kudu_127.1.135.193.keytab.
I20260923 22:13:15.706913 1567 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskRiQe_a/build/release/bin/kudu
/tmp/dist-test-taskRiQe_a/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskRiQe_a/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790201539513534-1567-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskRiQe_a/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790201539513534-1567-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskRiQe_a/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790201539513534-1567-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskRiQe_a/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790201539513534-1567-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.1.135.193:0
--local_ip_for_outbound_sockets=127.1.135.193
--webserver_interface=127.1.135.193
--webserver_port=0
--tserver_master_addrs=127.1.135.254:45027
--keytab_file=/tmp/dist-test-taskRiQe_a/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790201539513534-1567-0/krb5kdc/kudu_127.1.135.193.keytab
--principal=kudu/127.1.135.193
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.1.135.212:35883
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskRiQe_a/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790201539513534-1567-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskRiQe_a/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790201539513534-1567-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskRiQe_a/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790201539513534-1567-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260923 22:13:15.791126 11851 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260923 22:13:15.791327 11851 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260923 22:13:15.791364 11851 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260923 22:13:15.792531 11851 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260923 22:13:15.792588 11851 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260923 22:13:15.792631 11851 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.1.135.193
I20260923 22:13:15.793990 11851 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.1.135.212:35883
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskRiQe_a/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790201539513534-1567-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskRiQe_a/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790201539513534-1567-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskRiQe_a/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790201539513534-1567-0/krb5kdc/kudu_127.1.135.193.keytab
--principal=kudu/127.1.135.193
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.1.135.193:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskRiQe_a/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790201539513534-1567-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.1.135.193
--webserver_port=0
--tserver_master_addrs=127.1.135.254:45027
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.11851
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.1.135.193
--log_dir=/tmp/dist-test-taskRiQe_a/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790201539513534-1567-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 23 Sep 2026 20:53:37 UTC on ccycloud.quasar-rkviyx.root.comops.site
build id 4668083
I20260923 22:13:15.794245 11851 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260923 22:13:15.794450 11851 file_cache.cc:493] Constructed file cache file cache with capacity 419430
I20260923 22:13:15.800215 11851 server_base.cc:1034] running on GCE node
W20260923 22:13:15.800212 11856 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260923 22:13:15.800323 11859 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260923 22:13:15.800213 11857 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260923 22:13:15.800577 11851 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260923 22:13:15.800827 11851 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260923 22:13:15.801985 11851 hybrid_clock.cc:648] HybridClock initialized: now 1790201595801968 us; error 35 us; skew 500 ppm
Sep 23 22:13:15 dist-test-slave-s2fj krb5kdc[11744](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790201595, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.135.193@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260923 22:13:15.804404 11851 init.cc:377] Logged in from keytab as kudu/127.1.135.193@KRBTEST.COM (short username kudu)
I20260923 22:13:15.804795 11851 webserver.cc:460] Webserver started at http://127.1.135.193:39501/ using document root <none> and password file <none>
I20260923 22:13:15.805006 11851 fs_manager.cc:359] Metadata directory not provided
I20260923 22:13:15.805079 11851 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260923 22:13:15.805228 11851 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260923 22:13:15.806219 11851 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskRiQe_a/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790201539513534-1567-0/minicluster-data/ts-0/data/instance:
uuid: "893017a9339b4fe5a64028d7f1b7ff21"
format_stamp: "Formatted at 2026-09-23 22:13:15 on dist-test-slave-s2fj"
server_key: "87bf5d971de9bec5962bcccdb5ab3610"
server_key_iv: "f3a5328c747aee990fbe53e7fe15a45b"
server_key_version: "encryptionkey@0"
I20260923 22:13:15.806594 11851 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskRiQe_a/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790201539513534-1567-0/minicluster-data/ts-0/wal/instance:
uuid: "893017a9339b4fe5a64028d7f1b7ff21"
format_stamp: "Formatted at 2026-09-23 22:13:15 on dist-test-slave-s2fj"
server_key: "87bf5d971de9bec5962bcccdb5ab3610"
server_key_iv: "f3a5328c747aee990fbe53e7fe15a45b"
server_key_version: "encryptionkey@0"
I20260923 22:13:15.807879 11851 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.001s
I20260923 22:13:15.808669 11866 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260923 22:13:15.808840 11851 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.000s
I20260923 22:13:15.808931 11851 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskRiQe_a/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790201539513534-1567-0/minicluster-data/ts-0/data,/tmp/dist-test-taskRiQe_a/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790201539513534-1567-0/minicluster-data/ts-0/wal
uuid: "893017a9339b4fe5a64028d7f1b7ff21"
format_stamp: "Formatted at 2026-09-23 22:13:15 on dist-test-slave-s2fj"
server_key: "87bf5d971de9bec5962bcccdb5ab3610"
server_key_iv: "f3a5328c747aee990fbe53e7fe15a45b"
server_key_version: "encryptionkey@0"
I20260923 22:13:15.809017 11851 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskRiQe_a/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790201539513534-1567-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskRiQe_a/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790201539513534-1567-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskRiQe_a/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790201539513534-1567-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260923 22:13:15.820781 11851 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260923 22:13:15.822558 11851 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260923 22:13:15.822707 11851 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260923 22:13:15.822916 11851 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260923 22:13:15.823191 11851 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260923 22:13:15.823243 11851 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260923 22:13:15.823276 11851 ts_tablet_manager.cc:610] Registered 0 tablets
I20260923 22:13:15.823303 11851 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260923 22:13:15.828794 11851 rpc_server.cc:307] RPC server started. Bound to: 127.1.135.193:39785
I20260923 22:13:15.828828 11979 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.1.135.193:39785 every 8 connection(s)
I20260923 22:13:15.829123 11851 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskRiQe_a/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790201539513534-1567-0/minicluster-data/ts-0/data/info.pb
I20260923 22:13:15.831447 1567 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskRiQe_a/build/release/bin/kudu as pid 11851
I20260923 22:13:15.831534 1567 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskRiQe_a/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790201539513534-1567-0/minicluster-data/ts-0/wal/instance
I20260923 22:13:15.831763 1567 external_mini_cluster.cc:1428] Setting key ad9577bd37c394efbc01e6e79f811c3a
Sep 23 22:13:15 dist-test-slave-s2fj krb5kdc[11744](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790201595, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.135.193@KRBTEST.COM for kudu/127.1.135.254@KRBTEST.COM
I20260923 22:13:15.839074 11829 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 22:13:15.829832 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.135.193:52291 (local address 127.1.135.254:45027)
0923 22:13:15.829891 (+ 59us) server_negotiation.cc:206] Beginning negotiation
0923 22:13:15.829893 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0923 22:13:15.830097 (+ 204us) server_negotiation.cc:407] Connection header received
0923 22:13:15.831013 (+ 916us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 22:13:15.831015 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 22:13:15.831032 (+ 17us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 22:13:15.831123 (+ 91us) server_negotiation.cc:226] Negotiated authn=SASL
0923 22:13:15.831772 (+ 649us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 22:13:15.832539 (+ 767us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 22:13:15.833564 (+ 1025us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 22:13:15.833775 (+ 211us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 22:13:15.833832 (+ 57us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 22:13:15.836162 (+ 2330us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0923 22:13:15.836163 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0923 22:13:15.836163 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0923 22:13:15.836169 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0923 22:13:15.837552 (+ 1383us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0923 22:13:15.838109 (+ 557us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0923 22:13:15.838110 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0923 22:13:15.838110 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0923 22:13:15.838155 (+ 45us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0923 22:13:15.838489 (+ 334us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0923 22:13:15.838490 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0923 22:13:15.838490 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0923 22:13:15.838588 (+ 98us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0923 22:13:15.838637 (+ 49us) server_negotiation.cc:1035] Waiting for connection context
0923 22:13:15.838950 (+ 313us) server_negotiation.cc:299] Negotiation successful
0923 22:13:15.839008 (+ 58us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":36}
I20260923 22:13:15.839337 11982 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 22:13:15.829821 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.135.254:45027 (local address 127.1.135.193:52291)
0923 22:13:15.829974 (+ 153us) negotiation.cc:107] Waiting for socket to connect
0923 22:13:15.830014 (+ 40us) client_negotiation.cc:174] Beginning negotiation
0923 22:13:15.830891 (+ 877us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0923 22:13:15.831190 (+ 299us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0923 22:13:15.831193 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0923 22:13:15.831418 (+ 225us) client_negotiation.cc:189] Negotiated authn=SASL
0923 22:13:15.831682 (+ 264us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 22:13:15.831688 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 22:13:15.832632 (+ 944us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 22:13:15.832633 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 22:13:15.833462 (+ 829us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 22:13:15.833466 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 22:13:15.833856 (+ 390us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 22:13:15.833857 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 22:13:15.834012 (+ 155us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 22:13:15.834461 (+ 449us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0923 22:13:15.834470 (+ 9us) client_negotiation.cc:656] calling sasl_client_start()
0923 22:13:15.836083 (+ 1613us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0923 22:13:15.837636 (+ 1553us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0923 22:13:15.837643 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0923 22:13:15.837645 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0923 22:13:15.838029 (+ 384us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0923 22:13:15.838231 (+ 202us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0923 22:13:15.838232 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0923 22:13:15.838233 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0923 22:13:15.838372 (+ 139us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0923 22:13:15.838659 (+ 287us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0923 22:13:15.838666 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0923 22:13:15.838820 (+ 154us) client_negotiation.cc:769] Sending connection context
0923 22:13:15.838932 (+ 112us) client_negotiation.cc:240] Negotiation successful
0923 22:13:15.839098 (+ 166us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":121,"thread_start_us":72,"threads_started":1}
I20260923 22:13:15.839872 11980 heartbeater.cc:347] Connected to a master server at 127.1.135.254:45027
I20260923 22:13:15.839972 11980 heartbeater.cc:464] Registering TS with master...
I20260923 22:13:15.840184 11980 heartbeater.cc:511] Master 127.1.135.254:45027 requested a full tablet report, sending...
I20260923 22:13:15.840724 11789 ts_manager.cc:194] Registered new tserver with Master: 893017a9339b4fe5a64028d7f1b7ff21 (127.1.135.193:39785)
WARNING: no policy specified for kudu/127.1.135.194@KRBTEST.COM; defaulting to no policy
I20260923 22:13:15.841588 11789 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.1.135.193@KRBTEST.COM'} at 127.1.135.193:52291
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.1.135.194@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.1.135.194 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskRiQe_a/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790201539513534-1567-0/krb5kdc/kudu_127.1.135.194.keytab.
Entry for principal kudu/127.1.135.194 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskRiQe_a/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790201539513534-1567-0/krb5kdc/kudu_127.1.135.194.keytab.
I20260923 22:13:15.858026 1567 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskRiQe_a/build/release/bin/kudu
/tmp/dist-test-taskRiQe_a/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskRiQe_a/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790201539513534-1567-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskRiQe_a/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790201539513534-1567-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskRiQe_a/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790201539513534-1567-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskRiQe_a/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790201539513534-1567-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.1.135.194:0
--local_ip_for_outbound_sockets=127.1.135.194
--webserver_interface=127.1.135.194
--webserver_port=0
--tserver_master_addrs=127.1.135.254:45027
--keytab_file=/tmp/dist-test-taskRiQe_a/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790201539513534-1567-0/krb5kdc/kudu_127.1.135.194.keytab
--principal=kudu/127.1.135.194
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.1.135.212:35883
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskRiQe_a/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790201539513534-1567-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskRiQe_a/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790201539513534-1567-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskRiQe_a/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790201539513534-1567-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260923 22:13:15.937273 11985 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260923 22:13:15.937489 11985 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260923 22:13:15.937529 11985 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260923 22:13:15.938678 11985 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260923 22:13:15.938742 11985 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260923 22:13:15.938787 11985 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.1.135.194
I20260923 22:13:15.940326 11985 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.1.135.212:35883
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskRiQe_a/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790201539513534-1567-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskRiQe_a/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790201539513534-1567-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskRiQe_a/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790201539513534-1567-0/krb5kdc/kudu_127.1.135.194.keytab
--principal=kudu/127.1.135.194
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.1.135.194:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskRiQe_a/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790201539513534-1567-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.1.135.194
--webserver_port=0
--tserver_master_addrs=127.1.135.254:45027
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.11985
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.1.135.194
--log_dir=/tmp/dist-test-taskRiQe_a/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790201539513534-1567-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 23 Sep 2026 20:53:37 UTC on ccycloud.quasar-rkviyx.root.comops.site
build id 4668083
I20260923 22:13:15.940593 11985 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260923 22:13:15.940804 11985 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260923 22:13:15.945744 11993 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260923 22:13:15.945832 11985 server_base.cc:1034] running on GCE node
W20260923 22:13:15.945742 11990 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260923 22:13:15.945770 11991 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260923 22:13:15.946126 11985 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260923 22:13:15.946352 11985 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260923 22:13:15.947500 11985 hybrid_clock.cc:648] HybridClock initialized: now 1790201595947487 us; error 29 us; skew 500 ppm
Sep 23 22:13:15 dist-test-slave-s2fj krb5kdc[11744](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790201595, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.135.194@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260923 22:13:15.949895 11985 init.cc:377] Logged in from keytab as kudu/127.1.135.194@KRBTEST.COM (short username kudu)
I20260923 22:13:15.950484 11985 webserver.cc:460] Webserver started at http://127.1.135.194:35031/ using document root <none> and password file <none>
I20260923 22:13:15.950713 11985 fs_manager.cc:359] Metadata directory not provided
I20260923 22:13:15.950779 11985 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260923 22:13:15.950927 11985 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260923 22:13:15.951874 11985 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskRiQe_a/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790201539513534-1567-0/minicluster-data/ts-1/data/instance:
uuid: "560723db019242f5bf9eb64143be9386"
format_stamp: "Formatted at 2026-09-23 22:13:15 on dist-test-slave-s2fj"
server_key: "9aef4959cc570ad5b3d4bd9cd92e1bca"
server_key_iv: "4fb0a620cd463902fd5261daa0236e41"
server_key_version: "encryptionkey@0"
I20260923 22:13:15.952241 11985 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskRiQe_a/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790201539513534-1567-0/minicluster-data/ts-1/wal/instance:
uuid: "560723db019242f5bf9eb64143be9386"
format_stamp: "Formatted at 2026-09-23 22:13:15 on dist-test-slave-s2fj"
server_key: "9aef4959cc570ad5b3d4bd9cd92e1bca"
server_key_iv: "4fb0a620cd463902fd5261daa0236e41"
server_key_version: "encryptionkey@0"
I20260923 22:13:15.953491 11985 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.001s
I20260923 22:13:15.954188 12000 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260923 22:13:15.954358 11985 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.001s
I20260923 22:13:15.954454 11985 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskRiQe_a/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790201539513534-1567-0/minicluster-data/ts-1/data,/tmp/dist-test-taskRiQe_a/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790201539513534-1567-0/minicluster-data/ts-1/wal
uuid: "560723db019242f5bf9eb64143be9386"
format_stamp: "Formatted at 2026-09-23 22:13:15 on dist-test-slave-s2fj"
server_key: "9aef4959cc570ad5b3d4bd9cd92e1bca"
server_key_iv: "4fb0a620cd463902fd5261daa0236e41"
server_key_version: "encryptionkey@0"
I20260923 22:13:15.954535 11985 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskRiQe_a/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790201539513534-1567-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskRiQe_a/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790201539513534-1567-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskRiQe_a/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790201539513534-1567-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260923 22:13:15.970353 11985 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260923 22:13:15.972674 11985 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260923 22:13:15.972826 11985 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260923 22:13:15.973074 11985 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260923 22:13:15.973367 11985 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260923 22:13:15.973420 11985 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260923 22:13:15.973462 11985 ts_tablet_manager.cc:610] Registered 0 tablets
I20260923 22:13:15.973490 11985 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260923 22:13:15.979132 11985 rpc_server.cc:307] RPC server started. Bound to: 127.1.135.194:45271
I20260923 22:13:15.979184 12113 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.1.135.194:45271 every 8 connection(s)
I20260923 22:13:15.979497 11985 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskRiQe_a/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790201539513534-1567-0/minicluster-data/ts-1/data/info.pb
I20260923 22:13:15.982440 1567 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskRiQe_a/build/release/bin/kudu as pid 11985
I20260923 22:13:15.982549 1567 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskRiQe_a/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790201539513534-1567-0/minicluster-data/ts-1/wal/instance
I20260923 22:13:15.982754 1567 external_mini_cluster.cc:1428] Setting key b0c56373e67d20ff99fe97b6f30431e0
Sep 23 22:13:15 dist-test-slave-s2fj krb5kdc[11744](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790201595, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.135.194@KRBTEST.COM for kudu/127.1.135.254@KRBTEST.COM
I20260923 22:13:15.989270 12116 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 22:13:15.980174 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.135.254:45027 (local address 127.1.135.194:38493)
0923 22:13:15.980318 (+ 144us) negotiation.cc:107] Waiting for socket to connect
0923 22:13:15.980351 (+ 33us) client_negotiation.cc:174] Beginning negotiation
0923 22:13:15.980803 (+ 452us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0923 22:13:15.980995 (+ 192us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0923 22:13:15.980999 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0923 22:13:15.981550 (+ 551us) client_negotiation.cc:189] Negotiated authn=SASL
0923 22:13:15.981729 (+ 179us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 22:13:15.981734 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 22:13:15.982375 (+ 641us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 22:13:15.982376 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 22:13:15.983579 (+ 1203us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 22:13:15.983581 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 22:13:15.983919 (+ 338us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 22:13:15.983920 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 22:13:15.984112 (+ 192us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 22:13:15.984555 (+ 443us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0923 22:13:15.984564 (+ 9us) client_negotiation.cc:656] calling sasl_client_start()
0923 22:13:15.986048 (+ 1484us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0923 22:13:15.987966 (+ 1918us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0923 22:13:15.987972 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0923 22:13:15.987973 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0923 22:13:15.988237 (+ 264us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0923 22:13:15.988447 (+ 210us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0923 22:13:15.988447 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0923 22:13:15.988447 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0923 22:13:15.988545 (+ 98us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0923 22:13:15.988837 (+ 292us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0923 22:13:15.988841 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0923 22:13:15.988940 (+ 99us) client_negotiation.cc:769] Sending connection context
0923 22:13:15.989012 (+ 72us) client_negotiation.cc:240] Negotiation successful
0923 22:13:15.989122 (+ 110us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":110,"thread_start_us":81,"threads_started":1}
I20260923 22:13:15.989272 11829 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 22:13:15.980168 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.135.194:38493 (local address 127.1.135.254:45027)
0923 22:13:15.980225 (+ 57us) server_negotiation.cc:206] Beginning negotiation
0923 22:13:15.980227 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0923 22:13:15.980435 (+ 208us) server_negotiation.cc:407] Connection header received
0923 22:13:15.980879 (+ 444us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 22:13:15.980880 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 22:13:15.980890 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 22:13:15.980937 (+ 47us) server_negotiation.cc:226] Negotiated authn=SASL
0923 22:13:15.981800 (+ 863us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 22:13:15.982286 (+ 486us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 22:13:15.983652 (+ 1366us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 22:13:15.983830 (+ 178us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 22:13:15.983883 (+ 53us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 22:13:15.986132 (+ 2249us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0923 22:13:15.986134 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0923 22:13:15.986134 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0923 22:13:15.986142 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0923 22:13:15.987858 (+ 1716us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0923 22:13:15.988306 (+ 448us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0923 22:13:15.988308 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0923 22:13:15.988308 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0923 22:13:15.988374 (+ 66us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0923 22:13:15.988619 (+ 245us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0923 22:13:15.988621 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0923 22:13:15.988621 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0923 22:13:15.988770 (+ 149us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0923 22:13:15.988847 (+ 77us) server_negotiation.cc:1035] Waiting for connection context
0923 22:13:15.989081 (+ 234us) server_negotiation.cc:299] Negotiation successful
0923 22:13:15.989169 (+ 88us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":34}
I20260923 22:13:15.989774 12114 heartbeater.cc:347] Connected to a master server at 127.1.135.254:45027
I20260923 22:13:15.989885 12114 heartbeater.cc:464] Registering TS with master...
I20260923 22:13:15.990061 12114 heartbeater.cc:511] Master 127.1.135.254:45027 requested a full tablet report, sending...
I20260923 22:13:15.990422 11789 ts_manager.cc:194] Registered new tserver with Master: 560723db019242f5bf9eb64143be9386 (127.1.135.194:45271)
I20260923 22:13:15.990953 11789 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.1.135.194@KRBTEST.COM'} at 127.1.135.194:38493
WARNING: no policy specified for kudu/127.1.135.195@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.1.135.195@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.1.135.195 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskRiQe_a/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790201539513534-1567-0/krb5kdc/kudu_127.1.135.195.keytab.
Entry for principal kudu/127.1.135.195 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskRiQe_a/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790201539513534-1567-0/krb5kdc/kudu_127.1.135.195.keytab.
I20260923 22:13:16.007153 1567 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskRiQe_a/build/release/bin/kudu
/tmp/dist-test-taskRiQe_a/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskRiQe_a/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790201539513534-1567-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskRiQe_a/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790201539513534-1567-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskRiQe_a/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790201539513534-1567-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskRiQe_a/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790201539513534-1567-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.1.135.195:0
--local_ip_for_outbound_sockets=127.1.135.195
--webserver_interface=127.1.135.195
--webserver_port=0
--tserver_master_addrs=127.1.135.254:45027
--keytab_file=/tmp/dist-test-taskRiQe_a/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790201539513534-1567-0/krb5kdc/kudu_127.1.135.195.keytab
--principal=kudu/127.1.135.195
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.1.135.212:35883
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskRiQe_a/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790201539513534-1567-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskRiQe_a/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790201539513534-1567-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskRiQe_a/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790201539513534-1567-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260923 22:13:16.085489 12119 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260923 22:13:16.085717 12119 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260923 22:13:16.085754 12119 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260923 22:13:16.086874 12119 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260923 22:13:16.086941 12119 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260923 22:13:16.086980 12119 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.1.135.195
I20260923 22:13:16.088394 12119 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.1.135.212:35883
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskRiQe_a/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790201539513534-1567-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskRiQe_a/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790201539513534-1567-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskRiQe_a/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790201539513534-1567-0/krb5kdc/kudu_127.1.135.195.keytab
--principal=kudu/127.1.135.195
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.1.135.195:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskRiQe_a/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790201539513534-1567-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.1.135.195
--webserver_port=0
--tserver_master_addrs=127.1.135.254:45027
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.12119
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.1.135.195
--log_dir=/tmp/dist-test-taskRiQe_a/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790201539513534-1567-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 23 Sep 2026 20:53:37 UTC on ccycloud.quasar-rkviyx.root.comops.site
build id 4668083
I20260923 22:13:16.088636 12119 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260923 22:13:16.088842 12119 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260923 22:13:16.093698 12125 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260923 22:13:16.093678 12127 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260923 22:13:16.093648 12124 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260923 22:13:16.093845 12119 server_base.cc:1034] running on GCE node
I20260923 22:13:16.094048 12119 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260923 22:13:16.094281 12119 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260923 22:13:16.095459 12119 hybrid_clock.cc:648] HybridClock initialized: now 1790201596095443 us; error 28 us; skew 500 ppm
Sep 23 22:13:16 dist-test-slave-s2fj krb5kdc[11744](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790201596, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.135.195@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260923 22:13:16.097721 12119 init.cc:377] Logged in from keytab as kudu/127.1.135.195@KRBTEST.COM (short username kudu)
I20260923 22:13:16.098173 12119 webserver.cc:460] Webserver started at http://127.1.135.195:42531/ using document root <none> and password file <none>
I20260923 22:13:16.098368 12119 fs_manager.cc:359] Metadata directory not provided
I20260923 22:13:16.098428 12119 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260923 22:13:16.098583 12119 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260923 22:13:16.099539 12119 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskRiQe_a/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790201539513534-1567-0/minicluster-data/ts-2/data/instance:
uuid: "e61d269633ad49d69f33ef43b2c458ff"
format_stamp: "Formatted at 2026-09-23 22:13:16 on dist-test-slave-s2fj"
server_key: "562a20cce8df54ae6dd85a2e748cccad"
server_key_iv: "70660b95466cbd4892b8671e5da7ab46"
server_key_version: "encryptionkey@0"
I20260923 22:13:16.099917 12119 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskRiQe_a/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790201539513534-1567-0/minicluster-data/ts-2/wal/instance:
uuid: "e61d269633ad49d69f33ef43b2c458ff"
format_stamp: "Formatted at 2026-09-23 22:13:16 on dist-test-slave-s2fj"
server_key: "562a20cce8df54ae6dd85a2e748cccad"
server_key_iv: "70660b95466cbd4892b8671e5da7ab46"
server_key_version: "encryptionkey@0"
I20260923 22:13:16.101186 12119 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.001s
I20260923 22:13:16.101933 12134 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260923 22:13:16.102102 12119 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.001s
I20260923 22:13:16.102193 12119 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskRiQe_a/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790201539513534-1567-0/minicluster-data/ts-2/data,/tmp/dist-test-taskRiQe_a/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790201539513534-1567-0/minicluster-data/ts-2/wal
uuid: "e61d269633ad49d69f33ef43b2c458ff"
format_stamp: "Formatted at 2026-09-23 22:13:16 on dist-test-slave-s2fj"
server_key: "562a20cce8df54ae6dd85a2e748cccad"
server_key_iv: "70660b95466cbd4892b8671e5da7ab46"
server_key_version: "encryptionkey@0"
I20260923 22:13:16.102270 12119 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskRiQe_a/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790201539513534-1567-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskRiQe_a/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790201539513534-1567-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskRiQe_a/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790201539513534-1567-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260923 22:13:16.123451 12119 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260923 22:13:16.125265 12119 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260923 22:13:16.125402 12119 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260923 22:13:16.125617 12119 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260923 22:13:16.125931 12119 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260923 22:13:16.125986 12119 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260923 22:13:16.126034 12119 ts_tablet_manager.cc:610] Registered 0 tablets
I20260923 22:13:16.126062 12119 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260923 22:13:16.131466 12119 rpc_server.cc:307] RPC server started. Bound to: 127.1.135.195:44153
I20260923 22:13:16.131533 12247 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.1.135.195:44153 every 8 connection(s)
I20260923 22:13:16.131834 12119 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskRiQe_a/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790201539513534-1567-0/minicluster-data/ts-2/data/info.pb
Sep 23 22:13:16 dist-test-slave-s2fj krb5kdc[11744](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790201596, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.135.195@KRBTEST.COM for kudu/127.1.135.254@KRBTEST.COM
I20260923 22:13:16.140316 11829 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 22:13:16.132546 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.135.195:49959 (local address 127.1.135.254:45027)
0923 22:13:16.132624 (+ 78us) server_negotiation.cc:206] Beginning negotiation
0923 22:13:16.132626 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0923 22:13:16.132772 (+ 146us) server_negotiation.cc:407] Connection header received
0923 22:13:16.133250 (+ 478us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 22:13:16.133251 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 22:13:16.133261 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 22:13:16.133312 (+ 51us) server_negotiation.cc:226] Negotiated authn=SASL
0923 22:13:16.134132 (+ 820us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 22:13:16.134605 (+ 473us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 22:13:16.135557 (+ 952us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 22:13:16.135753 (+ 196us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 22:13:16.135803 (+ 50us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 22:13:16.137792 (+ 1989us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0923 22:13:16.137793 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0923 22:13:16.137794 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0923 22:13:16.137799 (+ 5us) server_negotiation.cc:922] calling sasl_server_start()
0923 22:13:16.139214 (+ 1415us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0923 22:13:16.139534 (+ 320us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0923 22:13:16.139534 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0923 22:13:16.139535 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0923 22:13:16.139581 (+ 46us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0923 22:13:16.139779 (+ 198us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0923 22:13:16.139780 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0923 22:13:16.139780 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0923 22:13:16.139870 (+ 90us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0923 22:13:16.139918 (+ 48us) server_negotiation.cc:1035] Waiting for connection context
0923 22:13:16.140198 (+ 280us) server_negotiation.cc:299] Negotiation successful
0923 22:13:16.140256 (+ 58us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":48}
I20260923 22:13:16.140479 12250 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 22:13:16.132489 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.135.254:45027 (local address 127.1.135.195:49959)
0923 22:13:16.132667 (+ 178us) negotiation.cc:107] Waiting for socket to connect
0923 22:13:16.132698 (+ 31us) client_negotiation.cc:174] Beginning negotiation
0923 22:13:16.133172 (+ 474us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0923 22:13:16.133360 (+ 188us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0923 22:13:16.133363 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0923 22:13:16.133883 (+ 520us) client_negotiation.cc:189] Negotiated authn=SASL
0923 22:13:16.134062 (+ 179us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 22:13:16.134067 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 22:13:16.134678 (+ 611us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 22:13:16.134679 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 22:13:16.135491 (+ 812us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 22:13:16.135493 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 22:13:16.135818 (+ 325us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 22:13:16.135818 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 22:13:16.135915 (+ 97us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 22:13:16.136344 (+ 429us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0923 22:13:16.136351 (+ 7us) client_negotiation.cc:656] calling sasl_client_start()
0923 22:13:16.137717 (+ 1366us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0923 22:13:16.139279 (+ 1562us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0923 22:13:16.139283 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0923 22:13:16.139285 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0923 22:13:16.139472 (+ 187us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0923 22:13:16.139637 (+ 165us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0923 22:13:16.139638 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0923 22:13:16.139638 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0923 22:13:16.139727 (+ 89us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0923 22:13:16.139922 (+ 195us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0923 22:13:16.139925 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0923 22:13:16.140022 (+ 97us) client_negotiation.cc:769] Sending connection context
0923 22:13:16.140182 (+ 160us) client_negotiation.cc:240] Negotiation successful
0923 22:13:16.140293 (+ 111us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":145,"thread_start_us":95,"threads_started":1}
I20260923 22:13:16.140956 12248 heartbeater.cc:347] Connected to a master server at 127.1.135.254:45027
I20260923 22:13:16.141151 12248 heartbeater.cc:464] Registering TS with master...
I20260923 22:13:16.141324 12248 heartbeater.cc:511] Master 127.1.135.254:45027 requested a full tablet report, sending...
I20260923 22:13:16.141376 1567 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskRiQe_a/build/release/bin/kudu as pid 12119
I20260923 22:13:16.141510 1567 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskRiQe_a/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790201539513534-1567-0/minicluster-data/ts-2/wal/instance
I20260923 22:13:16.141696 11789 ts_manager.cc:194] Registered new tserver with Master: e61d269633ad49d69f33ef43b2c458ff (127.1.135.195:44153)
I20260923 22:13:16.141718 1567 external_mini_cluster.cc:1428] Setting key 7c000ae6c2f57e8447f270045ea6e687
I20260923 22:13:16.142098 11789 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.1.135.195@KRBTEST.COM'} at 127.1.135.195:49959
I20260923 22:13:16.142477 1567 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260923 22:13:16.149490 11829 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 22:13:16.143911 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53228 (local address 127.1.135.254:45027)
0923 22:13:16.143988 (+ 77us) server_negotiation.cc:206] Beginning negotiation
0923 22:13:16.143990 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0923 22:13:16.144131 (+ 141us) server_negotiation.cc:407] Connection header received
0923 22:13:16.144259 (+ 128us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 22:13:16.144260 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 22:13:16.144268 (+ 8us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 22:13:16.144323 (+ 55us) server_negotiation.cc:226] Negotiated authn=SASL
0923 22:13:16.144845 (+ 522us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 22:13:16.145279 (+ 434us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 22:13:16.145926 (+ 647us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 22:13:16.146083 (+ 157us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 22:13:16.146127 (+ 44us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 22:13:16.147258 (+ 1131us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0923 22:13:16.147259 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0923 22:13:16.147259 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0923 22:13:16.147265 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0923 22:13:16.148576 (+ 1311us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0923 22:13:16.148847 (+ 271us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0923 22:13:16.148847 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0923 22:13:16.148848 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0923 22:13:16.148916 (+ 68us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0923 22:13:16.149098 (+ 182us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0923 22:13:16.149098 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0923 22:13:16.149098 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0923 22:13:16.149200 (+ 102us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0923 22:13:16.149265 (+ 65us) server_negotiation.cc:1035] Waiting for connection context
0923 22:13:16.149350 (+ 85us) server_negotiation.cc:299] Negotiation successful
0923 22:13:16.149407 (+ 57us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":56}
I20260923 22:13:16.150931 11789 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:53228:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260923 22:13:16.152513 11789 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260923 22:13:16.158392 12257 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 22:13:16.154494 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.135.194:45271 (local address 127.0.0.1:54402)
0923 22:13:16.154768 (+ 274us) negotiation.cc:107] Waiting for socket to connect
0923 22:13:16.154776 (+ 8us) client_negotiation.cc:174] Beginning negotiation
0923 22:13:16.154821 (+ 45us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0923 22:13:16.155083 (+ 262us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0923 22:13:16.155086 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0923 22:13:16.155088 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0923 22:13:16.155146 (+ 58us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 22:13:16.155147 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 22:13:16.156053 (+ 906us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 22:13:16.156054 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 22:13:16.157021 (+ 967us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 22:13:16.157024 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 22:13:16.158103 (+ 1079us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 22:13:16.158104 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 22:13:16.158196 (+ 92us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 22:13:16.158208 (+ 12us) client_negotiation.cc:769] Sending connection context
0923 22:13:16.158271 (+ 63us) client_negotiation.cc:240] Negotiation successful
0923 22:13:16.158320 (+ 49us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":263,"thread_start_us":64,"threads_started":1}
I20260923 22:13:16.158635 12258 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 22:13:16.154480 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.135.195:44153 (local address 127.0.0.1:44394)
0923 22:13:16.154707 (+ 227us) negotiation.cc:107] Waiting for socket to connect
0923 22:13:16.154717 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0923 22:13:16.154769 (+ 52us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0923 22:13:16.155319 (+ 550us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0923 22:13:16.155320 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0923 22:13:16.155322 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0923 22:13:16.155352 (+ 30us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 22:13:16.155354 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 22:13:16.156640 (+ 1286us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 22:13:16.156640 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 22:13:16.157656 (+ 1016us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 22:13:16.157658 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 22:13:16.158431 (+ 773us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 22:13:16.158432 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 22:13:16.158495 (+ 63us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 22:13:16.158499 (+ 4us) client_negotiation.cc:769] Sending connection context
0923 22:13:16.158529 (+ 30us) client_negotiation.cc:240] Negotiation successful
0923 22:13:16.158567 (+ 38us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":206,"thread_start_us":58,"threads_started":1}
I20260923 22:13:16.158645 12260 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 22:13:16.154723 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54402 (local address 127.1.135.194:45271)
0923 22:13:16.154917 (+ 194us) server_negotiation.cc:206] Beginning negotiation
0923 22:13:16.154921 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0923 22:13:16.154931 (+ 10us) server_negotiation.cc:407] Connection header received
0923 22:13:16.154969 (+ 38us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 22:13:16.154971 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 22:13:16.154982 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 22:13:16.155077 (+ 95us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0923 22:13:16.155201 (+ 124us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 22:13:16.155950 (+ 749us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 22:13:16.157239 (+ 1289us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 22:13:16.157955 (+ 716us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 22:13:16.158019 (+ 64us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 22:13:16.158073 (+ 54us) server_negotiation.cc:1035] Waiting for connection context
0923 22:13:16.158525 (+ 452us) server_negotiation.cc:299] Negotiation successful
0923 22:13:16.158568 (+ 43us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":157,"thread_start_us":98,"threads_started":1}
I20260923 22:13:16.159075 12259 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 22:13:16.154625 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44394 (local address 127.1.135.195:44153)
0923 22:13:16.155133 (+ 508us) server_negotiation.cc:206] Beginning negotiation
0923 22:13:16.155144 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0923 22:13:16.155152 (+ 8us) server_negotiation.cc:407] Connection header received
0923 22:13:16.155181 (+ 29us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 22:13:16.155183 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 22:13:16.155196 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 22:13:16.155278 (+ 82us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0923 22:13:16.155476 (+ 198us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 22:13:16.156532 (+ 1056us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 22:13:16.157730 (+ 1198us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 22:13:16.158342 (+ 612us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 22:13:16.158864 (+ 522us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 22:13:16.158929 (+ 65us) server_negotiation.cc:1035] Waiting for connection context
0923 22:13:16.158957 (+ 28us) server_negotiation.cc:299] Negotiation successful
0923 22:13:16.159003 (+ 46us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":482,"thread_start_us":76,"threads_started":1}
I20260923 22:13:16.159415 12048 tablet_service.cc:1467] Processing CreateTablet for tablet cdc3cc49ada04a2c9aae9e2cc0788d42 (DEFAULT_TABLE table=test-table [id=13d956b57819415db20b3b91f9ee8294]), partition=RANGE (key) PARTITION UNBOUNDED
I20260923 22:13:16.159782 12048 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet cdc3cc49ada04a2c9aae9e2cc0788d42. 1 dirs total, 0 dirs full, 0 dirs failed
I20260923 22:13:16.159716 12182 tablet_service.cc:1467] Processing CreateTablet for tablet cdc3cc49ada04a2c9aae9e2cc0788d42 (DEFAULT_TABLE table=test-table [id=13d956b57819415db20b3b91f9ee8294]), partition=RANGE (key) PARTITION UNBOUNDED
I20260923 22:13:16.159968 12182 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet cdc3cc49ada04a2c9aae9e2cc0788d42. 1 dirs total, 0 dirs full, 0 dirs failed
I20260923 22:13:16.162341 12262 tablet_bootstrap.cc:492] T cdc3cc49ada04a2c9aae9e2cc0788d42 P 560723db019242f5bf9eb64143be9386: Bootstrap starting.
I20260923 22:13:16.162606 12256 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 22:13:16.154476 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.135.193:39785 (local address 127.0.0.1:41020)
0923 22:13:16.154580 (+ 104us) negotiation.cc:107] Waiting for socket to connect
0923 22:13:16.154614 (+ 34us) client_negotiation.cc:174] Beginning negotiation
0923 22:13:16.154675 (+ 61us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0923 22:13:16.155370 (+ 695us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0923 22:13:16.155371 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0923 22:13:16.155373 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0923 22:13:16.155506 (+ 133us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 22:13:16.155509 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 22:13:16.156775 (+ 1266us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 22:13:16.156776 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 22:13:16.157917 (+ 1141us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 22:13:16.157920 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 22:13:16.162344 (+ 4424us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 22:13:16.162345 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 22:13:16.162409 (+ 64us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 22:13:16.162412 (+ 3us) client_negotiation.cc:769] Sending connection context
0923 22:13:16.162465 (+ 53us) client_negotiation.cc:240] Negotiation successful
0923 22:13:16.162533 (+ 68us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":83,"mutex_wait_us":7,"thread_start_us":66,"threads_started":1}
I20260923 22:13:16.162693 12263 tablet_bootstrap.cc:492] T cdc3cc49ada04a2c9aae9e2cc0788d42 P e61d269633ad49d69f33ef43b2c458ff: Bootstrap starting.
I20260923 22:13:16.162907 12261 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 22:13:16.154944 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41020 (local address 127.1.135.193:39785)
0923 22:13:16.155094 (+ 150us) server_negotiation.cc:206] Beginning negotiation
0923 22:13:16.155099 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0923 22:13:16.155110 (+ 11us) server_negotiation.cc:407] Connection header received
0923 22:13:16.155150 (+ 40us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 22:13:16.155152 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 22:13:16.155274 (+ 122us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 22:13:16.155590 (+ 316us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0923 22:13:16.155662 (+ 72us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 22:13:16.156686 (+ 1024us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 22:13:16.158032 (+ 1346us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 22:13:16.158949 (+ 917us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 22:13:16.159017 (+ 68us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 22:13:16.159064 (+ 47us) server_negotiation.cc:1035] Waiting for connection context
0923 22:13:16.162812 (+ 3748us) server_negotiation.cc:299] Negotiation successful
0923 22:13:16.162846 (+ 34us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":116,"thread_start_us":82,"threads_started":1}
I20260923 22:13:16.163069 12262 tablet_bootstrap.cc:654] T cdc3cc49ada04a2c9aae9e2cc0788d42 P 560723db019242f5bf9eb64143be9386: Neither blocks nor log segments found. Creating new log.
I20260923 22:13:16.163324 12263 tablet_bootstrap.cc:654] T cdc3cc49ada04a2c9aae9e2cc0788d42 P e61d269633ad49d69f33ef43b2c458ff: Neither blocks nor log segments found. Creating new log.
I20260923 22:13:16.163621 11914 tablet_service.cc:1467] Processing CreateTablet for tablet cdc3cc49ada04a2c9aae9e2cc0788d42 (DEFAULT_TABLE table=test-table [id=13d956b57819415db20b3b91f9ee8294]), partition=RANGE (key) PARTITION UNBOUNDED
I20260923 22:13:16.163698 12262 log.cc:824] T cdc3cc49ada04a2c9aae9e2cc0788d42 P 560723db019242f5bf9eb64143be9386: Log is configured to *not* fsync() on all Append() calls
I20260923 22:13:16.163921 11914 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet cdc3cc49ada04a2c9aae9e2cc0788d42. 1 dirs total, 0 dirs full, 0 dirs failed
I20260923 22:13:16.163961 12263 log.cc:824] T cdc3cc49ada04a2c9aae9e2cc0788d42 P e61d269633ad49d69f33ef43b2c458ff: Log is configured to *not* fsync() on all Append() calls
I20260923 22:13:16.164458 12262 tablet_bootstrap.cc:492] T cdc3cc49ada04a2c9aae9e2cc0788d42 P 560723db019242f5bf9eb64143be9386: No bootstrap required, opened a new log
I20260923 22:13:16.164546 12262 ts_tablet_manager.cc:1397] T cdc3cc49ada04a2c9aae9e2cc0788d42 P 560723db019242f5bf9eb64143be9386: Time spent bootstrapping tablet: real 0.002s user 0.002s sys 0.000s
I20260923 22:13:16.165807 12262 raft_consensus.cc:348] T cdc3cc49ada04a2c9aae9e2cc0788d42 P 560723db019242f5bf9eb64143be9386 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "893017a9339b4fe5a64028d7f1b7ff21" member_type: VOTER last_known_addr { host: "127.1.135.193" port: 39785 } } peers { permanent_uuid: "e61d269633ad49d69f33ef43b2c458ff" member_type: VOTER last_known_addr { host: "127.1.135.195" port: 44153 } } peers { permanent_uuid: "560723db019242f5bf9eb64143be9386" member_type: VOTER last_known_addr { host: "127.1.135.194" port: 45271 } }
I20260923 22:13:16.165999 12262 raft_consensus.cc:374] T cdc3cc49ada04a2c9aae9e2cc0788d42 P 560723db019242f5bf9eb64143be9386 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260923 22:13:16.166045 12262 raft_consensus.cc:729] T cdc3cc49ada04a2c9aae9e2cc0788d42 P 560723db019242f5bf9eb64143be9386 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 560723db019242f5bf9eb64143be9386, State: Initialized, Role: FOLLOWER
I20260923 22:13:16.166153 12262 consensus_queue.cc:260] T cdc3cc49ada04a2c9aae9e2cc0788d42 P 560723db019242f5bf9eb64143be9386 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "893017a9339b4fe5a64028d7f1b7ff21" member_type: VOTER last_known_addr { host: "127.1.135.193" port: 39785 } } peers { permanent_uuid: "e61d269633ad49d69f33ef43b2c458ff" member_type: VOTER last_known_addr { host: "127.1.135.195" port: 44153 } } peers { permanent_uuid: "560723db019242f5bf9eb64143be9386" member_type: VOTER last_known_addr { host: "127.1.135.194" port: 45271 } }
I20260923 22:13:16.166443 12262 ts_tablet_manager.cc:1428] T cdc3cc49ada04a2c9aae9e2cc0788d42 P 560723db019242f5bf9eb64143be9386: Time spent starting tablet: real 0.002s user 0.002s sys 0.000s
I20260923 22:13:16.166468 12114 heartbeater.cc:503] Master 127.1.135.254:45027 was elected leader, sending a full tablet report...
I20260923 22:13:16.166721 12267 tablet_bootstrap.cc:492] T cdc3cc49ada04a2c9aae9e2cc0788d42 P 893017a9339b4fe5a64028d7f1b7ff21: Bootstrap starting.
I20260923 22:13:16.167037 12263 tablet_bootstrap.cc:492] T cdc3cc49ada04a2c9aae9e2cc0788d42 P e61d269633ad49d69f33ef43b2c458ff: No bootstrap required, opened a new log
I20260923 22:13:16.167109 12263 ts_tablet_manager.cc:1397] T cdc3cc49ada04a2c9aae9e2cc0788d42 P e61d269633ad49d69f33ef43b2c458ff: Time spent bootstrapping tablet: real 0.004s user 0.002s sys 0.000s
I20260923 22:13:16.167403 12267 tablet_bootstrap.cc:654] T cdc3cc49ada04a2c9aae9e2cc0788d42 P 893017a9339b4fe5a64028d7f1b7ff21: Neither blocks nor log segments found. Creating new log.
I20260923 22:13:16.167944 12267 log.cc:824] T cdc3cc49ada04a2c9aae9e2cc0788d42 P 893017a9339b4fe5a64028d7f1b7ff21: Log is configured to *not* fsync() on all Append() calls
I20260923 22:13:16.168643 12267 tablet_bootstrap.cc:492] T cdc3cc49ada04a2c9aae9e2cc0788d42 P 893017a9339b4fe5a64028d7f1b7ff21: No bootstrap required, opened a new log
I20260923 22:13:16.168720 12267 ts_tablet_manager.cc:1397] T cdc3cc49ada04a2c9aae9e2cc0788d42 P 893017a9339b4fe5a64028d7f1b7ff21: Time spent bootstrapping tablet: real 0.002s user 0.000s sys 0.001s
I20260923 22:13:16.169042 12263 raft_consensus.cc:348] T cdc3cc49ada04a2c9aae9e2cc0788d42 P e61d269633ad49d69f33ef43b2c458ff [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "893017a9339b4fe5a64028d7f1b7ff21" member_type: VOTER last_known_addr { host: "127.1.135.193" port: 39785 } } peers { permanent_uuid: "e61d269633ad49d69f33ef43b2c458ff" member_type: VOTER last_known_addr { host: "127.1.135.195" port: 44153 } } peers { permanent_uuid: "560723db019242f5bf9eb64143be9386" member_type: VOTER last_known_addr { host: "127.1.135.194" port: 45271 } }
I20260923 22:13:16.169158 12263 raft_consensus.cc:374] T cdc3cc49ada04a2c9aae9e2cc0788d42 P e61d269633ad49d69f33ef43b2c458ff [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260923 22:13:16.169179 12263 raft_consensus.cc:729] T cdc3cc49ada04a2c9aae9e2cc0788d42 P e61d269633ad49d69f33ef43b2c458ff [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e61d269633ad49d69f33ef43b2c458ff, State: Initialized, Role: FOLLOWER
I20260923 22:13:16.169291 12263 consensus_queue.cc:260] T cdc3cc49ada04a2c9aae9e2cc0788d42 P e61d269633ad49d69f33ef43b2c458ff [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "893017a9339b4fe5a64028d7f1b7ff21" member_type: VOTER last_known_addr { host: "127.1.135.193" port: 39785 } } peers { permanent_uuid: "e61d269633ad49d69f33ef43b2c458ff" member_type: VOTER last_known_addr { host: "127.1.135.195" port: 44153 } } peers { permanent_uuid: "560723db019242f5bf9eb64143be9386" member_type: VOTER last_known_addr { host: "127.1.135.194" port: 45271 } }
I20260923 22:13:16.169572 12263 ts_tablet_manager.cc:1428] T cdc3cc49ada04a2c9aae9e2cc0788d42 P e61d269633ad49d69f33ef43b2c458ff: Time spent starting tablet: real 0.002s user 0.002s sys 0.000s
I20260923 22:13:16.169592 12248 heartbeater.cc:503] Master 127.1.135.254:45027 was elected leader, sending a full tablet report...
I20260923 22:13:16.170570 12267 raft_consensus.cc:348] T cdc3cc49ada04a2c9aae9e2cc0788d42 P 893017a9339b4fe5a64028d7f1b7ff21 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "893017a9339b4fe5a64028d7f1b7ff21" member_type: VOTER last_known_addr { host: "127.1.135.193" port: 39785 } } peers { permanent_uuid: "e61d269633ad49d69f33ef43b2c458ff" member_type: VOTER last_known_addr { host: "127.1.135.195" port: 44153 } } peers { permanent_uuid: "560723db019242f5bf9eb64143be9386" member_type: VOTER last_known_addr { host: "127.1.135.194" port: 45271 } }
I20260923 22:13:16.170692 12267 raft_consensus.cc:374] T cdc3cc49ada04a2c9aae9e2cc0788d42 P 893017a9339b4fe5a64028d7f1b7ff21 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260923 22:13:16.170713 12267 raft_consensus.cc:729] T cdc3cc49ada04a2c9aae9e2cc0788d42 P 893017a9339b4fe5a64028d7f1b7ff21 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 893017a9339b4fe5a64028d7f1b7ff21, State: Initialized, Role: FOLLOWER
I20260923 22:13:16.170814 12267 consensus_queue.cc:260] T cdc3cc49ada04a2c9aae9e2cc0788d42 P 893017a9339b4fe5a64028d7f1b7ff21 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "893017a9339b4fe5a64028d7f1b7ff21" member_type: VOTER last_known_addr { host: "127.1.135.193" port: 39785 } } peers { permanent_uuid: "e61d269633ad49d69f33ef43b2c458ff" member_type: VOTER last_known_addr { host: "127.1.135.195" port: 44153 } } peers { permanent_uuid: "560723db019242f5bf9eb64143be9386" member_type: VOTER last_known_addr { host: "127.1.135.194" port: 45271 } }
I20260923 22:13:16.171087 12267 ts_tablet_manager.cc:1428] T cdc3cc49ada04a2c9aae9e2cc0788d42 P 893017a9339b4fe5a64028d7f1b7ff21: Time spent starting tablet: real 0.002s user 0.000s sys 0.003s
I20260923 22:13:16.171099 11980 heartbeater.cc:503] Master 127.1.135.254:45027 was elected leader, sending a full tablet report...
W20260923 22:13:16.230034 12115 tablet.cc:2369] T cdc3cc49ada04a2c9aae9e2cc0788d42 P 560723db019242f5bf9eb64143be9386: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260923 22:13:16.329813 11981 tablet.cc:2369] T cdc3cc49ada04a2c9aae9e2cc0788d42 P 893017a9339b4fe5a64028d7f1b7ff21: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260923 22:13:16.359012 12269 raft_consensus.cc:482] T cdc3cc49ada04a2c9aae9e2cc0788d42 P e61d269633ad49d69f33ef43b2c458ff [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260923 22:13:16.359158 12269 raft_consensus.cc:504] T cdc3cc49ada04a2c9aae9e2cc0788d42 P e61d269633ad49d69f33ef43b2c458ff [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "893017a9339b4fe5a64028d7f1b7ff21" member_type: VOTER last_known_addr { host: "127.1.135.193" port: 39785 } } peers { permanent_uuid: "e61d269633ad49d69f33ef43b2c458ff" member_type: VOTER last_known_addr { host: "127.1.135.195" port: 44153 } } peers { permanent_uuid: "560723db019242f5bf9eb64143be9386" member_type: VOTER last_known_addr { host: "127.1.135.194" port: 45271 } }
I20260923 22:13:16.359562 12269 leader_election.cc:288] T cdc3cc49ada04a2c9aae9e2cc0788d42 P e61d269633ad49d69f33ef43b2c458ff [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 893017a9339b4fe5a64028d7f1b7ff21 (127.1.135.193:39785), 560723db019242f5bf9eb64143be9386 (127.1.135.194:45271)
I20260923 22:13:16.362425 12261 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 22:13:16.359812 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.135.195:50815 (local address 127.1.135.193:39785)
0923 22:13:16.359950 (+ 138us) server_negotiation.cc:206] Beginning negotiation
0923 22:13:16.359952 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0923 22:13:16.360025 (+ 73us) server_negotiation.cc:407] Connection header received
0923 22:13:16.360098 (+ 73us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 22:13:16.360099 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 22:13:16.360109 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 22:13:16.360169 (+ 60us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0923 22:13:16.360332 (+ 163us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 22:13:16.360803 (+ 471us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 22:13:16.361736 (+ 933us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 22:13:16.362081 (+ 345us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 22:13:16.362174 (+ 93us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 22:13:16.362185 (+ 11us) server_negotiation.cc:1035] Waiting for connection context
0923 22:13:16.362315 (+ 130us) server_negotiation.cc:299] Negotiation successful
0923 22:13:16.362358 (+ 43us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":113}
I20260923 22:13:16.362429 12271 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 22:13:16.359751 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.135.193:39785 (local address 127.1.135.195:50815)
0923 22:13:16.359931 (+ 180us) negotiation.cc:107] Waiting for socket to connect
0923 22:13:16.359941 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0923 22:13:16.360017 (+ 76us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0923 22:13:16.360201 (+ 184us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0923 22:13:16.360202 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0923 22:13:16.360203 (+ 1us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0923 22:13:16.360270 (+ 67us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 22:13:16.360271 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 22:13:16.360894 (+ 623us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 22:13:16.360895 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 22:13:16.361625 (+ 730us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 22:13:16.361626 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 22:13:16.362172 (+ 546us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 22:13:16.362173 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 22:13:16.362248 (+ 75us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 22:13:16.362252 (+ 4us) client_negotiation.cc:769] Sending connection context
0923 22:13:16.362293 (+ 41us) client_negotiation.cc:240] Negotiation successful
0923 22:13:16.362323 (+ 30us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":161,"thread_start_us":118,"threads_started":1}
I20260923 22:13:16.362509 12260 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 22:13:16.359797 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.135.195:52789 (local address 127.1.135.194:45271)
0923 22:13:16.359843 (+ 46us) server_negotiation.cc:206] Beginning negotiation
0923 22:13:16.359845 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0923 22:13:16.359857 (+ 12us) server_negotiation.cc:407] Connection header received
0923 22:13:16.359971 (+ 114us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 22:13:16.359972 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 22:13:16.359982 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 22:13:16.360040 (+ 58us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0923 22:13:16.360218 (+ 178us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 22:13:16.360742 (+ 524us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 22:13:16.361703 (+ 961us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 22:13:16.362085 (+ 382us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 22:13:16.362190 (+ 105us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 22:13:16.362198 (+ 8us) server_negotiation.cc:1035] Waiting for connection context
0923 22:13:16.362396 (+ 198us) server_negotiation.cc:299] Negotiation successful
0923 22:13:16.362438 (+ 42us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":21}
I20260923 22:13:16.362537 12250 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 22:13:16.359739 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.135.194:45271 (local address 127.1.135.195:52789)
0923 22:13:16.359805 (+ 66us) negotiation.cc:107] Waiting for socket to connect
0923 22:13:16.359818 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0923 22:13:16.359863 (+ 45us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0923 22:13:16.360046 (+ 183us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0923 22:13:16.360047 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0923 22:13:16.360049 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0923 22:13:16.360140 (+ 91us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 22:13:16.360142 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 22:13:16.360821 (+ 679us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 22:13:16.360822 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 22:13:16.361603 (+ 781us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 22:13:16.361605 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 22:13:16.362210 (+ 605us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 22:13:16.362211 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 22:13:16.362312 (+ 101us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 22:13:16.362315 (+ 3us) client_negotiation.cc:769] Sending connection context
0923 22:13:16.362370 (+ 55us) client_negotiation.cc:240] Negotiation successful
0923 22:13:16.362404 (+ 34us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":47}
I20260923 22:13:16.362823 12068 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "cdc3cc49ada04a2c9aae9e2cc0788d42" candidate_uuid: "e61d269633ad49d69f33ef43b2c458ff" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "560723db019242f5bf9eb64143be9386" is_pre_election: true
I20260923 22:13:16.362989 12068 raft_consensus.cc:2454] T cdc3cc49ada04a2c9aae9e2cc0788d42 P 560723db019242f5bf9eb64143be9386 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e61d269633ad49d69f33ef43b2c458ff in term 0.
I20260923 22:13:16.362950 11934 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "cdc3cc49ada04a2c9aae9e2cc0788d42" candidate_uuid: "e61d269633ad49d69f33ef43b2c458ff" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "893017a9339b4fe5a64028d7f1b7ff21" is_pre_election: true
I20260923 22:13:16.363109 11934 raft_consensus.cc:2454] T cdc3cc49ada04a2c9aae9e2cc0788d42 P 893017a9339b4fe5a64028d7f1b7ff21 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e61d269633ad49d69f33ef43b2c458ff in term 0.
I20260923 22:13:16.363196 12136 leader_election.cc:302] T cdc3cc49ada04a2c9aae9e2cc0788d42 P e61d269633ad49d69f33ef43b2c458ff [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 560723db019242f5bf9eb64143be9386, e61d269633ad49d69f33ef43b2c458ff; no voters:
I20260923 22:13:16.363327 12269 raft_consensus.cc:2781] T cdc3cc49ada04a2c9aae9e2cc0788d42 P e61d269633ad49d69f33ef43b2c458ff [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260923 22:13:16.363389 12269 raft_consensus.cc:482] T cdc3cc49ada04a2c9aae9e2cc0788d42 P e61d269633ad49d69f33ef43b2c458ff [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260923 22:13:16.363406 12269 raft_consensus.cc:3037] T cdc3cc49ada04a2c9aae9e2cc0788d42 P e61d269633ad49d69f33ef43b2c458ff [term 0 FOLLOWER]: Advancing to term 1
I20260923 22:13:16.364223 12269 raft_consensus.cc:504] T cdc3cc49ada04a2c9aae9e2cc0788d42 P e61d269633ad49d69f33ef43b2c458ff [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "893017a9339b4fe5a64028d7f1b7ff21" member_type: VOTER last_known_addr { host: "127.1.135.193" port: 39785 } } peers { permanent_uuid: "e61d269633ad49d69f33ef43b2c458ff" member_type: VOTER last_known_addr { host: "127.1.135.195" port: 44153 } } peers { permanent_uuid: "560723db019242f5bf9eb64143be9386" member_type: VOTER last_known_addr { host: "127.1.135.194" port: 45271 } }
I20260923 22:13:16.364352 12269 leader_election.cc:288] T cdc3cc49ada04a2c9aae9e2cc0788d42 P e61d269633ad49d69f33ef43b2c458ff [CANDIDATE]: Term 1 election: Requested vote from peers 893017a9339b4fe5a64028d7f1b7ff21 (127.1.135.193:39785), 560723db019242f5bf9eb64143be9386 (127.1.135.194:45271)
I20260923 22:13:16.364491 12068 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "cdc3cc49ada04a2c9aae9e2cc0788d42" candidate_uuid: "e61d269633ad49d69f33ef43b2c458ff" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "560723db019242f5bf9eb64143be9386"
I20260923 22:13:16.364552 12068 raft_consensus.cc:3037] T cdc3cc49ada04a2c9aae9e2cc0788d42 P 560723db019242f5bf9eb64143be9386 [term 0 FOLLOWER]: Advancing to term 1
I20260923 22:13:16.365284 12068 raft_consensus.cc:2454] T cdc3cc49ada04a2c9aae9e2cc0788d42 P 560723db019242f5bf9eb64143be9386 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e61d269633ad49d69f33ef43b2c458ff in term 1.
I20260923 22:13:16.365458 12136 leader_election.cc:302] T cdc3cc49ada04a2c9aae9e2cc0788d42 P e61d269633ad49d69f33ef43b2c458ff [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 560723db019242f5bf9eb64143be9386, e61d269633ad49d69f33ef43b2c458ff; no voters:
I20260923 22:13:16.365417 11934 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "cdc3cc49ada04a2c9aae9e2cc0788d42" candidate_uuid: "e61d269633ad49d69f33ef43b2c458ff" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "893017a9339b4fe5a64028d7f1b7ff21"
I20260923 22:13:16.365506 11934 raft_consensus.cc:3037] T cdc3cc49ada04a2c9aae9e2cc0788d42 P 893017a9339b4fe5a64028d7f1b7ff21 [term 0 FOLLOWER]: Advancing to term 1
I20260923 22:13:16.365547 12269 raft_consensus.cc:2781] T cdc3cc49ada04a2c9aae9e2cc0788d42 P e61d269633ad49d69f33ef43b2c458ff [term 1 FOLLOWER]: Leader election won for term 1
I20260923 22:13:16.365687 12269 raft_consensus.cc:686] T cdc3cc49ada04a2c9aae9e2cc0788d42 P e61d269633ad49d69f33ef43b2c458ff [term 1 LEADER]: Becoming Leader. State: Replica: e61d269633ad49d69f33ef43b2c458ff, State: Running, Role: LEADER
I20260923 22:13:16.365784 12269 consensus_queue.cc:237] T cdc3cc49ada04a2c9aae9e2cc0788d42 P e61d269633ad49d69f33ef43b2c458ff [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "893017a9339b4fe5a64028d7f1b7ff21" member_type: VOTER last_known_addr { host: "127.1.135.193" port: 39785 } } peers { permanent_uuid: "e61d269633ad49d69f33ef43b2c458ff" member_type: VOTER last_known_addr { host: "127.1.135.195" port: 44153 } } peers { permanent_uuid: "560723db019242f5bf9eb64143be9386" member_type: VOTER last_known_addr { host: "127.1.135.194" port: 45271 } }
I20260923 22:13:16.366410 11934 raft_consensus.cc:2454] T cdc3cc49ada04a2c9aae9e2cc0788d42 P 893017a9339b4fe5a64028d7f1b7ff21 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e61d269633ad49d69f33ef43b2c458ff in term 1.
I20260923 22:13:16.366865 11789 catalog_manager.cc:5495] T cdc3cc49ada04a2c9aae9e2cc0788d42 P e61d269633ad49d69f33ef43b2c458ff reported cstate change: term changed from 0 to 1, leader changed from <none> to e61d269633ad49d69f33ef43b2c458ff (127.1.135.195). New cstate: current_term: 1 leader_uuid: "e61d269633ad49d69f33ef43b2c458ff" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "893017a9339b4fe5a64028d7f1b7ff21" member_type: VOTER last_known_addr { host: "127.1.135.193" port: 39785 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "e61d269633ad49d69f33ef43b2c458ff" member_type: VOTER last_known_addr { host: "127.1.135.195" port: 44153 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "560723db019242f5bf9eb64143be9386" member_type: VOTER last_known_addr { host: "127.1.135.194" port: 45271 } health_report { overall_health: UNKNOWN } } }
W20260923 22:13:16.382340 12249 tablet.cc:2369] T cdc3cc49ada04a2c9aae9e2cc0788d42 P e61d269633ad49d69f33ef43b2c458ff: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260923 22:13:16.384011 12259 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 22:13:16.381923 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44406 (local address 127.1.135.195:44153)
0923 22:13:16.381976 (+ 53us) server_negotiation.cc:206] Beginning negotiation
0923 22:13:16.381978 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0923 22:13:16.381985 (+ 7us) server_negotiation.cc:407] Connection header received
0923 22:13:16.382030 (+ 45us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 22:13:16.382031 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 22:13:16.382044 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 22:13:16.382097 (+ 53us) server_negotiation.cc:226] Negotiated authn=TOKEN
0923 22:13:16.382213 (+ 116us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 22:13:16.382679 (+ 466us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 22:13:16.383344 (+ 665us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 22:13:16.383515 (+ 171us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 22:13:16.383582 (+ 67us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 22:13:16.383707 (+ 125us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0923 22:13:16.383789 (+ 82us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0923 22:13:16.383846 (+ 57us) server_negotiation.cc:1035] Waiting for connection context
0923 22:13:16.383890 (+ 44us) server_negotiation.cc:299] Negotiation successful
0923 22:13:16.383905 (+ 15us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":35}
I20260923 22:13:16.385677 11934 raft_consensus.cc:1261] T cdc3cc49ada04a2c9aae9e2cc0788d42 P 893017a9339b4fe5a64028d7f1b7ff21 [term 1 FOLLOWER]: Refusing update from remote peer e61d269633ad49d69f33ef43b2c458ff: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260923 22:13:16.385677 12068 raft_consensus.cc:1261] T cdc3cc49ada04a2c9aae9e2cc0788d42 P 560723db019242f5bf9eb64143be9386 [term 1 FOLLOWER]: Refusing update from remote peer e61d269633ad49d69f33ef43b2c458ff: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260923 22:13:16.386018 12272 consensus_queue.cc:1035] T cdc3cc49ada04a2c9aae9e2cc0788d42 P e61d269633ad49d69f33ef43b2c458ff [LEADER]: Connected to new peer: Peer: permanent_uuid: "893017a9339b4fe5a64028d7f1b7ff21" member_type: VOTER last_known_addr { host: "127.1.135.193" port: 39785 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260923 22:13:16.386147 12269 consensus_queue.cc:1035] T cdc3cc49ada04a2c9aae9e2cc0788d42 P e61d269633ad49d69f33ef43b2c458ff [LEADER]: Connected to new peer: Peer: permanent_uuid: "560723db019242f5bf9eb64143be9386" member_type: VOTER last_known_addr { host: "127.1.135.194" port: 45271 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260923 22:13:16.388813 12277 mvcc.cc:205] Tried to move back new op lower bound from 7332665738793222144 to 7332665738714882048. Current Snapshot: MvccSnapshot[applied={T|T < 7332665738793222144}]
I20260923 22:13:16.388803 12276 mvcc.cc:205] Tried to move back new op lower bound from 7332665738793222144 to 7332665738714882048. Current Snapshot: MvccSnapshot[applied={T|T < 7332665738793222144 or (T in {7332665738793222144})}]
I20260923 22:13:16.390901 11788 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:53228:
table { table_name: "test-table" } modify_external_catalogs: true
I20260923 22:13:16.391010 11788 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:53228:
table { table_name: "test-table" } modify_external_catalogs: true
I20260923 22:13:16.391696 11788 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 352c40227f674e1cb876aaa945ed8166: Sending DeleteTablet for 3 replicas of tablet cdc3cc49ada04a2c9aae9e2cc0788d42
I20260923 22:13:16.392002 12182 tablet_service.cc:1514] Processing DeleteTablet for tablet cdc3cc49ada04a2c9aae9e2cc0788d42 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-23 22:13:16 UTC) from {username='kudu', principal='kudu/127.1.135.254@KRBTEST.COM'} at 127.0.0.1:44394
I20260923 22:13:16.391997 12048 tablet_service.cc:1514] Processing DeleteTablet for tablet cdc3cc49ada04a2c9aae9e2cc0788d42 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-23 22:13:16 UTC) from {username='kudu', principal='kudu/127.1.135.254@KRBTEST.COM'} at 127.0.0.1:54402
I20260923 22:13:16.392019 11914 tablet_service.cc:1514] Processing DeleteTablet for tablet cdc3cc49ada04a2c9aae9e2cc0788d42 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-23 22:13:16 UTC) from {username='kudu', principal='kudu/127.1.135.254@KRBTEST.COM'} at 127.0.0.1:41020
I20260923 22:13:16.392436 12285 tablet_replica.cc:330] T cdc3cc49ada04a2c9aae9e2cc0788d42 P 560723db019242f5bf9eb64143be9386: stopping tablet replica
I20260923 22:13:16.392437 12287 tablet_replica.cc:330] T cdc3cc49ada04a2c9aae9e2cc0788d42 P 893017a9339b4fe5a64028d7f1b7ff21: stopping tablet replica
I20260923 22:13:16.392447 1567 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskRiQe_a/build/release/bin/kudu with pid 11851
I20260923 22:13:16.392508 12286 tablet_replica.cc:330] T cdc3cc49ada04a2c9aae9e2cc0788d42 P e61d269633ad49d69f33ef43b2c458ff: stopping tablet replica
I20260923 22:13:16.392545 12287 raft_consensus.cc:2229] T cdc3cc49ada04a2c9aae9e2cc0788d42 P 893017a9339b4fe5a64028d7f1b7ff21 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260923 22:13:16.392608 12285 raft_consensus.cc:2229] T cdc3cc49ada04a2c9aae9e2cc0788d42 P 560723db019242f5bf9eb64143be9386 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260923 22:13:16.392664 12286 raft_consensus.cc:2229] T cdc3cc49ada04a2c9aae9e2cc0788d42 P e61d269633ad49d69f33ef43b2c458ff [term 1 LEADER]: Raft consensus shutting down.
I20260923 22:13:16.392701 12287 raft_consensus.cc:2258] T cdc3cc49ada04a2c9aae9e2cc0788d42 P 893017a9339b4fe5a64028d7f1b7ff21 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260923 22:13:16.392758 12285 raft_consensus.cc:2258] T cdc3cc49ada04a2c9aae9e2cc0788d42 P 560723db019242f5bf9eb64143be9386 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260923 22:13:16.392827 12286 raft_consensus.cc:2258] T cdc3cc49ada04a2c9aae9e2cc0788d42 P e61d269633ad49d69f33ef43b2c458ff [term 1 FOLLOWER]: Raft consensus is shut down!
I20260923 22:13:16.393316 12286 ts_tablet_manager.cc:1905] T cdc3cc49ada04a2c9aae9e2cc0788d42 P e61d269633ad49d69f33ef43b2c458ff: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260923 22:13:16.393497 12285 ts_tablet_manager.cc:1905] T cdc3cc49ada04a2c9aae9e2cc0788d42 P 560723db019242f5bf9eb64143be9386: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260923 22:13:16.395016 12285 ts_tablet_manager.cc:1918] T cdc3cc49ada04a2c9aae9e2cc0788d42 P 560723db019242f5bf9eb64143be9386: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260923 22:13:16.395043 12286 ts_tablet_manager.cc:1918] T cdc3cc49ada04a2c9aae9e2cc0788d42 P e61d269633ad49d69f33ef43b2c458ff: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260923 22:13:16.395100 12285 log.cc:1196] T cdc3cc49ada04a2c9aae9e2cc0788d42 P 560723db019242f5bf9eb64143be9386: Deleting WAL directory at /tmp/dist-test-taskRiQe_a/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790201539513534-1567-0/minicluster-data/ts-1/wal/wals/cdc3cc49ada04a2c9aae9e2cc0788d42
I20260923 22:13:16.395105 12286 log.cc:1196] T cdc3cc49ada04a2c9aae9e2cc0788d42 P e61d269633ad49d69f33ef43b2c458ff: Deleting WAL directory at /tmp/dist-test-taskRiQe_a/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790201539513534-1567-0/minicluster-data/ts-2/wal/wals/cdc3cc49ada04a2c9aae9e2cc0788d42
I20260923 22:13:16.395399 12286 ts_tablet_manager.cc:1939] T cdc3cc49ada04a2c9aae9e2cc0788d42 P e61d269633ad49d69f33ef43b2c458ff: Deleting consensus metadata
I20260923 22:13:16.395399 12285 ts_tablet_manager.cc:1939] T cdc3cc49ada04a2c9aae9e2cc0788d42 P 560723db019242f5bf9eb64143be9386: Deleting consensus metadata
I20260923 22:13:16.395927 11773 catalog_manager.cc:4841] TS e61d269633ad49d69f33ef43b2c458ff (127.1.135.195:44153): tablet cdc3cc49ada04a2c9aae9e2cc0788d42 (table test-table [id=13d956b57819415db20b3b91f9ee8294]) successfully deleted
I20260923 22:13:16.395984 11774 catalog_manager.cc:4841] TS 560723db019242f5bf9eb64143be9386 (127.1.135.194:45271): tablet cdc3cc49ada04a2c9aae9e2cc0788d42 (table test-table [id=13d956b57819415db20b3b91f9ee8294]) successfully deleted
W20260923 22:13:16.398823 11776 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.1.135.193:39785 (error 108)
I20260923 22:13:16.399075 1567 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskRiQe_a/build/release/bin/kudu with pid 11985
I20260923 22:13:16.399246 12256 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 22:13:16.399139 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.135.193:39785 (local address 127.0.0.1:41024)
0923 22:13:16.399187 (+ 48us) negotiation.cc:107] Waiting for socket to connect
0923 22:13:16.399218 (+ 31us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.135.193:39785: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":29}
W20260923 22:13:16.399580 11776 catalog_manager.cc:4584] TS 893017a9339b4fe5a64028d7f1b7ff21 (127.1.135.193:39785): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet cdc3cc49ada04a2c9aae9e2cc0788d42: Network error: Client connection negotiation failed: client connection to 127.1.135.193:39785: connect: Connection refused (error 111)
I20260923 22:13:16.404812 1567 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskRiQe_a/build/release/bin/kudu with pid 12119
I20260923 22:13:16.409150 1567 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskRiQe_a/build/release/bin/kudu with pid 11757
2026-09-23T22:13:16Z chronyd exiting
[ OK ] SecurityITest.TestRequireAuthenticationSecureCluster (2009 ms)
[ RUN ] SecurityITest.TestEncryptionWithKMSIntegration
Loading random data
Initializing database '/tmp/dist-test-taskRiQe_a/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790201539513534-1567-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Sep 23 22:13:16 dist-test-slave-s2fj krb5kdc[12291](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Sep 23 22:13:16 dist-test-slave-s2fj krb5kdc[12291](info): set up 2 sockets
Sep 23 22:13:16 dist-test-slave-s2fj krb5kdc[12291](info): commencing operation
krb5kdc: starting...
W20260923 22:13:17.440958 1567 mini_kdc.cc:121] Time spent starting KDC: real 1.012s user 0.000s sys 0.005s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskRiQe_a/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790201539513534-1567-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskRiQe_a/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790201539513534-1567-0/krb5kdc/test-user.keytab.
Sep 23 22:13:17 dist-test-slave-s2fj krb5kdc[12291](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790201597, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-09-23T22:13:17Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-09-23T22:13:17Z Disabled control of system clock
WARNING: no policy specified for rangeradmin/127.1.135.212@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "rangeradmin/127.1.135.212@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal rangeradmin/127.1.135.212@KRBTEST.COM with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskRiQe_a/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790201539513534-1567-0/krb5kdc/rangeradmin_127.1.135.212@KRBTEST.COM.keytab.
Entry for principal rangeradmin/127.1.135.212@KRBTEST.COM with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskRiQe_a/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790201539513534-1567-0/krb5kdc/rangeradmin_127.1.135.212@KRBTEST.COM.keytab.
WARNING: no policy specified for rangerlookup/127.1.135.212@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "rangerlookup/127.1.135.212@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal rangerlookup/127.1.135.212@KRBTEST.COM with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskRiQe_a/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790201539513534-1567-0/krb5kdc/rangerlookup_127.1.135.212@KRBTEST.COM.keytab.
Entry for principal rangerlookup/127.1.135.212@KRBTEST.COM with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskRiQe_a/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790201539513534-1567-0/krb5kdc/rangerlookup_127.1.135.212@KRBTEST.COM.keytab.
WARNING: no policy specified for HTTP/127.1.135.212@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "HTTP/127.1.135.212@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal HTTP/127.1.135.212@KRBTEST.COM with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskRiQe_a/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790201539513534-1567-0/krb5kdc/HTTP_127.1.135.212@KRBTEST.COM.keytab.
Entry for principal HTTP/127.1.135.212@KRBTEST.COM with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskRiQe_a/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790201539513534-1567-0/krb5kdc/HTTP_127.1.135.212@KRBTEST.COM.keytab.
I20260923 22:13:17.590713 1567 mini_postgres.cc:62] Running initdb...
The files belonging to this database system will be owned by user "slave".
This user must also own the server process.
The database cluster will be initialized with locale "C".
The default database encoding has accordingly been set to "SQL_ASCII".
The default text search configuration will be set to "english".
Data page checksums are disabled.
creating directory /tmp/dist-test-taskRiQe_a/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790201539513534-1567-0/minicluster-data/postgres ... ok
creating subdirectories ... ok
selecting dynamic shared memory implementation ... posix
selecting default max_connections ... 100
selecting default shared_buffers ... 128MB
selecting default time zone ... Etc/UTC
creating configuration files ... ok
running bootstrap script ... ok
performing post-bootstrap initialization ... ok
syncing data to disk ... ok
initdb: warning: enabling "trust" authentication for local connections
You can change this by editing pg_hba.conf or using the option -A, or
--auth-local and --auth-host, the next time you run initdb.
Success. You can now start the database server using:
/tmp/dist-test-taskRiQe_a/build/release/bin/postgres/pg_ctl -D /tmp/dist-test-taskRiQe_a/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790201539513534-1567-0/minicluster-data/postgres -l logfile start
2026-09-23 22:13:19.983 UTC [12321] LOG: starting PostgreSQL 12.2 on x86_64-pc-linux-gnu, compiled by cc (GCC) 8.3.1 20190311 (Red Hat 8.3.1-3), 64-bit
2026-09-23 22:13:19.983 UTC [12321] LOG: listening on IPv4 address "127.1.135.212", port 48759
2026-09-23 22:13:19.987 UTC [12321] LOG: listening on Unix socket "/tmp/.s.PGSQL.48759"
2026-09-23 22:13:20.008 UTC [12324] LOG: database system was shut down at 2026-09-23 22:13:18 UTC
2026-09-23 22:13:20.012 UTC [12321] LOG: database system is ready to accept connections
I20260923 22:13:20.940745 1567 mini_postgres.cc:96] Postgres bound to 48759
2026-09-23 22:13:20.946 UTC [12332] FATAL: database "slave" does not exist
127.1.135.212:48759 - accepting connections
I20260923 22:13:20.947086 1567 mini_ranger.cc:161] Starting Ranger...
I20260923 22:13:20.960677 1567 mini_ranger.cc:84] Created miniranger Postgres user
I20260923 22:13:21.438266 1567 mini_ranger.cc:87] Created ranger Postgres database
I20260923 22:13:21.438354 1567 mini_ranger.cc:178] Starting Ranger out of /tmp/dist-test-taskRiQe_a/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790201539513534-1567-0/minicluster-data/ranger-admin
2026-09-23 22:13:21,714 [I] DB FLAVOR :POSTGRES
2026-09-23 22:13:21,715 [I] --------- Verifying Ranger DB connection ---------
2026-09-23 22:13:21,715 [I] Checking connection..
2026-09-23 22:13:21,715 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskRiQe_a/build/release/bin/postgresql.jar:/tmp/dist-test-taskRiQe_a/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.1.135.212:48759/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select 1;"
2026-09-23 22:13:21,943 [I] Checking connection passed.
2026-09-23 22:13:21,943 [I] --------- Verifying version history table ---------
2026-09-23 22:13:21,943 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskRiQe_a/build/release/bin/postgresql.jar:/tmp/dist-test-taskRiQe_a/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.1.135.212:48759/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_db_version_h') as temp;"
2026-09-23 22:13:22,199 [I] Table x_db_version_h does not exist in database ranger
2026-09-23 22:13:22,200 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskRiQe_a/build/release/bin/postgresql.jar:/tmp/dist-test-taskRiQe_a/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.1.135.212:48759/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_db_version_h') as temp;"
2026-09-23 22:13:22,441 [I] Table x_db_version_h does not exist in database ranger
2026-09-23 22:13:22,441 [I] Importing x_db_version_h table schema to database ranger from file: create_dbversion_catalog.sql
2026-09-23 22:13:22,442 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskRiQe_a/build/release/bin/postgresql.jar:/tmp/dist-test-taskRiQe_a/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.1.135.212:48759/ranger -u miniranger -p '********' -noheader -trim -c \; -input /tmp/dist-test-taskRiQe_a/thirdparty/src/ranger-2.1.0-admin/db/postgres/create_dbversion_catalog.sql
2026-09-23 22:13:22.671 UTC [12396] WARNING: there is no transaction in progress
2026-09-23 22:13:22,678 [I] create_dbversion_catalog.sql file imported successfully
2026-09-23 22:13:22,678 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskRiQe_a/build/release/bin/postgresql.jar:/tmp/dist-test-taskRiQe_a/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.1.135.212:48759/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_db_version_h') as temp;"
2026-09-23 22:13:22,933 [I] Table x_db_version_h already exists in database 'ranger'
2026-09-23 22:13:22,933 [I] --------- Importing Ranger Core DB Schema ---------
2026-09-23 22:13:22,934 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskRiQe_a/build/release/bin/postgresql.jar:/tmp/dist-test-taskRiQe_a/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.1.135.212:48759/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'CORE_DB_SCHEMA' and active = 'Y';"
2026-09-23 22:13:23,185 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskRiQe_a/build/release/bin/postgresql.jar:/tmp/dist-test-taskRiQe_a/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.1.135.212:48759/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'CORE_DB_SCHEMA' and active = 'N';"
2026-09-23 22:13:23,404 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskRiQe_a/build/release/bin/postgresql.jar:/tmp/dist-test-taskRiQe_a/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.1.135.212:48759/ranger -u miniranger -p '********' -noheader -trim -c \; -query "insert into x_db_version_h (version, inst_at, inst_by, updated_at, updated_by,active) values ('CORE_DB_SCHEMA', current_timestamp, 'Ranger 2.1.0', current_timestamp, 'dist-test-slave-s2fj.c.gcp-upstream.internal','N') ;"
2026-09-23 22:13:23,650 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskRiQe_a/build/release/bin/postgresql.jar:/tmp/dist-test-taskRiQe_a/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.1.135.212:48759/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_portal_user') as temp;"
2026-09-23 22:13:23,906 [I] Table x_portal_user does not exist in database ranger
2026-09-23 22:13:23,906 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskRiQe_a/build/release/bin/postgresql.jar:/tmp/dist-test-taskRiQe_a/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.1.135.212:48759/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_policy_ref_group') as temp;"
2026-09-23 22:13:24,142 [I] Table x_policy_ref_group does not exist in database ranger
2026-09-23 22:13:24,142 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskRiQe_a/build/release/bin/postgresql.jar:/tmp/dist-test-taskRiQe_a/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.1.135.212:48759/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'DB_PATCHES' and active = 'Y';"
2026-09-23 22:13:24,391 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskRiQe_a/build/release/bin/postgresql.jar:/tmp/dist-test-taskRiQe_a/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.1.135.212:48759/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'JAVA_PATCHES' and active = 'Y';"
2026-09-23 22:13:24,624 [I] Importing DB schema to database ranger from file: ranger_core_db_postgres.sql
2026-09-23 22:13:24,625 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskRiQe_a/build/release/bin/postgresql.jar:/tmp/dist-test-taskRiQe_a/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.1.135.212:48759/ranger -u miniranger -p '********' -noheader -trim -c \; -input /tmp/dist-test-taskRiQe_a/thirdparty/src/ranger-2.1.0-admin/db/postgres/optimized/current/ranger_core_db_postgres.sql
2026-09-23 22:13:25.523 UTC [12504] WARNING: there is no transaction in progress
2026-09-23 22:13:25.539 UTC [12504] WARNING: there is no transaction in progress
2026-09-23 22:13:25.554 UTC [12504] WARNING: there is no transaction in progress
2026-09-23 22:13:25.568 UTC [12504] WARNING: there is no transaction in progress
2026-09-23 22:13:25.583 UTC [12504] WARNING: there is no transaction in progress
2026-09-23 22:13:25.599 UTC [12504] WARNING: there is no transaction in progress
2026-09-23 22:13:25.650 UTC [12504] WARNING: there is no transaction in progress
2026-09-23 22:13:25.657 UTC [12504] WARNING: there is no transaction in progress
2026-09-23 22:13:25.670 UTC [12504] WARNING: there is no transaction in progress
2026-09-23 22:13:25.680 UTC [12504] WARNING: there is no transaction in progress
2026-09-23 22:13:25.690 UTC [12504] WARNING: there is no transaction in progress
2026-09-23 22:13:25.703 UTC [12504] WARNING: there is no transaction in progress
2026-09-23 22:13:25.711 UTC [12504] WARNING: there is no transaction in progress
2026-09-23 22:13:25.718 UTC [12504] WARNING: there is no transaction in progress
2026-09-23 22:13:25.723 UTC [12504] WARNING: there is no transaction in progress
2026-09-23 22:13:25.729 UTC [12504] WARNING: there is no transaction in progress
2026-09-23 22:13:26,329 [I] ranger_core_db_postgres.sql file imported successfully
2026-09-23 22:13:26,330 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskRiQe_a/build/release/bin/postgresql.jar:/tmp/dist-test-taskRiQe_a/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.1.135.212:48759/ranger -u miniranger -p '********' -noheader -trim -c \; -query "update x_db_version_h set inst_by='Ranger 2.1.0' where active='Y' and updated_by='localhost';"
2026-09-23 22:13:26,571 [I] Patches status entries updated from base ranger version to current installed ranger version:Ranger 2.1.0
2026-09-23 22:13:26,571 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskRiQe_a/build/release/bin/postgresql.jar:/tmp/dist-test-taskRiQe_a/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.1.135.212:48759/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_portal_user') as temp;"
2026-09-23 22:13:26,791 [I] Table x_portal_user already exists in database 'ranger'
2026-09-23 22:13:26,792 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskRiQe_a/build/release/bin/postgresql.jar:/tmp/dist-test-taskRiQe_a/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.1.135.212:48759/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_policy_ref_group') as temp;"
2026-09-23 22:13:27,027 [I] Table x_policy_ref_group already exists in database 'ranger'
2026-09-23 22:13:27,028 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskRiQe_a/build/release/bin/postgresql.jar:/tmp/dist-test-taskRiQe_a/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.1.135.212:48759/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'DB_PATCHES' and active = 'Y';"
2026-09-23 22:13:27,277 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskRiQe_a/build/release/bin/postgresql.jar:/tmp/dist-test-taskRiQe_a/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.1.135.212:48759/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'JAVA_PATCHES' and active = 'Y';"
2026-09-23 22:13:27,526 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskRiQe_a/build/release/bin/postgresql.jar:/tmp/dist-test-taskRiQe_a/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.1.135.212:48759/ranger -u miniranger -p '********' -noheader -trim -c \; -query "update x_db_version_h set active='Y' where version='CORE_DB_SCHEMA' and active='N' and updated_by='dist-test-slave-s2fj.c.gcp-upstream.internal';"
2026-09-23 22:13:27,771 [I] CORE_DB_SCHEMA import status has been updated
2026-09-23 22:13:27,772 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskRiQe_a/build/release/bin/postgresql.jar:/tmp/dist-test-taskRiQe_a/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.1.135.212:48759/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'DB_PATCHES' and inst_by = 'Ranger 2.1.0' and active = 'Y';"
2026-09-23 22:13:28,025 [I] DB_PATCHES have already been applied
I20260923 22:13:28.031078 1567 mini_ranger.cc:191] Using Ranger class path: /tmp/dist-test-taskRiQe_a/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790201539513534-1567-0/minicluster-data/ranger-admin:/tmp/dist-test-taskRiQe_a/thirdparty/src/ranger-2.1.0-admin/ews/lib/*:/usr/lib/jvm/java-1.8.0-openjdk-amd64/lib/*:/tmp/dist-test-taskRiQe_a/thirdparty/src/hadoop-3.2.0/*:/tmp/dist-test-taskRiQe_a/build/release/bin/postgresql.jar:/tmp/dist-test-taskRiQe_a/thirdparty/src/ranger-2.1.0-admin/ews/webapp
I20260923 22:13:28.031136 1567 mini_ranger.cc:193] Using host: 127.1.135.212
I20260923 22:13:28.034093 1567 mini_ranger.cc:227] Ranger admin URL: http://127.1.135.212:53083
Sep 23, 2026 10:13:28 PM org.apache.ranger.server.tomcat.EmbeddedServer getKeyManagers
WARNING: Config 'ranger.keystore.file' or 'ranger.service.https.attrib.keystore.file' is not found or contains blank value
Sep 23, 2026 10:13:28 PM org.apache.ranger.server.tomcat.EmbeddedServer getTrustManagers
WARNING: Config 'ranger.truststore.file' is not found or contains blank value!
Sep 23, 2026 10:13:28 PM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Deriving webapp folder from catalina.base property. folder=/tmp/dist-test-taskRiQe_a/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790201539513534-1567-0/minicluster-data/ranger-admin/ews/webapp
Sep 23, 2026 10:13:28 PM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Webapp file =/tmp/dist-test-taskRiQe_a/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790201539513534-1567-0/minicluster-data/ranger-admin/ews/webapp, webAppName = /
Sep 23, 2026 10:13:28 PM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Adding webapp [/] = path [/tmp/dist-test-taskRiQe_a/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790201539513534-1567-0/minicluster-data/ranger-admin/ews/webapp] .....
Sep 23, 2026 10:13:28 PM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Finished init of webapp [/] = path [/tmp/dist-test-taskRiQe_a/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790201539513534-1567-0/minicluster-data/ranger-admin/ews/webapp].
Sep 23, 2026 10:13:28 PM org.apache.ranger.server.tomcat.EmbeddedServer startServer
INFO: Server Name : miniranger
I20260923 22:13:29.149471 1567 mini_ranger.cc:160] Time spent starting Ranger: real 8.202s user 0.000s sys 0.009s
Sep 23, 2026 10:13:48 PM com.sun.jersey.api.core.PackagesResourceConfig init
INFO: Scanning for root resource and provider classes in the packages:
org.apache.ranger.rest
org.apache.ranger.common
xa.rest
Sep 23, 2026 10:13:48 PM com.sun.jersey.api.core.ScanningResourceConfig logClasses
INFO: Root resource classes found:
class org.apache.ranger.rest.XAuditREST
class org.apache.ranger.rest.SecurityZoneREST
class org.apache.ranger.rest.PublicAPIsv2
class org.apache.ranger.rest.XKeyREST
class org.apache.ranger.rest.RoleREST
class org.apache.ranger.rest.ServiceREST
class org.apache.ranger.rest.AssetREST
class org.apache.ranger.rest.XUserREST
class org.apache.ranger.rest.TagREST
class org.apache.ranger.rest.MetricsREST
class org.apache.ranger.rest.PublicAPIs
class org.apache.ranger.rest.UserREST
Sep 23, 2026 10:13:48 PM com.sun.jersey.api.core.ScanningResourceConfig logClasses
INFO: Provider classes found:
class org.apache.ranger.common.RangerJAXBContextResolver
Sep 23, 2026 10:13:48 PM com.sun.jersey.spi.spring.container.servlet.SpringServlet getContext
INFO: Using default applicationContext
Sep 23, 2026 10:13:48 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, assetREST, of type org.apache.ranger.rest.AssetREST as a root resource class
Sep 23, 2026 10:13:48 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, metricsREST, of type org.apache.ranger.rest.MetricsREST as a root resource class
Sep 23, 2026 10:13:48 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, publicAPIs, of type org.apache.ranger.rest.PublicAPIs as a root resource class
Sep 23, 2026 10:13:48 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, publicAPIsv2, of type org.apache.ranger.rest.PublicAPIsv2 as a root resource class
Sep 23, 2026 10:13:48 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, roleREST, of type org.apache.ranger.rest.RoleREST as a root resource class
Sep 23, 2026 10:13:48 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, securityZoneREST, of type org.apache.ranger.rest.SecurityZoneREST as a root resource class
Sep 23, 2026 10:13:48 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, serviceREST, of type org.apache.ranger.rest.ServiceREST as a root resource class
Sep 23, 2026 10:13:48 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, tagREST, of type org.apache.ranger.rest.TagREST as a root resource class
Sep 23, 2026 10:13:48 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, userREST, of type org.apache.ranger.rest.UserREST as a root resource class
Sep 23, 2026 10:13:48 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, XAuditREST, of type org.apache.ranger.rest.XAuditREST as a root resource class
Sep 23, 2026 10:13:48 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, XKeyREST, of type org.apache.ranger.rest.XKeyREST as a root resource class
Sep 23, 2026 10:13:48 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, XUserREST, of type org.apache.ranger.rest.XUserREST as a root resource class
Sep 23, 2026 10:13:48 PM com.sun.jersey.server.impl.application.WebApplicationImpl _initiate
INFO: Initiating Jersey application, version 'Jersey: 1.19.3 10/24/2016 03:58 PM'
Sep 23, 2026 10:13:49 PM com.sun.jersey.spi.inject.Errors processErrorMessages
WARNING: The following warnings have been detected with resource and/or provider classes:
WARNING: A HTTP GET method, public void org.apache.ranger.rest.ServiceREST.getPoliciesInExcel(javax.servlet.http.HttpServletRequest,javax.servlet.http.HttpServletResponse), MUST return a non-void type.
WARNING: A HTTP GET method, public void org.apache.ranger.rest.ServiceREST.getPoliciesInJson(javax.servlet.http.HttpServletRequest,javax.servlet.http.HttpServletResponse,java.lang.Boolean), MUST return a non-void type.
WARNING: A HTTP GET method, public void org.apache.ranger.rest.ServiceREST.getPoliciesInCsv(javax.servlet.http.HttpServletRequest,javax.servlet.http.HttpServletResponse) throws java.io.IOException, MUST return a non-void type.
I20260923 22:13:50.098292 1567 mini_ranger.cc:261] Created Kudu service
WARNING: no policy specified for rangerkms/127.1.135.212@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "rangerkms/127.1.135.212@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal rangerkms/127.1.135.212@KRBTEST.COM with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskRiQe_a/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790201539513534-1567-0/krb5kdc/rangerkms_127.1.135.212@KRBTEST.COM.keytab.
Entry for principal rangerkms/127.1.135.212@KRBTEST.COM with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskRiQe_a/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790201539513534-1567-0/krb5kdc/rangerkms_127.1.135.212@KRBTEST.COM.keytab.
WARNING: no policy specified for HTTP/127.1.135.212@KRBTEST.COM; defaulting to no policy
add_principal: Principal or policy already exists while creating "HTTP/127.1.135.212@KRBTEST.COM".
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal HTTP/127.1.135.212@KRBTEST.COM with kvno 3, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskRiQe_a/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790201539513534-1567-0/krb5kdc/HTTP_127.1.135.212@KRBTEST.COM.keytab.
Entry for principal HTTP/127.1.135.212@KRBTEST.COM with kvno 3, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskRiQe_a/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790201539513534-1567-0/krb5kdc/HTTP_127.1.135.212@KRBTEST.COM.keytab.
WARNING: no policy specified for keyadmin@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "keyadmin@KRBTEST.COM" created.
Sep 23 22:13:50 dist-test-slave-s2fj krb5kdc[12291](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790201630, etypes {rep=17 tkt=17 ses=17}, keyadmin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for keyadmin@KRBTEST.COM:
I20260923 22:13:50.174016 1567 mini_ranger_kms.cc:208] Starting Ranger KMS...
I20260923 22:13:50.182638 1567 mini_ranger_kms.cc:78] Created minirangerkms Postgres user
I20260923 22:13:50.991036 1567 mini_ranger_kms.cc:81] Created rangerkms Postgres database
I20260923 22:13:50.991122 1567 mini_ranger_kms.cc:226] Starting Ranger KMS out of /tmp/dist-test-taskRiQe_a/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790201539513534-1567-0/minicluster-data/ranger-kms
I20260923 22:13:50.991143 1567 mini_ranger_kms.cc:227] Using postgres at 127.1.135.212:48759
2026-09-23 22:13:51,161 [I] DB FLAVOR :POSTGRES
2026-09-23 22:13:51,162 [I] --------- Verifying Ranger DB connection ---------
2026-09-23 22:13:51,162 [I] Checking connection
2026-09-23 22:13:51,162 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskRiQe_a/build/release/bin/postgresql.jar:/tmp/dist-test-taskRiQe_a/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790201539513534-1567-0/minicluster-data/ranger-kms/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.1.135.212:48759/rangerkms -u rangerkms -p '********' -noheader -trim -c \; -query "SELECT 1;"
2026-09-23 22:13:51,405 [I] connection success
2026-09-23 22:13:51,405 [I] --------- Verifying Ranger DB tables ---------
2026-09-23 22:13:51,405 [I] Verifying table ranger_masterkey in database rangerkms
2026-09-23 22:13:51,405 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskRiQe_a/build/release/bin/postgresql.jar:/tmp/dist-test-taskRiQe_a/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790201539513534-1567-0/minicluster-data/ranger-kms/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.1.135.212:48759/rangerkms -u rangerkms -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='rangerkms' and table_name = 'ranger_masterkey') as temp;"
2026-09-23 22:13:51,626 [I] Table ranger_masterkey does not exist in database rangerkms
2026-09-23 22:13:51,627 [I] --------- Importing Ranger Core DB Schema ---------
2026-09-23 22:13:51,627 [I] Importing db schema to database rangerkms from file: kms_core_db_postgres.sql
2026-09-23 22:13:51,627 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskRiQe_a/build/release/bin/postgresql.jar:/tmp/dist-test-taskRiQe_a/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790201539513534-1567-0/minicluster-data/ranger-kms/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.1.135.212:48759/rangerkms -u rangerkms -p '********' -noheader -trim -c \; -input /tmp/dist-test-taskRiQe_a/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790201539513534-1567-0/minicluster-data/ranger-kms/db/postgres/kms_core_db_postgres.sql
2026-09-23 22:13:51,866 [I] kms_core_db_postgres.sql DB schema imported successfully
I20260923 22:13:52.305797 1567 mini_ranger_kms.cc:310] Created kms service
I20260923 22:13:52.475000 1567 mini_ranger_kms.cc:326] Created kudu user
I20260923 22:13:52.515070 1567 mini_ranger_kms.cc:343] Created rangerkms user
Sep 23 22:13:52 dist-test-slave-s2fj krb5kdc[12291](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790201630, etypes {rep=17 tkt=17 ses=17}, keyadmin@KRBTEST.COM for HTTP/127.1.135.212@KRBTEST.COM
I20260923 22:13:52.900084 1567 mini_ranger_kms.cc:384] Added ranger policy
I20260923 22:13:52.900200 1567 mini_ranger_kms.cc:238] Using RangerKMS classpath: /tmp/dist-test-taskRiQe_a/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790201539513534-1567-0/minicluster-data/ranger-kms:/tmp/dist-test-taskRiQe_a/build/release/bin/postgresql.jar:/tmp/dist-test-taskRiQe_a/thirdparty/src/ranger-2.1.0-kms/ews/webapp/WEB-INF/classes/lib/*:/tmp/dist-test-taskRiQe_a/thirdparty/src/ranger-2.1.0-kms/ews/webapp/lib/*:/usr/lib/jvm/java-1.8.0-openjdk-amd64/lib/*
I20260923 22:13:52.900221 1567 mini_ranger_kms.cc:240] Using host: 127.1.135.212
I20260923 22:13:52.903513 1567 mini_ranger_kms.cc:276] Ranger KMS PID: 12684
I20260923 22:13:52.903646 1567 mini_ranger_kms.cc:277] Ranger KMS URL: http://127.1.135.212:52811
Sep 23, 2026 10:13:53 PM org.apache.ranger.server.tomcat.EmbeddedServer getKeyManagers
WARNING: Config 'ranger.keystore.file' or 'ranger.service.https.attrib.keystore.file' is not found or contains blank value
Sep 23, 2026 10:13:53 PM org.apache.ranger.server.tomcat.EmbeddedServer getTrustManagers
WARNING: Config 'ranger.truststore.file' is not found or contains blank value!
Sep 23, 2026 10:13:53 PM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Webapp file =/tmp/dist-test-taskRiQe_a/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790201539513534-1567-0/minicluster-data/ranger-kms/ews/webapp, webAppName = /kms
Sep 23, 2026 10:13:53 PM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Adding webapp [/kms] = path [/tmp/dist-test-taskRiQe_a/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790201539513534-1567-0/minicluster-data/ranger-kms/ews/webapp] .....
Sep 23, 2026 10:13:54 PM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Finished init of webapp [/kms] = path [/tmp/dist-test-taskRiQe_a/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790201539513534-1567-0/minicluster-data/ranger-kms/ews/webapp].
Sep 23, 2026 10:13:54 PM org.apache.ranger.server.tomcat.EmbeddedServer startServer
INFO: Server Name : minirangerkms
Sep 23 22:13:54 dist-test-slave-s2fj krb5kdc[12291](info): AS_REQ (1 etypes {17}) 127.0.0.1: ISSUE: authtime 1790201634, etypes {rep=17 tkt=17 ses=17}, rangerkms/127.1.135.212@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260923 22:13:55.200625 1567 mini_ranger_kms.cc:207] Time spent starting Ranger KMS: real 5.027s user 0.006s sys 0.148s
I20260923 22:13:55.200718 1567 mini_ranger_kms.cc:397] {"name":"kuduclusterkey","cipher":"AES/CTR/NoPadding","length":128,"description":"kuduclusterkey"}
I20260923 22:13:55.200788 1567 mini_ranger_kms.cc:401] 127.1.135.212:52811/kms/v1/keys
Sep 23 22:13:55 dist-test-slave-s2fj krb5kdc[12291](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790201634, etypes {rep=17 tkt=17 ses=17}, rangerkms/127.1.135.212@KRBTEST.COM for HTTP/127.1.135.212@KRBTEST.COM
Sep 23 22:13:55 dist-test-slave-s2fj krb5kdc[12291](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790201634, etypes {rep=17 tkt=17 ses=17}, rangerkms/127.1.135.212@KRBTEST.COM for HTTP/127.1.135.212@KRBTEST.COM
Sep 23 22:13:57 dist-test-slave-s2fj krb5kdc[12291](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790201637, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
WARNING: no policy specified for kudu/127.1.135.254@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.1.135.254@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.1.135.254 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskRiQe_a/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790201539513534-1567-0/krb5kdc/kudu_127.1.135.254.keytab.
Entry for principal kudu/127.1.135.254 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskRiQe_a/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790201539513534-1567-0/krb5kdc/kudu_127.1.135.254.keytab.
I20260923 22:13:57.946441 1567 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskRiQe_a/build/release/bin/kudu
/tmp/dist-test-taskRiQe_a/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskRiQe_a/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790201539513534-1567-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskRiQe_a/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790201539513534-1567-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskRiQe_a/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790201539513534-1567-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskRiQe_a/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790201539513534-1567-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.1.135.254:39417
--webserver_interface=127.1.135.254
--webserver_port=0
--keytab_file=/tmp/dist-test-taskRiQe_a/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790201539513534-1567-0/krb5kdc/kudu_127.1.135.254.keytab
--principal=kudu/127.1.135.254
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.1.135.212:34907
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.1.135.254:39417
--ranger_config_path=/tmp/dist-test-taskRiQe_a/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790201539513534-1567-0/minicluster-data/ranger-client
--trusted_user_acl=test-admin
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.1.135.212:52811/kms
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskRiQe_a/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790201539513534-1567-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskRiQe_a/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790201539513534-1567-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskRiQe_a/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790201539513534-1567-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260923 22:13:58.042806 12735 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260923 22:13:58.043031 12735 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260923 22:13:58.043067 12735 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260923 22:13:58.044206 12735 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260923 22:13:58.044276 12735 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260923 22:13:58.044301 12735 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260923 22:13:58.044328 12735 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260923 22:13:58.044360 12735 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260923 22:13:58.045811 12735 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.1.135.212:34907
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-taskRiQe_a/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790201539513534-1567-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskRiQe_a/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790201539513534-1567-0/minicluster-data/master-0/wal
--ranger_kms_url=127.1.135.212:52811/kms
--trusted_user_acl=<redacted>
--ipki_ca_key_size=768
--master_addresses=127.1.135.254:39417
--ranger_config_path=/tmp/dist-test-taskRiQe_a/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790201539513534-1567-0/minicluster-data/ranger-client
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskRiQe_a/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790201539513534-1567-0/krb5kdc/kudu_127.1.135.254.keytab
--principal=kudu/127.1.135.254
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.1.135.254:39417
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskRiQe_a/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790201539513534-1567-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.1.135.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.12735
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskRiQe_a/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790201539513534-1567-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 23 Sep 2026 20:53:37 UTC on ccycloud.quasar-rkviyx.root.comops.site
build id 4668083
I20260923 22:13:58.046065 12735 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260923 22:13:58.046280 12735 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260923 22:13:58.052022 12741 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260923 22:13:58.052055 12735 server_base.cc:1034] running on GCE node
W20260923 22:13:58.052013 12743 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260923 22:13:58.052311 12740 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260923 22:13:58.052632 12735 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260923 22:13:58.052938 12735 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260923 22:13:58.054109 12735 hybrid_clock.cc:648] HybridClock initialized: now 1790201638054097 us; error 28 us; skew 500 ppm
Sep 23 22:13:58 dist-test-slave-s2fj krb5kdc[12291](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790201638, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.135.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260923 22:13:58.056898 12735 init.cc:377] Logged in from keytab as kudu/127.1.135.254@KRBTEST.COM (short username kudu)
I20260923 22:13:58.057381 12735 webserver.cc:460] Webserver started at http://127.1.135.254:36497/ using document root <none> and password file <none>
I20260923 22:13:58.057601 12735 fs_manager.cc:359] Metadata directory not provided
I20260923 22:13:58.057641 12735 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260923 22:13:58.057770 12735 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
Sep 23 22:13:58 dist-test-slave-s2fj krb5kdc[12291](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790201638, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.135.254@KRBTEST.COM for HTTP/127.1.135.212@KRBTEST.COM
I20260923 22:13:58.108064 12735 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskRiQe_a/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790201539513534-1567-0/minicluster-data/master-0/data/instance:
uuid: "a48af14123054627ae158c6e22d475c5"
format_stamp: "Formatted at 2026-09-23 22:13:58 on dist-test-slave-s2fj"
server_key: "33d1a03ca209741a2e6852c9fc363cf3"
server_key_iv: "3eb6551fb2439e703c5ea67621dc655d"
server_key_version: "kuduclusterkey@0"
I20260923 22:13:58.108536 12735 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskRiQe_a/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790201539513534-1567-0/minicluster-data/master-0/wal/instance:
uuid: "a48af14123054627ae158c6e22d475c5"
format_stamp: "Formatted at 2026-09-23 22:13:58 on dist-test-slave-s2fj"
server_key: "33d1a03ca209741a2e6852c9fc363cf3"
server_key_iv: "3eb6551fb2439e703c5ea67621dc655d"
server_key_version: "kuduclusterkey@0"
I20260923 22:13:58.110256 12735 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.001s sys 0.000s
I20260923 22:13:58.123505 12750 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260923 22:13:58.123731 12735 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260923 22:13:58.123834 12735 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskRiQe_a/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790201539513534-1567-0/minicluster-data/master-0/data,/tmp/dist-test-taskRiQe_a/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790201539513534-1567-0/minicluster-data/master-0/wal
uuid: "a48af14123054627ae158c6e22d475c5"
format_stamp: "Formatted at 2026-09-23 22:13:58 on dist-test-slave-s2fj"
server_key: "33d1a03ca209741a2e6852c9fc363cf3"
server_key_iv: "3eb6551fb2439e703c5ea67621dc655d"
server_key_version: "kuduclusterkey@0"
I20260923 22:13:58.123920 12735 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskRiQe_a/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790201539513534-1567-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskRiQe_a/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790201539513534-1567-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskRiQe_a/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790201539513534-1567-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260923 22:13:58.139866 12735 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260923 22:13:58.141799 12735 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260923 22:13:58.141942 12735 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260923 22:13:58.147184 12735 rpc_server.cc:307] RPC server started. Bound to: 127.1.135.254:39417
I20260923 22:13:58.147226 12802 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.1.135.254:39417 every 8 connection(s)
I20260923 22:13:58.147632 12735 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskRiQe_a/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790201539513534-1567-0/minicluster-data/master-0/data/info.pb
I20260923 22:13:58.148340 12803 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260923 22:13:58.150477 12803 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P a48af14123054627ae158c6e22d475c5: Bootstrap starting.
I20260923 22:13:58.151019 12803 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P a48af14123054627ae158c6e22d475c5: Neither blocks nor log segments found. Creating new log.
I20260923 22:13:58.151298 12803 log.cc:824] T 00000000000000000000000000000000 P a48af14123054627ae158c6e22d475c5: Log is configured to *not* fsync() on all Append() calls
I20260923 22:13:58.152002 12803 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P a48af14123054627ae158c6e22d475c5: No bootstrap required, opened a new log
I20260923 22:13:58.152273 1567 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskRiQe_a/build/release/bin/kudu as pid 12735
I20260923 22:13:58.152374 1567 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskRiQe_a/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790201539513534-1567-0/minicluster-data/master-0/wal/instance
I20260923 22:13:58.152577 1567 external_mini_cluster.cc:1428] Setting key 19fb8a1688235e30044278e3d61c16d9
I20260923 22:13:58.154599 12803 raft_consensus.cc:348] T 00000000000000000000000000000000 P a48af14123054627ae158c6e22d475c5 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a48af14123054627ae158c6e22d475c5" member_type: VOTER last_known_addr { host: "127.1.135.254" port: 39417 } }
I20260923 22:13:58.154757 12803 raft_consensus.cc:374] T 00000000000000000000000000000000 P a48af14123054627ae158c6e22d475c5 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260923 22:13:58.154816 12803 raft_consensus.cc:729] T 00000000000000000000000000000000 P a48af14123054627ae158c6e22d475c5 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: a48af14123054627ae158c6e22d475c5, State: Initialized, Role: FOLLOWER
I20260923 22:13:58.154971 12803 consensus_queue.cc:260] T 00000000000000000000000000000000 P a48af14123054627ae158c6e22d475c5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a48af14123054627ae158c6e22d475c5" member_type: VOTER last_known_addr { host: "127.1.135.254" port: 39417 } }
I20260923 22:13:58.155062 12803 raft_consensus.cc:388] T 00000000000000000000000000000000 P a48af14123054627ae158c6e22d475c5 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260923 22:13:58.155105 12803 raft_consensus.cc:482] T 00000000000000000000000000000000 P a48af14123054627ae158c6e22d475c5 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260923 22:13:58.155176 12803 raft_consensus.cc:3037] T 00000000000000000000000000000000 P a48af14123054627ae158c6e22d475c5 [term 0 FOLLOWER]: Advancing to term 1
I20260923 22:13:58.155856 12803 raft_consensus.cc:504] T 00000000000000000000000000000000 P a48af14123054627ae158c6e22d475c5 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a48af14123054627ae158c6e22d475c5" member_type: VOTER last_known_addr { host: "127.1.135.254" port: 39417 } }
I20260923 22:13:58.155969 12803 leader_election.cc:302] T 00000000000000000000000000000000 P a48af14123054627ae158c6e22d475c5 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: a48af14123054627ae158c6e22d475c5; no voters:
I20260923 22:13:58.156191 12803 leader_election.cc:288] T 00000000000000000000000000000000 P a48af14123054627ae158c6e22d475c5 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260923 22:13:58.156214 12808 raft_consensus.cc:2781] T 00000000000000000000000000000000 P a48af14123054627ae158c6e22d475c5 [term 1 FOLLOWER]: Leader election won for term 1
I20260923 22:13:58.156404 12808 raft_consensus.cc:686] T 00000000000000000000000000000000 P a48af14123054627ae158c6e22d475c5 [term 1 LEADER]: Becoming Leader. State: Replica: a48af14123054627ae158c6e22d475c5, State: Running, Role: LEADER
I20260923 22:13:58.156481 12803 sys_catalog.cc:564] T 00000000000000000000000000000000 P a48af14123054627ae158c6e22d475c5 [sys.catalog]: configured and running, proceeding with master startup.
I20260923 22:13:58.156517 12808 consensus_queue.cc:237] T 00000000000000000000000000000000 P a48af14123054627ae158c6e22d475c5 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a48af14123054627ae158c6e22d475c5" member_type: VOTER last_known_addr { host: "127.1.135.254" port: 39417 } }
I20260923 22:13:58.156859 12808 sys_catalog.cc:455] T 00000000000000000000000000000000 P a48af14123054627ae158c6e22d475c5 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "a48af14123054627ae158c6e22d475c5" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a48af14123054627ae158c6e22d475c5" member_type: VOTER last_known_addr { host: "127.1.135.254" port: 39417 } } }
I20260923 22:13:58.157004 12808 sys_catalog.cc:458] T 00000000000000000000000000000000 P a48af14123054627ae158c6e22d475c5 [sys.catalog]: This master's current role is: LEADER
I20260923 22:13:58.157917 12803 ranger_client.cc:318] Using new properties file: /tmp/dist-test-taskRiQe_a/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790201539513534-1567-0/minicluster-data/master-0/logs/kudu-ranger-subprocess-log4j2.properties
I20260923 22:13:58.158241 12808 sys_catalog.cc:455] T 00000000000000000000000000000000 P a48af14123054627ae158c6e22d475c5 [sys.catalog]: SysCatalogTable state changed. Reason: New leader a48af14123054627ae158c6e22d475c5. Latest consensus state: current_term: 1 leader_uuid: "a48af14123054627ae158c6e22d475c5" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a48af14123054627ae158c6e22d475c5" member_type: VOTER last_known_addr { host: "127.1.135.254" port: 39417 } } }
I20260923 22:13:58.158377 12808 sys_catalog.cc:458] T 00000000000000000000000000000000 P a48af14123054627ae158c6e22d475c5 [sys.catalog]: This master's current role is: LEADER
I20260923 22:13:58.160748 12823 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260923 22:13:58.162864 12823 catalog_manager.cc:1484] Initializing Kudu cluster ID...
Sep 23 22:13:58 dist-test-slave-s2fj krb5kdc[12291](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790201637, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.1.135.254@KRBTEST.COM
I20260923 22:13:58.168010 12807 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 22:13:58.153059 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40196 (local address 127.1.135.254:39417)
0923 22:13:58.153233 (+ 174us) server_negotiation.cc:206] Beginning negotiation
0923 22:13:58.153255 (+ 22us) server_negotiation.cc:399] Waiting for connection header
0923 22:13:58.153336 (+ 81us) server_negotiation.cc:407] Connection header received
0923 22:13:58.154613 (+ 1277us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 22:13:58.154623 (+ 10us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 22:13:58.154690 (+ 67us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 22:13:58.154830 (+ 140us) server_negotiation.cc:226] Negotiated authn=SASL
0923 22:13:58.155718 (+ 888us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 22:13:58.157096 (+ 1378us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 22:13:58.158194 (+ 1098us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 22:13:58.161182 (+ 2988us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 22:13:58.161324 (+ 142us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 22:13:58.164152 (+ 2828us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0923 22:13:58.164156 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0923 22:13:58.164157 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0923 22:13:58.164169 (+ 12us) server_negotiation.cc:922] calling sasl_server_start()
0923 22:13:58.166096 (+ 1927us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0923 22:13:58.166462 (+ 366us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0923 22:13:58.166464 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0923 22:13:58.166465 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0923 22:13:58.166577 (+ 112us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0923 22:13:58.166764 (+ 187us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0923 22:13:58.166764 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0923 22:13:58.166765 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0923 22:13:58.167092 (+ 327us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0923 22:13:58.167616 (+ 524us) server_negotiation.cc:1035] Waiting for connection context
0923 22:13:58.167683 (+ 67us) server_negotiation.cc:299] Negotiation successful
0923 22:13:58.167797 (+ 114us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":118,"thread_start_us":72,"threads_started":1}
I20260923 22:13:58.170697 12823 catalog_manager.cc:1347] Generated new cluster ID: 62eaa82742a942b9a7d67affd93718db
I20260923 22:13:58.170804 12823 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260923 22:13:58.191108 12823 catalog_manager.cc:1370] Generated new certificate authority record
I20260923 22:13:58.191887 12823 catalog_manager.cc:1504] Loading token signing keys...
I20260923 22:13:58.195363 12823 catalog_manager.cc:5868] T 00000000000000000000000000000000 P a48af14123054627ae158c6e22d475c5: Generated new TSK 0
I20260923 22:13:58.195596 12823 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.1.135.193@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.1.135.193@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.1.135.193 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskRiQe_a/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790201539513534-1567-0/krb5kdc/kudu_127.1.135.193.keytab.
Entry for principal kudu/127.1.135.193 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskRiQe_a/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790201539513534-1567-0/krb5kdc/kudu_127.1.135.193.keytab.
I20260923 22:13:58.923157 1567 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskRiQe_a/build/release/bin/kudu
/tmp/dist-test-taskRiQe_a/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskRiQe_a/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790201539513534-1567-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskRiQe_a/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790201539513534-1567-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskRiQe_a/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790201539513534-1567-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskRiQe_a/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790201539513534-1567-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.1.135.212:52811/kms
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.1.135.193:0
--local_ip_for_outbound_sockets=127.1.135.193
--webserver_interface=127.1.135.193
--webserver_port=0
--tserver_master_addrs=127.1.135.254:39417
--keytab_file=/tmp/dist-test-taskRiQe_a/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790201539513534-1567-0/krb5kdc/kudu_127.1.135.193.keytab
--principal=kudu/127.1.135.193
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.1.135.212:34907
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskRiQe_a/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790201539513534-1567-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskRiQe_a/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790201539513534-1567-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskRiQe_a/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790201539513534-1567-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260923 22:13:59.032706 12848 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260923 22:13:59.032934 12848 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260923 22:13:59.032971 12848 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260923 22:13:59.034440 12848 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260923 22:13:59.034513 12848 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260923 22:13:59.034557 12848 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.1.135.193
I20260923 22:13:59.036113 12848 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.1.135.212:34907
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-taskRiQe_a/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790201539513534-1567-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskRiQe_a/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790201539513534-1567-0/minicluster-data/ts-0/wal
--ranger_kms_url=127.1.135.212:52811/kms
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskRiQe_a/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790201539513534-1567-0/krb5kdc/kudu_127.1.135.193.keytab
--principal=kudu/127.1.135.193
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.1.135.193:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskRiQe_a/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790201539513534-1567-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.1.135.193
--webserver_port=0
--tserver_master_addrs=127.1.135.254:39417
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.12848
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.1.135.193
--log_dir=/tmp/dist-test-taskRiQe_a/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790201539513534-1567-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 23 Sep 2026 20:53:37 UTC on ccycloud.quasar-rkviyx.root.comops.site
build id 4668083
I20260923 22:13:59.036355 12848 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260923 22:13:59.036617 12848 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260923 22:13:59.041778 12856 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260923 22:13:59.041786 12853 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260923 22:13:59.041788 12854 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260923 22:13:59.042225 12848 server_base.cc:1034] running on GCE node
I20260923 22:13:59.042436 12848 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260923 22:13:59.042719 12848 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260923 22:13:59.043893 12848 hybrid_clock.cc:648] HybridClock initialized: now 1790201639043867 us; error 33 us; skew 500 ppm
Sep 23 22:13:59 dist-test-slave-s2fj krb5kdc[12291](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790201639, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.135.193@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260923 22:13:59.046427 12848 init.cc:377] Logged in from keytab as kudu/127.1.135.193@KRBTEST.COM (short username kudu)
I20260923 22:13:59.046927 12848 webserver.cc:460] Webserver started at http://127.1.135.193:34801/ using document root <none> and password file <none>
I20260923 22:13:59.047164 12848 fs_manager.cc:359] Metadata directory not provided
I20260923 22:13:59.047207 12848 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260923 22:13:59.047330 12848 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
Sep 23 22:13:59 dist-test-slave-s2fj krb5kdc[12291](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790201639, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.135.193@KRBTEST.COM for HTTP/127.1.135.212@KRBTEST.COM
I20260923 22:13:59.061623 12848 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskRiQe_a/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790201539513534-1567-0/minicluster-data/ts-0/data/instance:
uuid: "50a730daa7ac44c896d7621ce906e828"
format_stamp: "Formatted at 2026-09-23 22:13:59 on dist-test-slave-s2fj"
server_key: "05f1b33ff075e55462a5e5cc95001c13"
server_key_iv: "946a3481b48ab5c7bb2e960a55bc213c"
server_key_version: "kuduclusterkey@0"
I20260923 22:13:59.062165 12848 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskRiQe_a/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790201539513534-1567-0/minicluster-data/ts-0/wal/instance:
uuid: "50a730daa7ac44c896d7621ce906e828"
format_stamp: "Formatted at 2026-09-23 22:13:59 on dist-test-slave-s2fj"
server_key: "05f1b33ff075e55462a5e5cc95001c13"
server_key_iv: "946a3481b48ab5c7bb2e960a55bc213c"
server_key_version: "kuduclusterkey@0"
I20260923 22:13:59.064158 12848 fs_manager.cc:691] Time spent creating directory manager: real 0.002s user 0.001s sys 0.000s
I20260923 22:13:59.077518 12865 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260923 22:13:59.077790 12848 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.001s
I20260923 22:13:59.077903 12848 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskRiQe_a/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790201539513534-1567-0/minicluster-data/ts-0/data,/tmp/dist-test-taskRiQe_a/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790201539513534-1567-0/minicluster-data/ts-0/wal
uuid: "50a730daa7ac44c896d7621ce906e828"
format_stamp: "Formatted at 2026-09-23 22:13:59 on dist-test-slave-s2fj"
server_key: "05f1b33ff075e55462a5e5cc95001c13"
server_key_iv: "946a3481b48ab5c7bb2e960a55bc213c"
server_key_version: "kuduclusterkey@0"
I20260923 22:13:59.078011 12848 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskRiQe_a/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790201539513534-1567-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskRiQe_a/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790201539513534-1567-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskRiQe_a/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790201539513534-1567-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260923 22:13:59.109161 12848 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260923 22:13:59.114117 12848 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260923 22:13:59.114348 12848 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260923 22:13:59.114640 12848 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260923 22:13:59.115065 12848 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260923 22:13:59.115162 12848 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260923 22:13:59.115260 12848 ts_tablet_manager.cc:610] Registered 0 tablets
I20260923 22:13:59.115334 12848 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260923 22:13:59.121878 12848 rpc_server.cc:307] RPC server started. Bound to: 127.1.135.193:33909
I20260923 22:13:59.122360 12848 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskRiQe_a/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790201539513534-1567-0/minicluster-data/ts-0/data/info.pb
I20260923 22:13:59.128172 12978 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.1.135.193:33909 every 8 connection(s)
I20260923 22:13:59.129935 1567 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskRiQe_a/build/release/bin/kudu as pid 12848
I20260923 22:13:59.130048 1567 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskRiQe_a/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790201539513534-1567-0/minicluster-data/ts-0/wal/instance
I20260923 22:13:59.130278 1567 external_mini_cluster.cc:1428] Setting key 2fdb9915da5fcf7e488fcfe6bf2a3639
Sep 23 22:13:59 dist-test-slave-s2fj krb5kdc[12291](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790201639, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.135.193@KRBTEST.COM for kudu/127.1.135.254@KRBTEST.COM
WARNING: no policy specified for kudu/127.1.135.194@KRBTEST.COM; defaulting to no policy
I20260923 22:13:59.141139 12981 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 22:13:59.128846 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.135.254:39417 (local address 127.1.135.193:57087)
0923 22:13:59.131304 (+ 2458us) negotiation.cc:107] Waiting for socket to connect
0923 22:13:59.131342 (+ 38us) client_negotiation.cc:174] Beginning negotiation
0923 22:13:59.131858 (+ 516us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0923 22:13:59.132148 (+ 290us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0923 22:13:59.132152 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0923 22:13:59.132335 (+ 183us) client_negotiation.cc:189] Negotiated authn=SASL
0923 22:13:59.132564 (+ 229us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 22:13:59.132570 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 22:13:59.133608 (+ 1038us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 22:13:59.133610 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 22:13:59.134405 (+ 795us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 22:13:59.134408 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 22:13:59.134926 (+ 518us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 22:13:59.134928 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 22:13:59.135085 (+ 157us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 22:13:59.135704 (+ 619us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0923 22:13:59.135714 (+ 10us) client_negotiation.cc:656] calling sasl_client_start()
0923 22:13:59.137445 (+ 1731us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0923 22:13:59.139553 (+ 2108us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0923 22:13:59.139559 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0923 22:13:59.139561 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0923 22:13:59.139843 (+ 282us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0923 22:13:59.140042 (+ 199us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0923 22:13:59.140043 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0923 22:13:59.140043 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0923 22:13:59.140192 (+ 149us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0923 22:13:59.140491 (+ 299us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0923 22:13:59.140496 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0923 22:13:59.140675 (+ 179us) client_negotiation.cc:769] Sending connection context
0923 22:13:59.140768 (+ 93us) client_negotiation.cc:240] Negotiation successful
0923 22:13:59.140914 (+ 146us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":2418,"thread_start_us":87,"threads_started":1}
I20260923 22:13:59.141562 12982 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 22:13:59.128897 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.135.193:57087 (local address 127.1.135.254:39417)
0923 22:13:59.129045 (+ 148us) server_negotiation.cc:206] Beginning negotiation
0923 22:13:59.129048 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0923 22:13:59.131452 (+ 2404us) server_negotiation.cc:407] Connection header received
0923 22:13:59.131965 (+ 513us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 22:13:59.131966 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 22:13:59.131979 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 22:13:59.132051 (+ 72us) server_negotiation.cc:226] Negotiated authn=SASL
0923 22:13:59.132683 (+ 632us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 22:13:59.133486 (+ 803us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 22:13:59.134496 (+ 1010us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 22:13:59.134820 (+ 324us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 22:13:59.134891 (+ 71us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 22:13:59.137577 (+ 2686us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0923 22:13:59.137579 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0923 22:13:59.137580 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0923 22:13:59.137587 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0923 22:13:59.139464 (+ 1877us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0923 22:13:59.139920 (+ 456us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0923 22:13:59.139921 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0923 22:13:59.139922 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0923 22:13:59.139979 (+ 57us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0923 22:13:59.140259 (+ 280us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0923 22:13:59.140260 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0923 22:13:59.140260 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0923 22:13:59.140389 (+ 129us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0923 22:13:59.140466 (+ 77us) server_negotiation.cc:1035] Waiting for connection context
0923 22:13:59.141370 (+ 904us) server_negotiation.cc:299] Negotiation successful
0923 22:13:59.141450 (+ 80us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":125,"thread_start_us":82,"threads_started":1}
I20260923 22:13:59.142259 12979 heartbeater.cc:347] Connected to a master server at 127.1.135.254:39417
I20260923 22:13:59.142372 12979 heartbeater.cc:464] Registering TS with master...
I20260923 22:13:59.142567 12979 heartbeater.cc:511] Master 127.1.135.254:39417 requested a full tablet report, sending...
I20260923 22:13:59.143093 12767 ts_manager.cc:194] Registered new tserver with Master: 50a730daa7ac44c896d7621ce906e828 (127.1.135.193:33909)
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.1.135.194@KRBTEST.COM" created.
I20260923 22:13:59.143990 12767 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.1.135.193@KRBTEST.COM'} at 127.1.135.193:57087
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.1.135.194 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskRiQe_a/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790201539513534-1567-0/krb5kdc/kudu_127.1.135.194.keytab.
Entry for principal kudu/127.1.135.194 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskRiQe_a/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790201539513534-1567-0/krb5kdc/kudu_127.1.135.194.keytab.
I20260923 22:13:59.164901 1567 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskRiQe_a/build/release/bin/kudu
/tmp/dist-test-taskRiQe_a/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskRiQe_a/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790201539513534-1567-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskRiQe_a/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790201539513534-1567-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskRiQe_a/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790201539513534-1567-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskRiQe_a/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790201539513534-1567-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.1.135.212:52811/kms
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.1.135.194:0
--local_ip_for_outbound_sockets=127.1.135.194
--webserver_interface=127.1.135.194
--webserver_port=0
--tserver_master_addrs=127.1.135.254:39417
--keytab_file=/tmp/dist-test-taskRiQe_a/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790201539513534-1567-0/krb5kdc/kudu_127.1.135.194.keytab
--principal=kudu/127.1.135.194
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.1.135.212:34907
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskRiQe_a/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790201539513534-1567-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskRiQe_a/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790201539513534-1567-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskRiQe_a/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790201539513534-1567-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
Sep 23 22:13:59 dist-test-slave-s2fj krb5kdc[12291](info): AS_REQ (1 etypes {17}) 127.0.0.1: ISSUE: authtime 1790201639, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.135.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
W20260923 22:13:59.283438 12985 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260923 22:13:59.284282 12985 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260923 22:13:59.284365 12985 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260923 22:13:59.286217 12985 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260923 22:13:59.286274 12985 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260923 22:13:59.286314 12985 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.1.135.194
I20260923 22:13:59.288625 12985 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.1.135.212:34907
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-taskRiQe_a/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790201539513534-1567-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskRiQe_a/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790201539513534-1567-0/minicluster-data/ts-1/wal
--ranger_kms_url=127.1.135.212:52811/kms
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskRiQe_a/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790201539513534-1567-0/krb5kdc/kudu_127.1.135.194.keytab
--principal=kudu/127.1.135.194
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.1.135.194:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskRiQe_a/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790201539513534-1567-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.1.135.194
--webserver_port=0
--tserver_master_addrs=127.1.135.254:39417
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.12985
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.1.135.194
--log_dir=/tmp/dist-test-taskRiQe_a/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790201539513534-1567-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 23 Sep 2026 20:53:37 UTC on ccycloud.quasar-rkviyx.root.comops.site
build id 4668083
I20260923 22:13:59.289001 12985 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260923 22:13:59.289287 12985 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260923 22:13:59.295933 12995 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260923 22:13:59.296388 12992 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260923 22:13:59.296665 12993 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260923 22:13:59.296845 12985 server_base.cc:1034] running on GCE node
I20260923 22:13:59.297053 12985 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260923 22:13:59.297336 12985 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260923 22:13:59.298496 12985 hybrid_clock.cc:648] HybridClock initialized: now 1790201639298462 us; error 51 us; skew 500 ppm
Sep 23 22:13:59 dist-test-slave-s2fj krb5kdc[12291](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790201639, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.135.194@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260923 22:13:59.301676 12985 init.cc:377] Logged in from keytab as kudu/127.1.135.194@KRBTEST.COM (short username kudu)
I20260923 22:13:59.302155 12985 webserver.cc:460] Webserver started at http://127.1.135.194:34937/ using document root <none> and password file <none>
I20260923 22:13:59.302397 12985 fs_manager.cc:359] Metadata directory not provided
I20260923 22:13:59.302450 12985 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260923 22:13:59.302620 12985 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
Sep 23 22:13:59 dist-test-slave-s2fj krb5kdc[12291](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790201639, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.135.194@KRBTEST.COM for HTTP/127.1.135.212@KRBTEST.COM
I20260923 22:13:59.315707 12985 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskRiQe_a/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790201539513534-1567-0/minicluster-data/ts-1/data/instance:
uuid: "483ba5db7c5545d396351b7e2d699667"
format_stamp: "Formatted at 2026-09-23 22:13:59 on dist-test-slave-s2fj"
server_key: "30d6c51d8595c8c4f4df85f762af3629"
server_key_iv: "a95b0db15f2b37f10f1a711c30e46e09"
server_key_version: "kuduclusterkey@0"
I20260923 22:13:59.316177 12985 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskRiQe_a/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790201539513534-1567-0/minicluster-data/ts-1/wal/instance:
uuid: "483ba5db7c5545d396351b7e2d699667"
format_stamp: "Formatted at 2026-09-23 22:13:59 on dist-test-slave-s2fj"
server_key: "30d6c51d8595c8c4f4df85f762af3629"
server_key_iv: "a95b0db15f2b37f10f1a711c30e46e09"
server_key_version: "kuduclusterkey@0"
I20260923 22:13:59.319499 12985 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.002s sys 0.000s
I20260923 22:13:59.338856 13003 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260923 22:13:59.339051 12985 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.001s
I20260923 22:13:59.339141 12985 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskRiQe_a/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790201539513534-1567-0/minicluster-data/ts-1/data,/tmp/dist-test-taskRiQe_a/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790201539513534-1567-0/minicluster-data/ts-1/wal
uuid: "483ba5db7c5545d396351b7e2d699667"
format_stamp: "Formatted at 2026-09-23 22:13:59 on dist-test-slave-s2fj"
server_key: "30d6c51d8595c8c4f4df85f762af3629"
server_key_iv: "a95b0db15f2b37f10f1a711c30e46e09"
server_key_version: "kuduclusterkey@0"
I20260923 22:13:59.339248 12985 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskRiQe_a/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790201539513534-1567-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskRiQe_a/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790201539513534-1567-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskRiQe_a/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790201539513534-1567-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260923 22:13:59.372515 12985 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260923 22:13:59.375070 12985 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260923 22:13:59.375377 12985 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260923 22:13:59.375697 12985 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260923 22:13:59.376106 12985 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260923 22:13:59.376166 12985 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260923 22:13:59.376200 12985 ts_tablet_manager.cc:610] Registered 0 tablets
I20260923 22:13:59.376266 12985 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260923 22:13:59.384235 12985 rpc_server.cc:307] RPC server started. Bound to: 127.1.135.194:33083
I20260923 22:13:59.384708 12985 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskRiQe_a/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790201539513534-1567-0/minicluster-data/ts-1/data/info.pb
I20260923 22:13:59.385864 1567 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskRiQe_a/build/release/bin/kudu as pid 12985
I20260923 22:13:59.385962 1567 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskRiQe_a/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790201539513534-1567-0/minicluster-data/ts-1/wal/instance
I20260923 22:13:59.386189 1567 external_mini_cluster.cc:1428] Setting key 1afcef37afbfe2eedef5afdd48851c03
I20260923 22:13:59.388495 13116 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.1.135.194:33083 every 8 connection(s)
Sep 23 22:13:59 dist-test-slave-s2fj krb5kdc[12291](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790201639, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.135.194@KRBTEST.COM for kudu/127.1.135.254@KRBTEST.COM
I20260923 22:13:59.397792 12982 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 22:13:59.385484 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.135.194:46695 (local address 127.1.135.254:39417)
0923 22:13:59.385534 (+ 50us) server_negotiation.cc:206] Beginning negotiation
0923 22:13:59.385536 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0923 22:13:59.385783 (+ 247us) server_negotiation.cc:407] Connection header received
0923 22:13:59.386603 (+ 820us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 22:13:59.386605 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 22:13:59.386620 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 22:13:59.386650 (+ 30us) server_negotiation.cc:226] Negotiated authn=SASL
0923 22:13:59.387575 (+ 925us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 22:13:59.388344 (+ 769us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 22:13:59.389782 (+ 1438us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 22:13:59.390095 (+ 313us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 22:13:59.390136 (+ 41us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 22:13:59.392774 (+ 2638us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0923 22:13:59.392776 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0923 22:13:59.392776 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0923 22:13:59.392783 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0923 22:13:59.394582 (+ 1799us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0923 22:13:59.396553 (+ 1971us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0923 22:13:59.396554 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0923 22:13:59.396554 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0923 22:13:59.396613 (+ 59us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0923 22:13:59.396897 (+ 284us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0923 22:13:59.396898 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0923 22:13:59.396898 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0923 22:13:59.397045 (+ 147us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0923 22:13:59.397084 (+ 39us) server_negotiation.cc:1035] Waiting for connection context
0923 22:13:59.397601 (+ 517us) server_negotiation.cc:299] Negotiation successful
0923 22:13:59.397693 (+ 92us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":24}
I20260923 22:13:59.398608 13119 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 22:13:59.385477 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.135.254:39417 (local address 127.1.135.194:46695)
0923 22:13:59.385650 (+ 173us) negotiation.cc:107] Waiting for socket to connect
0923 22:13:59.385697 (+ 47us) client_negotiation.cc:174] Beginning negotiation
0923 22:13:59.386488 (+ 791us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0923 22:13:59.386940 (+ 452us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0923 22:13:59.386945 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0923 22:13:59.387198 (+ 253us) client_negotiation.cc:189] Negotiated authn=SASL
0923 22:13:59.387467 (+ 269us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 22:13:59.387474 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 22:13:59.388446 (+ 972us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 22:13:59.388446 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 22:13:59.389686 (+ 1240us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 22:13:59.389689 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 22:13:59.390219 (+ 530us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 22:13:59.390220 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 22:13:59.390378 (+ 158us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 22:13:59.390986 (+ 608us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0923 22:13:59.390996 (+ 10us) client_negotiation.cc:656] calling sasl_client_start()
0923 22:13:59.392666 (+ 1670us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0923 22:13:59.394679 (+ 2013us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0923 22:13:59.394685 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0923 22:13:59.394687 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0923 22:13:59.394955 (+ 268us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0923 22:13:59.396682 (+ 1727us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0923 22:13:59.396682 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0923 22:13:59.396683 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0923 22:13:59.396813 (+ 130us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0923 22:13:59.397288 (+ 475us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0923 22:13:59.397293 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0923 22:13:59.397449 (+ 156us) client_negotiation.cc:769] Sending connection context
0923 22:13:59.398133 (+ 684us) client_negotiation.cc:240] Negotiation successful
0923 22:13:59.398293 (+ 160us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":128,"thread_start_us":87,"threads_started":1}
I20260923 22:13:59.399268 13117 heartbeater.cc:347] Connected to a master server at 127.1.135.254:39417
I20260923 22:13:59.399362 13117 heartbeater.cc:464] Registering TS with master...
I20260923 22:13:59.399541 13117 heartbeater.cc:511] Master 127.1.135.254:39417 requested a full tablet report, sending...
I20260923 22:13:59.399936 12767 ts_manager.cc:194] Registered new tserver with Master: 483ba5db7c5545d396351b7e2d699667 (127.1.135.194:33083)
I20260923 22:13:59.400460 12767 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.1.135.194@KRBTEST.COM'} at 127.1.135.194:46695
WARNING: no policy specified for kudu/127.1.135.195@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.1.135.195@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.1.135.195 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskRiQe_a/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790201539513534-1567-0/krb5kdc/kudu_127.1.135.195.keytab.
Entry for principal kudu/127.1.135.195 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskRiQe_a/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790201539513534-1567-0/krb5kdc/kudu_127.1.135.195.keytab.
I20260923 22:13:59.429703 1567 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskRiQe_a/build/release/bin/kudu
/tmp/dist-test-taskRiQe_a/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskRiQe_a/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790201539513534-1567-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskRiQe_a/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790201539513534-1567-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskRiQe_a/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790201539513534-1567-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskRiQe_a/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790201539513534-1567-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.1.135.212:52811/kms
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.1.135.195:0
--local_ip_for_outbound_sockets=127.1.135.195
--webserver_interface=127.1.135.195
--webserver_port=0
--tserver_master_addrs=127.1.135.254:39417
--keytab_file=/tmp/dist-test-taskRiQe_a/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790201539513534-1567-0/krb5kdc/kudu_127.1.135.195.keytab
--principal=kudu/127.1.135.195
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.1.135.212:34907
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskRiQe_a/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790201539513534-1567-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskRiQe_a/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790201539513534-1567-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskRiQe_a/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790201539513534-1567-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260923 22:13:59.552663 13122 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260923 22:13:59.552994 13122 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260923 22:13:59.553076 13122 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260923 22:13:59.554726 13122 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260923 22:13:59.554831 13122 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260923 22:13:59.554914 13122 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.1.135.195
I20260923 22:13:59.557041 13122 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.1.135.212:34907
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-taskRiQe_a/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790201539513534-1567-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskRiQe_a/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790201539513534-1567-0/minicluster-data/ts-2/wal
--ranger_kms_url=127.1.135.212:52811/kms
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskRiQe_a/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790201539513534-1567-0/krb5kdc/kudu_127.1.135.195.keytab
--principal=kudu/127.1.135.195
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.1.135.195:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskRiQe_a/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790201539513534-1567-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.1.135.195
--webserver_port=0
--tserver_master_addrs=127.1.135.254:39417
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.13122
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.1.135.195
--log_dir=/tmp/dist-test-taskRiQe_a/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790201539513534-1567-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 23 Sep 2026 20:53:37 UTC on ccycloud.quasar-rkviyx.root.comops.site
build id 4668083
I20260923 22:13:59.557345 13122 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260923 22:13:59.557629 13122 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260923 22:13:59.565171 13128 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260923 22:13:59.565284 13130 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260923 22:13:59.566406 13122 server_base.cc:1034] running on GCE node
W20260923 22:13:59.566694 13127 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260923 22:13:59.566982 13122 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260923 22:13:59.567250 13122 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260923 22:13:59.568398 13122 hybrid_clock.cc:648] HybridClock initialized: now 1790201639568375 us; error 35 us; skew 500 ppm
Sep 23 22:13:59 dist-test-slave-s2fj krb5kdc[12291](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790201639, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.135.195@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260923 22:13:59.572281 13122 init.cc:377] Logged in from keytab as kudu/127.1.135.195@KRBTEST.COM (short username kudu)
I20260923 22:13:59.572803 13122 webserver.cc:460] Webserver started at http://127.1.135.195:45477/ using document root <none> and password file <none>
I20260923 22:13:59.573081 13122 fs_manager.cc:359] Metadata directory not provided
I20260923 22:13:59.573151 13122 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260923 22:13:59.573305 13122 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
Sep 23 22:13:59 dist-test-slave-s2fj krb5kdc[12291](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790201639, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.135.195@KRBTEST.COM for HTTP/127.1.135.212@KRBTEST.COM
I20260923 22:13:59.590979 13122 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskRiQe_a/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790201539513534-1567-0/minicluster-data/ts-2/data/instance:
uuid: "825d87e809214dbcab7e3e5e2455d8f0"
format_stamp: "Formatted at 2026-09-23 22:13:59 on dist-test-slave-s2fj"
server_key: "532f0ebe482c394e537c928d5276f130"
server_key_iv: "694382c7b5e8e1f0c6a5dc784be295bb"
server_key_version: "kuduclusterkey@0"
I20260923 22:13:59.591508 13122 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskRiQe_a/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790201539513534-1567-0/minicluster-data/ts-2/wal/instance:
uuid: "825d87e809214dbcab7e3e5e2455d8f0"
format_stamp: "Formatted at 2026-09-23 22:13:59 on dist-test-slave-s2fj"
server_key: "532f0ebe482c394e537c928d5276f130"
server_key_iv: "694382c7b5e8e1f0c6a5dc784be295bb"
server_key_version: "kuduclusterkey@0"
I20260923 22:13:59.594054 13122 fs_manager.cc:691] Time spent creating directory manager: real 0.002s user 0.001s sys 0.000s
I20260923 22:13:59.621543 13137 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260923 22:13:59.621826 13122 fs_manager.cc:725] Time spent opening block manager: real 0.006s user 0.001s sys 0.000s
I20260923 22:13:59.621922 13122 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskRiQe_a/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790201539513534-1567-0/minicluster-data/ts-2/data,/tmp/dist-test-taskRiQe_a/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790201539513534-1567-0/minicluster-data/ts-2/wal
uuid: "825d87e809214dbcab7e3e5e2455d8f0"
format_stamp: "Formatted at 2026-09-23 22:13:59 on dist-test-slave-s2fj"
server_key: "532f0ebe482c394e537c928d5276f130"
server_key_iv: "694382c7b5e8e1f0c6a5dc784be295bb"
server_key_version: "kuduclusterkey@0"
I20260923 22:13:59.622018 13122 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskRiQe_a/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790201539513534-1567-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskRiQe_a/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790201539513534-1567-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskRiQe_a/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790201539513534-1567-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260923 22:13:59.656847 13122 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260923 22:13:59.659307 13122 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260923 22:13:59.659533 13122 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260923 22:13:59.659812 13122 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260923 22:13:59.660274 13122 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260923 22:13:59.660456 13122 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260923 22:13:59.660539 13122 ts_tablet_manager.cc:610] Registered 0 tablets
I20260923 22:13:59.660600 13122 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260923 22:13:59.667304 13122 rpc_server.cc:307] RPC server started. Bound to: 127.1.135.195:38139
I20260923 22:13:59.667908 13122 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskRiQe_a/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790201539513534-1567-0/minicluster-data/ts-2/data/info.pb
I20260923 22:13:59.676908 1567 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskRiQe_a/build/release/bin/kudu as pid 13122
I20260923 22:13:59.677021 1567 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskRiQe_a/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790201539513534-1567-0/minicluster-data/ts-2/wal/instance
I20260923 22:13:59.677214 1567 external_mini_cluster.cc:1428] Setting key 79052494620613647956b8a7785cdb1a
Sep 23 22:13:59 dist-test-slave-s2fj krb5kdc[12291](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790201639, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.135.195@KRBTEST.COM for kudu/127.1.135.254@KRBTEST.COM
I20260923 22:13:59.670397 13250 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.1.135.195:38139 every 8 connection(s)
I20260923 22:13:59.682417 12982 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 22:13:59.671268 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.135.195:35809 (local address 127.1.135.254:39417)
0923 22:13:59.671311 (+ 43us) server_negotiation.cc:206] Beginning negotiation
0923 22:13:59.671313 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0923 22:13:59.671636 (+ 323us) server_negotiation.cc:407] Connection header received
0923 22:13:59.672339 (+ 703us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 22:13:59.672340 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 22:13:59.672352 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 22:13:59.672384 (+ 32us) server_negotiation.cc:226] Negotiated authn=SASL
0923 22:13:59.673184 (+ 800us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 22:13:59.674001 (+ 817us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 22:13:59.675464 (+ 1463us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 22:13:59.675787 (+ 323us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 22:13:59.675829 (+ 42us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 22:13:59.678816 (+ 2987us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0923 22:13:59.678818 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0923 22:13:59.678818 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0923 22:13:59.678825 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0923 22:13:59.680758 (+ 1933us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0923 22:13:59.681321 (+ 563us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0923 22:13:59.681322 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0923 22:13:59.681322 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0923 22:13:59.681377 (+ 55us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0923 22:13:59.681639 (+ 262us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0923 22:13:59.681640 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0923 22:13:59.681640 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0923 22:13:59.681776 (+ 136us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0923 22:13:59.681818 (+ 42us) server_negotiation.cc:1035] Waiting for connection context
0923 22:13:59.682244 (+ 426us) server_negotiation.cc:299] Negotiation successful
0923 22:13:59.682324 (+ 80us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":20}
I20260923 22:13:59.683168 13253 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 22:13:59.670982 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.135.254:39417 (local address 127.1.135.195:35809)
0923 22:13:59.671512 (+ 530us) negotiation.cc:107] Waiting for socket to connect
0923 22:13:59.671555 (+ 43us) client_negotiation.cc:174] Beginning negotiation
0923 22:13:59.672215 (+ 660us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0923 22:13:59.672564 (+ 349us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0923 22:13:59.672569 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0923 22:13:59.672808 (+ 239us) client_negotiation.cc:189] Negotiated authn=SASL
0923 22:13:59.673072 (+ 264us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 22:13:59.673078 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 22:13:59.674114 (+ 1036us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 22:13:59.674115 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 22:13:59.675351 (+ 1236us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 22:13:59.675354 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 22:13:59.675914 (+ 560us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 22:13:59.675914 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 22:13:59.676048 (+ 134us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 22:13:59.676645 (+ 597us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0923 22:13:59.676664 (+ 19us) client_negotiation.cc:656] calling sasl_client_start()
0923 22:13:59.678711 (+ 2047us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0923 22:13:59.680955 (+ 2244us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0923 22:13:59.680960 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0923 22:13:59.680962 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0923 22:13:59.681231 (+ 269us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0923 22:13:59.681447 (+ 216us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0923 22:13:59.681448 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0923 22:13:59.681448 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0923 22:13:59.681567 (+ 119us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0923 22:13:59.681863 (+ 296us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0923 22:13:59.681869 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0923 22:13:59.682080 (+ 211us) client_negotiation.cc:769] Sending connection context
0923 22:13:59.682742 (+ 662us) client_negotiation.cc:240] Negotiation successful
0923 22:13:59.682898 (+ 156us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":493,"thread_start_us":78,"threads_started":1}
I20260923 22:13:59.683784 13251 heartbeater.cc:347] Connected to a master server at 127.1.135.254:39417
I20260923 22:13:59.683871 13251 heartbeater.cc:464] Registering TS with master...
I20260923 22:13:59.684038 13251 heartbeater.cc:511] Master 127.1.135.254:39417 requested a full tablet report, sending...
I20260923 22:13:59.684450 12767 ts_manager.cc:194] Registered new tserver with Master: 825d87e809214dbcab7e3e5e2455d8f0 (127.1.135.195:38139)
I20260923 22:13:59.684952 12767 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.1.135.195@KRBTEST.COM'} at 127.1.135.195:35809
I20260923 22:13:59.690471 1567 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260923 22:13:59.700940 12982 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 22:13:59.692363 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40200 (local address 127.1.135.254:39417)
0923 22:13:59.692407 (+ 44us) server_negotiation.cc:206] Beginning negotiation
0923 22:13:59.692409 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0923 22:13:59.693686 (+ 1277us) server_negotiation.cc:407] Connection header received
0923 22:13:59.693787 (+ 101us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 22:13:59.693788 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 22:13:59.693798 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 22:13:59.693827 (+ 29us) server_negotiation.cc:226] Negotiated authn=SASL
0923 22:13:59.694628 (+ 801us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 22:13:59.695245 (+ 617us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 22:13:59.695955 (+ 710us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 22:13:59.696262 (+ 307us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 22:13:59.696327 (+ 65us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 22:13:59.697938 (+ 1611us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0923 22:13:59.697940 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0923 22:13:59.697941 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0923 22:13:59.697948 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0923 22:13:59.699710 (+ 1762us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0923 22:13:59.700133 (+ 423us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0923 22:13:59.700134 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0923 22:13:59.700135 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0923 22:13:59.700189 (+ 54us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0923 22:13:59.700378 (+ 189us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0923 22:13:59.700379 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0923 22:13:59.700379 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0923 22:13:59.700490 (+ 111us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0923 22:13:59.700556 (+ 66us) server_negotiation.cc:1035] Waiting for connection context
0923 22:13:59.700780 (+ 224us) server_negotiation.cc:299] Negotiation successful
0923 22:13:59.700853 (+ 73us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":24}
I20260923 22:13:59.703001 12767 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:40200:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260923 22:13:59.705389 12767 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260923 22:13:59.718907 13266 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 22:13:59.708247 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.135.195:38139 (local address 127.0.0.1:54604)
0923 22:13:59.709549 (+ 1302us) negotiation.cc:107] Waiting for socket to connect
0923 22:13:59.709590 (+ 41us) client_negotiation.cc:174] Beginning negotiation
0923 22:13:59.709667 (+ 77us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0923 22:13:59.711539 (+ 1872us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0923 22:13:59.711540 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0923 22:13:59.711541 (+ 1us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0923 22:13:59.711580 (+ 39us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 22:13:59.711582 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 22:13:59.715198 (+ 3616us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 22:13:59.715199 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 22:13:59.716637 (+ 1438us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 22:13:59.716638 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 22:13:59.718615 (+ 1977us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 22:13:59.718616 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 22:13:59.718682 (+ 66us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 22:13:59.718704 (+ 22us) client_negotiation.cc:769] Sending connection context
0923 22:13:59.718763 (+ 59us) client_negotiation.cc:240] Negotiation successful
0923 22:13:59.718810 (+ 47us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1269,"thread_start_us":64,"threads_started":1}
I20260923 22:13:59.719342 13268 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 22:13:59.708697 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.135.193:33909 (local address 127.0.0.1:57506)
0923 22:13:59.709773 (+ 1076us) negotiation.cc:107] Waiting for socket to connect
0923 22:13:59.709782 (+ 9us) client_negotiation.cc:174] Beginning negotiation
0923 22:13:59.709816 (+ 34us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0923 22:13:59.710128 (+ 312us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0923 22:13:59.710131 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0923 22:13:59.710133 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0923 22:13:59.710191 (+ 58us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 22:13:59.710193 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 22:13:59.712730 (+ 2537us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 22:13:59.712731 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 22:13:59.714326 (+ 1595us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 22:13:59.714329 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 22:13:59.717602 (+ 3273us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 22:13:59.717603 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 22:13:59.717699 (+ 96us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 22:13:59.719177 (+ 1478us) client_negotiation.cc:769] Sending connection context
0923 22:13:59.719236 (+ 59us) client_negotiation.cc:240] Negotiation successful
0923 22:13:59.719274 (+ 38us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1063,"mutex_wait_us":1470,"thread_start_us":60,"threads_started":1}
I20260923 22:13:59.720273 13267 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 22:13:59.708888 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57506 (local address 127.1.135.193:33909)
0923 22:13:59.709915 (+ 1027us) server_negotiation.cc:206] Beginning negotiation
0923 22:13:59.709920 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0923 22:13:59.709930 (+ 10us) server_negotiation.cc:407] Connection header received
0923 22:13:59.709964 (+ 34us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 22:13:59.709966 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 22:13:59.709982 (+ 16us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 22:13:59.710049 (+ 67us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0923 22:13:59.710268 (+ 219us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 22:13:59.712619 (+ 2351us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 22:13:59.716731 (+ 4112us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 22:13:59.717441 (+ 710us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 22:13:59.717508 (+ 67us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 22:13:59.717570 (+ 62us) server_negotiation.cc:1035] Waiting for connection context
0923 22:13:59.720151 (+ 2581us) server_negotiation.cc:299] Negotiation successful
0923 22:13:59.720200 (+ 49us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":993,"thread_start_us":47,"threads_started":1}
I20260923 22:13:59.721082 12913 tablet_service.cc:1467] Processing CreateTablet for tablet 363acddfee544ea4b5e4d27f3c4f4fbe (DEFAULT_TABLE table=test-table [id=478801440d8246b69098108bee99f2a1]), partition=RANGE (key) PARTITION UNBOUNDED
I20260923 22:13:59.721457 12913 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 363acddfee544ea4b5e4d27f3c4f4fbe. 1 dirs total, 0 dirs full, 0 dirs failed
I20260923 22:13:59.722843 13264 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 22:13:59.709379 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.135.194:33083 (local address 127.0.0.1:35340)
0923 22:13:59.711160 (+ 1781us) negotiation.cc:107] Waiting for socket to connect
0923 22:13:59.711171 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0923 22:13:59.711213 (+ 42us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0923 22:13:59.716390 (+ 5177us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0923 22:13:59.716391 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0923 22:13:59.716392 (+ 1us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0923 22:13:59.716425 (+ 33us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 22:13:59.716427 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 22:13:59.717600 (+ 1173us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 22:13:59.717601 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 22:13:59.718690 (+ 1089us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 22:13:59.718692 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 22:13:59.722217 (+ 3525us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 22:13:59.722218 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 22:13:59.722682 (+ 464us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 22:13:59.722686 (+ 4us) client_negotiation.cc:769] Sending connection context
0923 22:13:59.722740 (+ 54us) client_negotiation.cc:240] Negotiation successful
0923 22:13:59.722777 (+ 37us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1765,"mutex_wait_us":7,"thread_start_us":70,"threads_started":1}
I20260923 22:13:59.724308 13269 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 22:13:59.709480 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35340 (local address 127.1.135.194:33083)
0923 22:13:59.716166 (+ 6686us) server_negotiation.cc:206] Beginning negotiation
0923 22:13:59.716170 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0923 22:13:59.716181 (+ 11us) server_negotiation.cc:407] Connection header received
0923 22:13:59.716221 (+ 40us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 22:13:59.716222 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 22:13:59.716238 (+ 16us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 22:13:59.716312 (+ 74us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0923 22:13:59.716492 (+ 180us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 22:13:59.717515 (+ 1023us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 22:13:59.721327 (+ 3812us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 22:13:59.722032 (+ 705us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 22:13:59.722103 (+ 71us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 22:13:59.722181 (+ 78us) server_negotiation.cc:1035] Waiting for connection context
0923 22:13:59.724154 (+ 1973us) server_negotiation.cc:299] Negotiation successful
0923 22:13:59.724212 (+ 58us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":6636,"thread_start_us":68,"threads_started":1}
I20260923 22:13:59.724951 13033 tablet_service.cc:1467] Processing CreateTablet for tablet 363acddfee544ea4b5e4d27f3c4f4fbe (DEFAULT_TABLE table=test-table [id=478801440d8246b69098108bee99f2a1]), partition=RANGE (key) PARTITION UNBOUNDED
I20260923 22:13:59.725293 13033 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 363acddfee544ea4b5e4d27f3c4f4fbe. 1 dirs total, 0 dirs full, 0 dirs failed
I20260923 22:13:59.726639 13265 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 22:13:59.708482 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54604 (local address 127.1.135.195:38139)
0923 22:13:59.711332 (+ 2850us) server_negotiation.cc:206] Beginning negotiation
0923 22:13:59.711337 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0923 22:13:59.711345 (+ 8us) server_negotiation.cc:407] Connection header received
0923 22:13:59.711382 (+ 37us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 22:13:59.711383 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 22:13:59.711398 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 22:13:59.711467 (+ 69us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0923 22:13:59.711651 (+ 184us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 22:13:59.715099 (+ 3448us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 22:13:59.717752 (+ 2653us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 22:13:59.718447 (+ 695us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 22:13:59.718526 (+ 79us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 22:13:59.718586 (+ 60us) server_negotiation.cc:1035] Waiting for connection context
0923 22:13:59.726507 (+ 7921us) server_negotiation.cc:299] Negotiation successful
0923 22:13:59.726555 (+ 48us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":2812,"thread_start_us":70,"threads_started":1}
I20260923 22:13:59.727406 13185 tablet_service.cc:1467] Processing CreateTablet for tablet 363acddfee544ea4b5e4d27f3c4f4fbe (DEFAULT_TABLE table=test-table [id=478801440d8246b69098108bee99f2a1]), partition=RANGE (key) PARTITION UNBOUNDED
I20260923 22:13:59.727752 13185 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 363acddfee544ea4b5e4d27f3c4f4fbe. 1 dirs total, 0 dirs full, 0 dirs failed
I20260923 22:13:59.731686 13272 tablet_bootstrap.cc:492] T 363acddfee544ea4b5e4d27f3c4f4fbe P 825d87e809214dbcab7e3e5e2455d8f0: Bootstrap starting.
I20260923 22:13:59.732750 13272 tablet_bootstrap.cc:654] T 363acddfee544ea4b5e4d27f3c4f4fbe P 825d87e809214dbcab7e3e5e2455d8f0: Neither blocks nor log segments found. Creating new log.
I20260923 22:13:59.733098 13272 log.cc:824] T 363acddfee544ea4b5e4d27f3c4f4fbe P 825d87e809214dbcab7e3e5e2455d8f0: Log is configured to *not* fsync() on all Append() calls
I20260923 22:13:59.733451 13270 tablet_bootstrap.cc:492] T 363acddfee544ea4b5e4d27f3c4f4fbe P 50a730daa7ac44c896d7621ce906e828: Bootstrap starting.
I20260923 22:13:59.734174 13270 tablet_bootstrap.cc:654] T 363acddfee544ea4b5e4d27f3c4f4fbe P 50a730daa7ac44c896d7621ce906e828: Neither blocks nor log segments found. Creating new log.
I20260923 22:13:59.734536 13270 log.cc:824] T 363acddfee544ea4b5e4d27f3c4f4fbe P 50a730daa7ac44c896d7621ce906e828: Log is configured to *not* fsync() on all Append() calls
I20260923 22:13:59.734931 13271 tablet_bootstrap.cc:492] T 363acddfee544ea4b5e4d27f3c4f4fbe P 483ba5db7c5545d396351b7e2d699667: Bootstrap starting.
I20260923 22:13:59.735975 13271 tablet_bootstrap.cc:654] T 363acddfee544ea4b5e4d27f3c4f4fbe P 483ba5db7c5545d396351b7e2d699667: Neither blocks nor log segments found. Creating new log.
I20260923 22:13:59.736341 13271 log.cc:824] T 363acddfee544ea4b5e4d27f3c4f4fbe P 483ba5db7c5545d396351b7e2d699667: Log is configured to *not* fsync() on all Append() calls
I20260923 22:13:59.737169 13272 tablet_bootstrap.cc:492] T 363acddfee544ea4b5e4d27f3c4f4fbe P 825d87e809214dbcab7e3e5e2455d8f0: No bootstrap required, opened a new log
I20260923 22:13:59.737286 13272 ts_tablet_manager.cc:1397] T 363acddfee544ea4b5e4d27f3c4f4fbe P 825d87e809214dbcab7e3e5e2455d8f0: Time spent bootstrapping tablet: real 0.006s user 0.002s sys 0.000s
I20260923 22:13:59.740255 13272 raft_consensus.cc:348] T 363acddfee544ea4b5e4d27f3c4f4fbe P 825d87e809214dbcab7e3e5e2455d8f0 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "825d87e809214dbcab7e3e5e2455d8f0" member_type: VOTER last_known_addr { host: "127.1.135.195" port: 38139 } } peers { permanent_uuid: "50a730daa7ac44c896d7621ce906e828" member_type: VOTER last_known_addr { host: "127.1.135.193" port: 33909 } } peers { permanent_uuid: "483ba5db7c5545d396351b7e2d699667" member_type: VOTER last_known_addr { host: "127.1.135.194" port: 33083 } }
I20260923 22:13:59.740535 13272 raft_consensus.cc:374] T 363acddfee544ea4b5e4d27f3c4f4fbe P 825d87e809214dbcab7e3e5e2455d8f0 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260923 22:13:59.740628 13272 raft_consensus.cc:729] T 363acddfee544ea4b5e4d27f3c4f4fbe P 825d87e809214dbcab7e3e5e2455d8f0 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 825d87e809214dbcab7e3e5e2455d8f0, State: Initialized, Role: FOLLOWER
I20260923 22:13:59.740849 13272 consensus_queue.cc:260] T 363acddfee544ea4b5e4d27f3c4f4fbe P 825d87e809214dbcab7e3e5e2455d8f0 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "825d87e809214dbcab7e3e5e2455d8f0" member_type: VOTER last_known_addr { host: "127.1.135.195" port: 38139 } } peers { permanent_uuid: "50a730daa7ac44c896d7621ce906e828" member_type: VOTER last_known_addr { host: "127.1.135.193" port: 33909 } } peers { permanent_uuid: "483ba5db7c5545d396351b7e2d699667" member_type: VOTER last_known_addr { host: "127.1.135.194" port: 33083 } }
I20260923 22:13:59.741218 13272 ts_tablet_manager.cc:1428] T 363acddfee544ea4b5e4d27f3c4f4fbe P 825d87e809214dbcab7e3e5e2455d8f0: Time spent starting tablet: real 0.004s user 0.003s sys 0.000s
I20260923 22:13:59.741381 13251 heartbeater.cc:503] Master 127.1.135.254:39417 was elected leader, sending a full tablet report...
I20260923 22:13:59.742928 13270 tablet_bootstrap.cc:492] T 363acddfee544ea4b5e4d27f3c4f4fbe P 50a730daa7ac44c896d7621ce906e828: No bootstrap required, opened a new log
I20260923 22:13:59.743048 13270 ts_tablet_manager.cc:1397] T 363acddfee544ea4b5e4d27f3c4f4fbe P 50a730daa7ac44c896d7621ce906e828: Time spent bootstrapping tablet: real 0.010s user 0.002s sys 0.000s
I20260923 22:13:59.746469 13270 raft_consensus.cc:348] T 363acddfee544ea4b5e4d27f3c4f4fbe P 50a730daa7ac44c896d7621ce906e828 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "825d87e809214dbcab7e3e5e2455d8f0" member_type: VOTER last_known_addr { host: "127.1.135.195" port: 38139 } } peers { permanent_uuid: "50a730daa7ac44c896d7621ce906e828" member_type: VOTER last_known_addr { host: "127.1.135.193" port: 33909 } } peers { permanent_uuid: "483ba5db7c5545d396351b7e2d699667" member_type: VOTER last_known_addr { host: "127.1.135.194" port: 33083 } }
I20260923 22:13:59.746706 13270 raft_consensus.cc:374] T 363acddfee544ea4b5e4d27f3c4f4fbe P 50a730daa7ac44c896d7621ce906e828 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260923 22:13:59.746799 13270 raft_consensus.cc:729] T 363acddfee544ea4b5e4d27f3c4f4fbe P 50a730daa7ac44c896d7621ce906e828 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 50a730daa7ac44c896d7621ce906e828, State: Initialized, Role: FOLLOWER
I20260923 22:13:59.746992 13270 consensus_queue.cc:260] T 363acddfee544ea4b5e4d27f3c4f4fbe P 50a730daa7ac44c896d7621ce906e828 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "825d87e809214dbcab7e3e5e2455d8f0" member_type: VOTER last_known_addr { host: "127.1.135.195" port: 38139 } } peers { permanent_uuid: "50a730daa7ac44c896d7621ce906e828" member_type: VOTER last_known_addr { host: "127.1.135.193" port: 33909 } } peers { permanent_uuid: "483ba5db7c5545d396351b7e2d699667" member_type: VOTER last_known_addr { host: "127.1.135.194" port: 33083 } }
I20260923 22:13:59.747321 13270 ts_tablet_manager.cc:1428] T 363acddfee544ea4b5e4d27f3c4f4fbe P 50a730daa7ac44c896d7621ce906e828: Time spent starting tablet: real 0.004s user 0.003s sys 0.000s
I20260923 22:13:59.747517 12979 heartbeater.cc:503] Master 127.1.135.254:39417 was elected leader, sending a full tablet report...
I20260923 22:13:59.749111 13271 tablet_bootstrap.cc:492] T 363acddfee544ea4b5e4d27f3c4f4fbe P 483ba5db7c5545d396351b7e2d699667: No bootstrap required, opened a new log
I20260923 22:13:59.749239 13271 ts_tablet_manager.cc:1397] T 363acddfee544ea4b5e4d27f3c4f4fbe P 483ba5db7c5545d396351b7e2d699667: Time spent bootstrapping tablet: real 0.014s user 0.002s sys 0.000s
I20260923 22:13:59.751293 13271 raft_consensus.cc:348] T 363acddfee544ea4b5e4d27f3c4f4fbe P 483ba5db7c5545d396351b7e2d699667 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "825d87e809214dbcab7e3e5e2455d8f0" member_type: VOTER last_known_addr { host: "127.1.135.195" port: 38139 } } peers { permanent_uuid: "50a730daa7ac44c896d7621ce906e828" member_type: VOTER last_known_addr { host: "127.1.135.193" port: 33909 } } peers { permanent_uuid: "483ba5db7c5545d396351b7e2d699667" member_type: VOTER last_known_addr { host: "127.1.135.194" port: 33083 } }
I20260923 22:13:59.751526 13271 raft_consensus.cc:374] T 363acddfee544ea4b5e4d27f3c4f4fbe P 483ba5db7c5545d396351b7e2d699667 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260923 22:13:59.751610 13271 raft_consensus.cc:729] T 363acddfee544ea4b5e4d27f3c4f4fbe P 483ba5db7c5545d396351b7e2d699667 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 483ba5db7c5545d396351b7e2d699667, State: Initialized, Role: FOLLOWER
I20260923 22:13:59.751811 13271 consensus_queue.cc:260] T 363acddfee544ea4b5e4d27f3c4f4fbe P 483ba5db7c5545d396351b7e2d699667 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "825d87e809214dbcab7e3e5e2455d8f0" member_type: VOTER last_known_addr { host: "127.1.135.195" port: 38139 } } peers { permanent_uuid: "50a730daa7ac44c896d7621ce906e828" member_type: VOTER last_known_addr { host: "127.1.135.193" port: 33909 } } peers { permanent_uuid: "483ba5db7c5545d396351b7e2d699667" member_type: VOTER last_known_addr { host: "127.1.135.194" port: 33083 } }
I20260923 22:13:59.752163 13271 ts_tablet_manager.cc:1428] T 363acddfee544ea4b5e4d27f3c4f4fbe P 483ba5db7c5545d396351b7e2d699667: Time spent starting tablet: real 0.003s user 0.003s sys 0.000s
I20260923 22:13:59.752354 13117 heartbeater.cc:503] Master 127.1.135.254:39417 was elected leader, sending a full tablet report...
Sep 23 22:13:59 dist-test-slave-s2fj krb5kdc[12291](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790201639, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.135.254@KRBTEST.COM for HTTP/127.1.135.212@KRBTEST.COM
W20260923 22:13:59.879171 12980 tablet.cc:2369] T 363acddfee544ea4b5e4d27f3c4f4fbe P 50a730daa7ac44c896d7621ce906e828: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260923 22:13:59.889623 13118 tablet.cc:2369] T 363acddfee544ea4b5e4d27f3c4f4fbe P 483ba5db7c5545d396351b7e2d699667: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260923 22:13:59.920380 13276 raft_consensus.cc:482] T 363acddfee544ea4b5e4d27f3c4f4fbe P 825d87e809214dbcab7e3e5e2455d8f0 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260923 22:13:59.920534 13276 raft_consensus.cc:504] T 363acddfee544ea4b5e4d27f3c4f4fbe P 825d87e809214dbcab7e3e5e2455d8f0 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "825d87e809214dbcab7e3e5e2455d8f0" member_type: VOTER last_known_addr { host: "127.1.135.195" port: 38139 } } peers { permanent_uuid: "50a730daa7ac44c896d7621ce906e828" member_type: VOTER last_known_addr { host: "127.1.135.193" port: 33909 } } peers { permanent_uuid: "483ba5db7c5545d396351b7e2d699667" member_type: VOTER last_known_addr { host: "127.1.135.194" port: 33083 } }
I20260923 22:13:59.920948 13276 leader_election.cc:288] T 363acddfee544ea4b5e4d27f3c4f4fbe P 825d87e809214dbcab7e3e5e2455d8f0 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 50a730daa7ac44c896d7621ce906e828 (127.1.135.193:33909), 483ba5db7c5545d396351b7e2d699667 (127.1.135.194:33083)
W20260923 22:13:59.921710 13252 tablet.cc:2369] T 363acddfee544ea4b5e4d27f3c4f4fbe P 825d87e809214dbcab7e3e5e2455d8f0: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260923 22:13:59.925091 13269 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 22:13:59.921567 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.135.195:46051 (local address 127.1.135.194:33083)
0923 22:13:59.921614 (+ 47us) server_negotiation.cc:206] Beginning negotiation
0923 22:13:59.921616 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0923 22:13:59.921625 (+ 9us) server_negotiation.cc:407] Connection header received
0923 22:13:59.921653 (+ 28us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 22:13:59.921654 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 22:13:59.921666 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 22:13:59.921715 (+ 49us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0923 22:13:59.921930 (+ 215us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 22:13:59.922660 (+ 730us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 22:13:59.924116 (+ 1456us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 22:13:59.924695 (+ 579us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 22:13:59.924746 (+ 51us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 22:13:59.924755 (+ 9us) server_negotiation.cc:1035] Waiting for connection context
0923 22:13:59.924975 (+ 220us) server_negotiation.cc:299] Negotiation successful
0923 22:13:59.925024 (+ 49us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":24}
I20260923 22:13:59.925163 13253 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 22:13:59.921110 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.135.193:33909 (local address 127.1.135.195:40341)
0923 22:13:59.921165 (+ 55us) negotiation.cc:107] Waiting for socket to connect
0923 22:13:59.921179 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0923 22:13:59.921227 (+ 48us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0923 22:13:59.922150 (+ 923us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0923 22:13:59.922151 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0923 22:13:59.922152 (+ 1us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0923 22:13:59.922207 (+ 55us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 22:13:59.922208 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 22:13:59.923070 (+ 862us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 22:13:59.923071 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 22:13:59.924152 (+ 1081us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 22:13:59.924154 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 22:13:59.924906 (+ 752us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 22:13:59.924907 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 22:13:59.924987 (+ 80us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 22:13:59.924989 (+ 2us) client_negotiation.cc:769] Sending connection context
0923 22:13:59.925032 (+ 43us) client_negotiation.cc:240] Negotiation successful
0923 22:13:59.925076 (+ 44us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":24}
I20260923 22:13:59.925328 13279 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 22:13:59.921252 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.135.194:33083 (local address 127.1.135.195:46051)
0923 22:13:59.921443 (+ 191us) negotiation.cc:107] Waiting for socket to connect
0923 22:13:59.921453 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0923 22:13:59.921501 (+ 48us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0923 22:13:59.921771 (+ 270us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0923 22:13:59.921772 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0923 22:13:59.921774 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0923 22:13:59.921849 (+ 75us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 22:13:59.921851 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 22:13:59.922752 (+ 901us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 22:13:59.922753 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 22:13:59.923987 (+ 1234us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 22:13:59.923989 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 22:13:59.924816 (+ 827us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 22:13:59.924816 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 22:13:59.924891 (+ 75us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 22:13:59.924895 (+ 4us) client_negotiation.cc:769] Sending connection context
0923 22:13:59.925184 (+ 289us) client_negotiation.cc:240] Negotiation successful
0923 22:13:59.925225 (+ 41us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":163,"thread_start_us":102,"threads_started":1}
I20260923 22:13:59.925891 13267 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 22:13:59.921516 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.135.195:40341 (local address 127.1.135.193:33909)
0923 22:13:59.921969 (+ 453us) server_negotiation.cc:206] Beginning negotiation
0923 22:13:59.921971 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0923 22:13:59.921979 (+ 8us) server_negotiation.cc:407] Connection header received
0923 22:13:59.922002 (+ 23us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 22:13:59.922003 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 22:13:59.922012 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 22:13:59.922061 (+ 49us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0923 22:13:59.922329 (+ 268us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 22:13:59.922992 (+ 663us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 22:13:59.924235 (+ 1243us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 22:13:59.924813 (+ 578us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 22:13:59.924870 (+ 57us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 22:13:59.924879 (+ 9us) server_negotiation.cc:1035] Waiting for connection context
0923 22:13:59.925765 (+ 886us) server_negotiation.cc:299] Negotiation successful
0923 22:13:59.925822 (+ 57us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":426}
I20260923 22:13:59.925840 13055 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "363acddfee544ea4b5e4d27f3c4f4fbe" candidate_uuid: "825d87e809214dbcab7e3e5e2455d8f0" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "483ba5db7c5545d396351b7e2d699667" is_pre_election: true
I20260923 22:13:59.925993 13055 raft_consensus.cc:2454] T 363acddfee544ea4b5e4d27f3c4f4fbe P 483ba5db7c5545d396351b7e2d699667 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 825d87e809214dbcab7e3e5e2455d8f0 in term 0.
I20260923 22:13:59.926156 12933 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "363acddfee544ea4b5e4d27f3c4f4fbe" candidate_uuid: "825d87e809214dbcab7e3e5e2455d8f0" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "50a730daa7ac44c896d7621ce906e828" is_pre_election: true
I20260923 22:13:59.926709 12933 raft_consensus.cc:2454] T 363acddfee544ea4b5e4d27f3c4f4fbe P 50a730daa7ac44c896d7621ce906e828 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 825d87e809214dbcab7e3e5e2455d8f0 in term 0.
I20260923 22:13:59.926704 13138 leader_election.cc:302] T 363acddfee544ea4b5e4d27f3c4f4fbe P 825d87e809214dbcab7e3e5e2455d8f0 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 483ba5db7c5545d396351b7e2d699667, 825d87e809214dbcab7e3e5e2455d8f0; no voters:
I20260923 22:13:59.926854 13276 raft_consensus.cc:2781] T 363acddfee544ea4b5e4d27f3c4f4fbe P 825d87e809214dbcab7e3e5e2455d8f0 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260923 22:13:59.926901 13276 raft_consensus.cc:482] T 363acddfee544ea4b5e4d27f3c4f4fbe P 825d87e809214dbcab7e3e5e2455d8f0 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260923 22:13:59.926929 13276 raft_consensus.cc:3037] T 363acddfee544ea4b5e4d27f3c4f4fbe P 825d87e809214dbcab7e3e5e2455d8f0 [term 0 FOLLOWER]: Advancing to term 1
I20260923 22:13:59.927798 13276 raft_consensus.cc:504] T 363acddfee544ea4b5e4d27f3c4f4fbe P 825d87e809214dbcab7e3e5e2455d8f0 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "825d87e809214dbcab7e3e5e2455d8f0" member_type: VOTER last_known_addr { host: "127.1.135.195" port: 38139 } } peers { permanent_uuid: "50a730daa7ac44c896d7621ce906e828" member_type: VOTER last_known_addr { host: "127.1.135.193" port: 33909 } } peers { permanent_uuid: "483ba5db7c5545d396351b7e2d699667" member_type: VOTER last_known_addr { host: "127.1.135.194" port: 33083 } }
I20260923 22:13:59.927942 13276 leader_election.cc:288] T 363acddfee544ea4b5e4d27f3c4f4fbe P 825d87e809214dbcab7e3e5e2455d8f0 [CANDIDATE]: Term 1 election: Requested vote from peers 50a730daa7ac44c896d7621ce906e828 (127.1.135.193:33909), 483ba5db7c5545d396351b7e2d699667 (127.1.135.194:33083)
I20260923 22:13:59.928135 12933 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "363acddfee544ea4b5e4d27f3c4f4fbe" candidate_uuid: "825d87e809214dbcab7e3e5e2455d8f0" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "50a730daa7ac44c896d7621ce906e828"
I20260923 22:13:59.928171 13055 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "363acddfee544ea4b5e4d27f3c4f4fbe" candidate_uuid: "825d87e809214dbcab7e3e5e2455d8f0" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "483ba5db7c5545d396351b7e2d699667"
I20260923 22:13:59.928212 12933 raft_consensus.cc:3037] T 363acddfee544ea4b5e4d27f3c4f4fbe P 50a730daa7ac44c896d7621ce906e828 [term 0 FOLLOWER]: Advancing to term 1
I20260923 22:13:59.928233 13055 raft_consensus.cc:3037] T 363acddfee544ea4b5e4d27f3c4f4fbe P 483ba5db7c5545d396351b7e2d699667 [term 0 FOLLOWER]: Advancing to term 1
I20260923 22:13:59.929106 13055 raft_consensus.cc:2454] T 363acddfee544ea4b5e4d27f3c4f4fbe P 483ba5db7c5545d396351b7e2d699667 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 825d87e809214dbcab7e3e5e2455d8f0 in term 1.
I20260923 22:13:59.929121 12933 raft_consensus.cc:2454] T 363acddfee544ea4b5e4d27f3c4f4fbe P 50a730daa7ac44c896d7621ce906e828 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 825d87e809214dbcab7e3e5e2455d8f0 in term 1.
I20260923 22:13:59.929327 13138 leader_election.cc:302] T 363acddfee544ea4b5e4d27f3c4f4fbe P 825d87e809214dbcab7e3e5e2455d8f0 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 483ba5db7c5545d396351b7e2d699667, 825d87e809214dbcab7e3e5e2455d8f0; no voters:
I20260923 22:13:59.929420 13276 raft_consensus.cc:2781] T 363acddfee544ea4b5e4d27f3c4f4fbe P 825d87e809214dbcab7e3e5e2455d8f0 [term 1 FOLLOWER]: Leader election won for term 1
I20260923 22:13:59.929559 13276 raft_consensus.cc:686] T 363acddfee544ea4b5e4d27f3c4f4fbe P 825d87e809214dbcab7e3e5e2455d8f0 [term 1 LEADER]: Becoming Leader. State: Replica: 825d87e809214dbcab7e3e5e2455d8f0, State: Running, Role: LEADER
I20260923 22:13:59.929670 13276 consensus_queue.cc:237] T 363acddfee544ea4b5e4d27f3c4f4fbe P 825d87e809214dbcab7e3e5e2455d8f0 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "825d87e809214dbcab7e3e5e2455d8f0" member_type: VOTER last_known_addr { host: "127.1.135.195" port: 38139 } } peers { permanent_uuid: "50a730daa7ac44c896d7621ce906e828" member_type: VOTER last_known_addr { host: "127.1.135.193" port: 33909 } } peers { permanent_uuid: "483ba5db7c5545d396351b7e2d699667" member_type: VOTER last_known_addr { host: "127.1.135.194" port: 33083 } }
I20260923 22:13:59.930449 12766 catalog_manager.cc:5495] T 363acddfee544ea4b5e4d27f3c4f4fbe P 825d87e809214dbcab7e3e5e2455d8f0 reported cstate change: term changed from 0 to 1, leader changed from <none> to 825d87e809214dbcab7e3e5e2455d8f0 (127.1.135.195). New cstate: current_term: 1 leader_uuid: "825d87e809214dbcab7e3e5e2455d8f0" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "825d87e809214dbcab7e3e5e2455d8f0" member_type: VOTER last_known_addr { host: "127.1.135.195" port: 38139 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "50a730daa7ac44c896d7621ce906e828" member_type: VOTER last_known_addr { host: "127.1.135.193" port: 33909 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "483ba5db7c5545d396351b7e2d699667" member_type: VOTER last_known_addr { host: "127.1.135.194" port: 33083 } health_report { overall_health: UNKNOWN } } }
I20260923 22:13:59.942408 13265 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 22:13:59.940183 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54612 (local address 127.1.135.195:38139)
0923 22:13:59.940231 (+ 48us) server_negotiation.cc:206] Beginning negotiation
0923 22:13:59.940233 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0923 22:13:59.940250 (+ 17us) server_negotiation.cc:407] Connection header received
0923 22:13:59.940280 (+ 30us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 22:13:59.940280 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 22:13:59.940299 (+ 19us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 22:13:59.940360 (+ 61us) server_negotiation.cc:226] Negotiated authn=TOKEN
0923 22:13:59.940493 (+ 133us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 22:13:59.940925 (+ 432us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 22:13:59.941656 (+ 731us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 22:13:59.941852 (+ 196us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 22:13:59.941923 (+ 71us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 22:13:59.942107 (+ 184us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0923 22:13:59.942205 (+ 98us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0923 22:13:59.942284 (+ 79us) server_negotiation.cc:1035] Waiting for connection context
0923 22:13:59.942321 (+ 37us) server_negotiation.cc:299] Negotiation successful
0923 22:13:59.942339 (+ 18us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":30}
I20260923 22:13:59.944962 12933 raft_consensus.cc:1261] T 363acddfee544ea4b5e4d27f3c4f4fbe P 50a730daa7ac44c896d7621ce906e828 [term 1 FOLLOWER]: Refusing update from remote peer 825d87e809214dbcab7e3e5e2455d8f0: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260923 22:13:59.945303 13280 consensus_queue.cc:1035] T 363acddfee544ea4b5e4d27f3c4f4fbe P 825d87e809214dbcab7e3e5e2455d8f0 [LEADER]: Connected to new peer: Peer: permanent_uuid: "50a730daa7ac44c896d7621ce906e828" member_type: VOTER last_known_addr { host: "127.1.135.193" port: 33909 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260923 22:13:59.948163 13283 mvcc.cc:205] Tried to move back new op lower bound from 7332665917211836416 to 7332665917152522240. Current Snapshot: MvccSnapshot[applied={T|T < 7332665917211836416}]
I20260923 22:13:59.949116 13284 mvcc.cc:205] Tried to move back new op lower bound from 7332665917211836416 to 7332665917152522240. Current Snapshot: MvccSnapshot[applied={T|T < 7332665917211836416}]
I20260923 22:13:59.951431 12766 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:40200:
table { table_name: "test-table" } modify_external_catalogs: true
I20260923 22:13:59.951535 12766 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:40200:
table { table_name: "test-table" } modify_external_catalogs: true
I20260923 22:13:59.952373 13055 raft_consensus.cc:1261] T 363acddfee544ea4b5e4d27f3c4f4fbe P 483ba5db7c5545d396351b7e2d699667 [term 1 FOLLOWER]: Refusing update from remote peer 825d87e809214dbcab7e3e5e2455d8f0: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260923 22:13:59.952593 12766 catalog_manager.cc:5782] T 00000000000000000000000000000000 P a48af14123054627ae158c6e22d475c5: Sending DeleteTablet for 3 replicas of tablet 363acddfee544ea4b5e4d27f3c4f4fbe
I20260923 22:13:59.952656 13280 consensus_queue.cc:1035] T 363acddfee544ea4b5e4d27f3c4f4fbe P 825d87e809214dbcab7e3e5e2455d8f0 [LEADER]: Connected to new peer: Peer: permanent_uuid: "483ba5db7c5545d396351b7e2d699667" member_type: VOTER last_known_addr { host: "127.1.135.194" port: 33083 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260923 22:13:59.952908 13185 tablet_service.cc:1514] Processing DeleteTablet for tablet 363acddfee544ea4b5e4d27f3c4f4fbe with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-23 22:13:59 UTC) from {username='kudu', principal='kudu/127.1.135.254@KRBTEST.COM'} at 127.0.0.1:54604
I20260923 22:13:59.953001 12913 tablet_service.cc:1514] Processing DeleteTablet for tablet 363acddfee544ea4b5e4d27f3c4f4fbe with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-23 22:13:59 UTC) from {username='kudu', principal='kudu/127.1.135.254@KRBTEST.COM'} at 127.0.0.1:57506
I20260923 22:13:59.953318 13033 tablet_service.cc:1514] Processing DeleteTablet for tablet 363acddfee544ea4b5e4d27f3c4f4fbe with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-23 22:13:59 UTC) from {username='kudu', principal='kudu/127.1.135.254@KRBTEST.COM'} at 127.0.0.1:35340
I20260923 22:13:59.953495 13291 tablet_replica.cc:330] T 363acddfee544ea4b5e4d27f3c4f4fbe P 483ba5db7c5545d396351b7e2d699667: stopping tablet replica
I20260923 22:13:59.953611 13291 raft_consensus.cc:2229] T 363acddfee544ea4b5e4d27f3c4f4fbe P 483ba5db7c5545d396351b7e2d699667 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260923 22:13:59.953698 13291 raft_consensus.cc:2258] T 363acddfee544ea4b5e4d27f3c4f4fbe P 483ba5db7c5545d396351b7e2d699667 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260923 22:13:59.953922 1567 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskRiQe_a/build/release/bin/kudu with pid 12848
W20260923 22:13:59.955421 13138 consensus_peers.cc:487] T 363acddfee544ea4b5e4d27f3c4f4fbe P 825d87e809214dbcab7e3e5e2455d8f0 -> Peer 483ba5db7c5545d396351b7e2d699667 (127.1.135.194:33083): Couldn't send request to peer 483ba5db7c5545d396351b7e2d699667. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: STOPPING. This is attempt 1: this message will repeat every 5th retry.
I20260923 22:13:59.961406 13291 ts_tablet_manager.cc:1905] T 363acddfee544ea4b5e4d27f3c4f4fbe P 483ba5db7c5545d396351b7e2d699667: Deleting tablet data with delete state TABLET_DATA_DELETED
W20260923 22:13:59.963254 12754 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.1.135.193:33909 (error 108)
I20260923 22:13:59.964164 13264 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 22:13:59.964014 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.135.193:33909 (local address 127.0.0.1:57516)
0923 22:13:59.964077 (+ 63us) negotiation.cc:107] Waiting for socket to connect
0923 22:13:59.964113 (+ 36us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.135.193:33909: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":22}
I20260923 22:13:59.964169 1567 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskRiQe_a/build/release/bin/kudu with pid 12985
I20260923 22:13:59.964772 13289 tablet_replica.cc:330] T 363acddfee544ea4b5e4d27f3c4f4fbe P 825d87e809214dbcab7e3e5e2455d8f0: stopping tablet replica
I20260923 22:13:59.964882 13289 raft_consensus.cc:2229] T 363acddfee544ea4b5e4d27f3c4f4fbe P 825d87e809214dbcab7e3e5e2455d8f0 [term 1 LEADER]: Raft consensus shutting down.
I20260923 22:13:59.965003 13289 raft_consensus.cc:2258] T 363acddfee544ea4b5e4d27f3c4f4fbe P 825d87e809214dbcab7e3e5e2455d8f0 [term 1 FOLLOWER]: Raft consensus is shut down!
W20260923 22:13:59.965696 12754 catalog_manager.cc:4584] TS 50a730daa7ac44c896d7621ce906e828 (127.1.135.193:33909): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet 363acddfee544ea4b5e4d27f3c4f4fbe: Network error: Client connection negotiation failed: client connection to 127.1.135.193:33909: connect: Connection refused (error 111)
I20260923 22:13:59.966208 13289 ts_tablet_manager.cc:1905] T 363acddfee544ea4b5e4d27f3c4f4fbe P 825d87e809214dbcab7e3e5e2455d8f0: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260923 22:13:59.967804 13289 ts_tablet_manager.cc:1918] T 363acddfee544ea4b5e4d27f3c4f4fbe P 825d87e809214dbcab7e3e5e2455d8f0: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260923 22:13:59.967921 13289 log.cc:1196] T 363acddfee544ea4b5e4d27f3c4f4fbe P 825d87e809214dbcab7e3e5e2455d8f0: Deleting WAL directory at /tmp/dist-test-taskRiQe_a/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790201539513534-1567-0/minicluster-data/ts-2/wal/wals/363acddfee544ea4b5e4d27f3c4f4fbe
I20260923 22:13:59.968264 13289 ts_tablet_manager.cc:1939] T 363acddfee544ea4b5e4d27f3c4f4fbe P 825d87e809214dbcab7e3e5e2455d8f0: Deleting consensus metadata
I20260923 22:13:59.968699 12753 catalog_manager.cc:4841] TS 825d87e809214dbcab7e3e5e2455d8f0 (127.1.135.195:38139): tablet 363acddfee544ea4b5e4d27f3c4f4fbe (table test-table [id=478801440d8246b69098108bee99f2a1]) successfully deleted
W20260923 22:13:59.976344 13264 negotiation.cc:337] Failed RPC negotiation. Trace:
0923 22:13:59.971939 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.135.194:33083 (local address 127.0.0.1:35356)
0923 22:13:59.972001 (+ 62us) negotiation.cc:107] Waiting for socket to connect
0923 22:13:59.972013 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0923 22:13:59.972054 (+ 41us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0923 22:13:59.976300 (+ 4246us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.135.194:33083: BlockingRecv error: recv error from unknown peer: Transport endpoint is not connected (error 107)
Metrics: {"client-negotiator.queue_time_us":42}
I20260923 22:13:59.976871 1567 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskRiQe_a/build/release/bin/kudu with pid 13122
I20260923 22:13:59.984424 1567 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskRiQe_a/build/release/bin/kudu with pid 12735
I20260923 22:13:59.990509 1567 mini_ranger_kms.cc:60] Stopping Ranger KMS...
I20260923 22:14:00.334456 1567 mini_ranger_kms.cc:62] Stopped Ranger KMS
I20260923 22:14:00.334571 1567 mini_ranger.cc:66] Stopping Ranger...
I20260923 22:14:00.678586 1567 mini_ranger.cc:68] Stopped Ranger
2026-09-23 22:14:00.678 UTC [12321] LOG: received smart shutdown request
2026-09-23 22:14:00.682 UTC [12321] LOG: background worker "logical replication launcher" (PID 12330) exited with exit code 1
2026-09-23 22:14:00.682 UTC [12325] LOG: shutting down
2026-09-23 22:14:00.836 UTC [12321] LOG: database system is shut down
2026-09-23T22:14:00Z chronyd exiting
[ OK ] SecurityITest.TestEncryptionWithKMSIntegration (44532 ms)
[ RUN ] SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers
Loading random data
Initializing database '/tmp/dist-test-taskRiQe_a/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790201539513534-1567-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Sep 23 22:14:00 dist-test-slave-s2fj krb5kdc[13301](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Sep 23 22:14:00 dist-test-slave-s2fj krb5kdc[13301](info): set up 2 sockets
Sep 23 22:14:00 dist-test-slave-s2fj krb5kdc[13301](info): commencing operation
krb5kdc: starting...
W20260923 22:14:01.971887 1567 mini_kdc.cc:121] Time spent starting KDC: real 1.010s user 0.000s sys 0.005s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskRiQe_a/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790201539513534-1567-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskRiQe_a/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790201539513534-1567-0/krb5kdc/test-user.keytab.
Sep 23 22:14:02 dist-test-slave-s2fj krb5kdc[13301](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790201642, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-09-23T22:14:02Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-09-23T22:14:02Z Disabled control of system clock
WARNING: no policy specified for rangeradmin/127.1.135.212@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "rangeradmin/127.1.135.212@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal rangeradmin/127.1.135.212@KRBTEST.COM with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskRiQe_a/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790201539513534-1567-0/krb5kdc/rangeradmin_127.1.135.212@KRBTEST.COM.keytab.
Entry for principal rangeradmin/127.1.135.212@KRBTEST.COM with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskRiQe_a/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790201539513534-1567-0/krb5kdc/rangeradmin_127.1.135.212@KRBTEST.COM.keytab.
WARNING: no policy specified for rangerlookup/127.1.135.212@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "rangerlookup/127.1.135.212@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal rangerlookup/127.1.135.212@KRBTEST.COM with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskRiQe_a/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790201539513534-1567-0/krb5kdc/rangerlookup_127.1.135.212@KRBTEST.COM.keytab.
Entry for principal rangerlookup/127.1.135.212@KRBTEST.COM with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskRiQe_a/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790201539513534-1567-0/krb5kdc/rangerlookup_127.1.135.212@KRBTEST.COM.keytab.
WARNING: no policy specified for HTTP/127.1.135.212@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "HTTP/127.1.135.212@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal HTTP/127.1.135.212@KRBTEST.COM with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskRiQe_a/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790201539513534-1567-0/krb5kdc/HTTP_127.1.135.212@KRBTEST.COM.keytab.
Entry for principal HTTP/127.1.135.212@KRBTEST.COM with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskRiQe_a/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790201539513534-1567-0/krb5kdc/HTTP_127.1.135.212@KRBTEST.COM.keytab.
I20260923 22:14:02.122200 1567 mini_postgres.cc:62] Running initdb...
The files belonging to this database system will be owned by user "slave".
This user must also own the server process.
The database cluster will be initialized with locale "C".
The default database encoding has accordingly been set to "SQL_ASCII".
The default text search configuration will be set to "english".
Data page checksums are disabled.
creating directory /tmp/dist-test-taskRiQe_a/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790201539513534-1567-0/minicluster-data/postgres ... ok
creating subdirectories ... ok
selecting dynamic shared memory implementation ... posix
selecting default max_connections ... 100
selecting default shared_buffers ... 128MB
selecting default time zone ... Etc/UTC
creating configuration files ... ok
running bootstrap script ... ok
performing post-bootstrap initialization ... ok
syncing data to disk ... ok
initdb: warning: enabling "trust" authentication for local connections
You can change this by editing pg_hba.conf or using the option -A, or
--auth-local and --auth-host, the next time you run initdb.
Success. You can now start the database server using:
/tmp/dist-test-taskRiQe_a/build/release/bin/postgres/pg_ctl -D /tmp/dist-test-taskRiQe_a/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790201539513534-1567-0/minicluster-data/postgres -l logfile start
2026-09-23 22:14:04.457 UTC [13331] LOG: starting PostgreSQL 12.2 on x86_64-pc-linux-gnu, compiled by cc (GCC) 8.3.1 20190311 (Red Hat 8.3.1-3), 64-bit
2026-09-23 22:14:04.457 UTC [13331] LOG: listening on IPv4 address "127.1.135.212", port 59003
2026-09-23 22:14:04.460 UTC [13331] LOG: listening on Unix socket "/tmp/.s.PGSQL.59003"
2026-09-23 22:14:04.494 UTC [13334] LOG: database system was shut down at 2026-09-23 22:14:03 UTC
2026-09-23 22:14:04.498 UTC [13331] LOG: database system is ready to accept connections
I20260923 22:14:05.427831 1567 mini_postgres.cc:96] Postgres bound to 59003
2026-09-23 22:14:05.433 UTC [13342] FATAL: database "slave" does not exist
127.1.135.212:59003 - accepting connections
I20260923 22:14:05.434130 1567 mini_ranger.cc:161] Starting Ranger...
I20260923 22:14:05.448038 1567 mini_ranger.cc:84] Created miniranger Postgres user
I20260923 22:14:05.890961 1567 mini_ranger.cc:87] Created ranger Postgres database
I20260923 22:14:05.891038 1567 mini_ranger.cc:178] Starting Ranger out of /tmp/dist-test-taskRiQe_a/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790201539513534-1567-0/minicluster-data/ranger-admin
2026-09-23 22:14:06,173 [I] DB FLAVOR :POSTGRES
2026-09-23 22:14:06,174 [I] --------- Verifying Ranger DB connection ---------
2026-09-23 22:14:06,174 [I] Checking connection..
2026-09-23 22:14:06,174 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskRiQe_a/build/release/bin/postgresql.jar:/tmp/dist-test-taskRiQe_a/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.1.135.212:59003/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select 1;"
2026-09-23 22:14:06,412 [I] Checking connection passed.
2026-09-23 22:14:06,412 [I] --------- Verifying version history table ---------
2026-09-23 22:14:06,412 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskRiQe_a/build/release/bin/postgresql.jar:/tmp/dist-test-taskRiQe_a/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.1.135.212:59003/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_db_version_h') as temp;"
2026-09-23 22:14:06,651 [I] Table x_db_version_h does not exist in database ranger
2026-09-23 22:14:06,651 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskRiQe_a/build/release/bin/postgresql.jar:/tmp/dist-test-taskRiQe_a/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.1.135.212:59003/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_db_version_h') as temp;"
2026-09-23 22:14:06,904 [I] Table x_db_version_h does not exist in database ranger
2026-09-23 22:14:06,905 [I] Importing x_db_version_h table schema to database ranger from file: create_dbversion_catalog.sql
2026-09-23 22:14:06,905 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskRiQe_a/build/release/bin/postgresql.jar:/tmp/dist-test-taskRiQe_a/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.1.135.212:59003/ranger -u miniranger -p '********' -noheader -trim -c \; -input /tmp/dist-test-taskRiQe_a/thirdparty/src/ranger-2.1.0-admin/db/postgres/create_dbversion_catalog.sql
2026-09-23 22:14:07.131 UTC [13406] WARNING: there is no transaction in progress
2026-09-23 22:14:07,136 [I] create_dbversion_catalog.sql file imported successfully
2026-09-23 22:14:07,137 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskRiQe_a/build/release/bin/postgresql.jar:/tmp/dist-test-taskRiQe_a/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.1.135.212:59003/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_db_version_h') as temp;"
2026-09-23 22:14:07,389 [I] Table x_db_version_h already exists in database 'ranger'
2026-09-23 22:14:07,390 [I] --------- Importing Ranger Core DB Schema ---------
2026-09-23 22:14:07,390 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskRiQe_a/build/release/bin/postgresql.jar:/tmp/dist-test-taskRiQe_a/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.1.135.212:59003/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'CORE_DB_SCHEMA' and active = 'Y';"
2026-09-23 22:14:07,607 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskRiQe_a/build/release/bin/postgresql.jar:/tmp/dist-test-taskRiQe_a/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.1.135.212:59003/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'CORE_DB_SCHEMA' and active = 'N';"
2026-09-23 22:14:07,855 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskRiQe_a/build/release/bin/postgresql.jar:/tmp/dist-test-taskRiQe_a/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.1.135.212:59003/ranger -u miniranger -p '********' -noheader -trim -c \; -query "insert into x_db_version_h (version, inst_at, inst_by, updated_at, updated_by,active) values ('CORE_DB_SCHEMA', current_timestamp, 'Ranger 2.1.0', current_timestamp, 'dist-test-slave-s2fj.c.gcp-upstream.internal','N') ;"
2026-09-23 22:14:08,093 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskRiQe_a/build/release/bin/postgresql.jar:/tmp/dist-test-taskRiQe_a/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.1.135.212:59003/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_portal_user') as temp;"
2026-09-23 22:14:08,318 [I] Table x_portal_user does not exist in database ranger
2026-09-23 22:14:08,318 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskRiQe_a/build/release/bin/postgresql.jar:/tmp/dist-test-taskRiQe_a/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.1.135.212:59003/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_policy_ref_group') as temp;"
2026-09-23 22:14:08,565 [I] Table x_policy_ref_group does not exist in database ranger
2026-09-23 22:14:08,566 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskRiQe_a/build/release/bin/postgresql.jar:/tmp/dist-test-taskRiQe_a/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.1.135.212:59003/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'DB_PATCHES' and active = 'Y';"
2026-09-23 22:14:08,798 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskRiQe_a/build/release/bin/postgresql.jar:/tmp/dist-test-taskRiQe_a/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.1.135.212:59003/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'JAVA_PATCHES' and active = 'Y';"
2026-09-23 22:14:09,026 [I] Importing DB schema to database ranger from file: ranger_core_db_postgres.sql
2026-09-23 22:14:09,026 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskRiQe_a/build/release/bin/postgresql.jar:/tmp/dist-test-taskRiQe_a/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.1.135.212:59003/ranger -u miniranger -p '********' -noheader -trim -c \; -input /tmp/dist-test-taskRiQe_a/thirdparty/src/ranger-2.1.0-admin/db/postgres/optimized/current/ranger_core_db_postgres.sql
2026-09-23 22:14:09.926 UTC [13514] WARNING: there is no transaction in progress
2026-09-23 22:14:09.941 UTC [13514] WARNING: there is no transaction in progress
2026-09-23 22:14:09.956 UTC [13514] WARNING: there is no transaction in progress
2026-09-23 22:14:09.970 UTC [13514] WARNING: there is no transaction in progress
2026-09-23 22:14:09.984 UTC [13514] WARNING: there is no transaction in progress
2026-09-23 22:14:10.001 UTC [13514] WARNING: there is no transaction in progress
2026-09-23 22:14:10.051 UTC [13514] WARNING: there is no transaction in progress
2026-09-23 22:14:10.058 UTC [13514] WARNING: there is no transaction in progress
2026-09-23 22:14:10.071 UTC [13514] WARNING: there is no transaction in progress
2026-09-23 22:14:10.080 UTC [13514] WARNING: there is no transaction in progress
2026-09-23 22:14:10.090 UTC [13514] WARNING: there is no transaction in progress
2026-09-23 22:14:10.103 UTC [13514] WARNING: there is no transaction in progress
2026-09-23 22:14:10.112 UTC [13514] WARNING: there is no transaction in progress
2026-09-23 22:14:10.118 UTC [13514] WARNING: there is no transaction in progress
2026-09-23 22:14:10.124 UTC [13514] WARNING: there is no transaction in progress
2026-09-23 22:14:10.129 UTC [13514] WARNING: there is no transaction in progress
2026-09-23 22:14:10,731 [I] ranger_core_db_postgres.sql file imported successfully
2026-09-23 22:14:10,731 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskRiQe_a/build/release/bin/postgresql.jar:/tmp/dist-test-taskRiQe_a/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.1.135.212:59003/ranger -u miniranger -p '********' -noheader -trim -c \; -query "update x_db_version_h set inst_by='Ranger 2.1.0' where active='Y' and updated_by='localhost';"
2026-09-23 22:14:10,971 [I] Patches status entries updated from base ranger version to current installed ranger version:Ranger 2.1.0
2026-09-23 22:14:10,971 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskRiQe_a/build/release/bin/postgresql.jar:/tmp/dist-test-taskRiQe_a/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.1.135.212:59003/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_portal_user') as temp;"
2026-09-23 22:14:11,219 [I] Table x_portal_user already exists in database 'ranger'
2026-09-23 22:14:11,220 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskRiQe_a/build/release/bin/postgresql.jar:/tmp/dist-test-taskRiQe_a/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.1.135.212:59003/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_policy_ref_group') as temp;"
2026-09-23 22:14:11,455 [I] Table x_policy_ref_group already exists in database 'ranger'
2026-09-23 22:14:11,456 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskRiQe_a/build/release/bin/postgresql.jar:/tmp/dist-test-taskRiQe_a/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.1.135.212:59003/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'DB_PATCHES' and active = 'Y';"
2026-09-23 22:14:11,678 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskRiQe_a/build/release/bin/postgresql.jar:/tmp/dist-test-taskRiQe_a/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.1.135.212:59003/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'JAVA_PATCHES' and active = 'Y';"
2026-09-23 22:14:11,891 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskRiQe_a/build/release/bin/postgresql.jar:/tmp/dist-test-taskRiQe_a/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.1.135.212:59003/ranger -u miniranger -p '********' -noheader -trim -c \; -query "update x_db_version_h set active='Y' where version='CORE_DB_SCHEMA' and active='N' and updated_by='dist-test-slave-s2fj.c.gcp-upstream.internal';"
2026-09-23 22:14:12,098 [I] CORE_DB_SCHEMA import status has been updated
2026-09-23 22:14:12,098 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskRiQe_a/build/release/bin/postgresql.jar:/tmp/dist-test-taskRiQe_a/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.1.135.212:59003/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'DB_PATCHES' and inst_by = 'Ranger 2.1.0' and active = 'Y';"
2026-09-23 22:14:12,346 [I] DB_PATCHES have already been applied
I20260923 22:14:12.352273 1567 mini_ranger.cc:191] Using Ranger class path: /tmp/dist-test-taskRiQe_a/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790201539513534-1567-0/minicluster-data/ranger-admin:/tmp/dist-test-taskRiQe_a/thirdparty/src/ranger-2.1.0-admin/ews/lib/*:/usr/lib/jvm/java-1.8.0-openjdk-amd64/lib/*:/tmp/dist-test-taskRiQe_a/thirdparty/src/hadoop-3.2.0/*:/tmp/dist-test-taskRiQe_a/build/release/bin/postgresql.jar:/tmp/dist-test-taskRiQe_a/thirdparty/src/ranger-2.1.0-admin/ews/webapp
I20260923 22:14:12.352355 1567 mini_ranger.cc:193] Using host: 127.1.135.212
I20260923 22:14:12.355293 1567 mini_ranger.cc:227] Ranger admin URL: http://127.1.135.212:44579
Sep 23, 2026 10:14:12 PM org.apache.ranger.server.tomcat.EmbeddedServer getKeyManagers
WARNING: Config 'ranger.keystore.file' or 'ranger.service.https.attrib.keystore.file' is not found or contains blank value
Sep 23, 2026 10:14:12 PM org.apache.ranger.server.tomcat.EmbeddedServer getTrustManagers
WARNING: Config 'ranger.truststore.file' is not found or contains blank value!
Sep 23, 2026 10:14:12 PM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Deriving webapp folder from catalina.base property. folder=/tmp/dist-test-taskRiQe_a/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790201539513534-1567-0/minicluster-data/ranger-admin/ews/webapp
Sep 23, 2026 10:14:12 PM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Webapp file =/tmp/dist-test-taskRiQe_a/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790201539513534-1567-0/minicluster-data/ranger-admin/ews/webapp, webAppName = /
Sep 23, 2026 10:14:12 PM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Adding webapp [/] = path [/tmp/dist-test-taskRiQe_a/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790201539513534-1567-0/minicluster-data/ranger-admin/ews/webapp] .....
Sep 23, 2026 10:14:13 PM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Finished init of webapp [/] = path [/tmp/dist-test-taskRiQe_a/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790201539513534-1567-0/minicluster-data/ranger-admin/ews/webapp].
Sep 23, 2026 10:14:13 PM org.apache.ranger.server.tomcat.EmbeddedServer startServer
INFO: Server Name : miniranger
I20260923 22:14:13.438908 1567 mini_ranger.cc:160] Time spent starting Ranger: real 8.005s user 0.000s sys 0.008s
Sep 23, 2026 10:14:32 PM com.sun.jersey.api.core.PackagesResourceConfig init
INFO: Scanning for root resource and provider classes in the packages:
org.apache.ranger.rest
org.apache.ranger.common
xa.rest
Sep 23, 2026 10:14:32 PM com.sun.jersey.api.core.ScanningResourceConfig logClasses
INFO: Root resource classes found:
class org.apache.ranger.rest.XAuditREST
class org.apache.ranger.rest.SecurityZoneREST
class org.apache.ranger.rest.PublicAPIsv2
class org.apache.ranger.rest.XKeyREST
class org.apache.ranger.rest.RoleREST
class org.apache.ranger.rest.ServiceREST
class org.apache.ranger.rest.AssetREST
class org.apache.ranger.rest.XUserREST
class org.apache.ranger.rest.TagREST
class org.apache.ranger.rest.MetricsREST
class org.apache.ranger.rest.PublicAPIs
class org.apache.ranger.rest.UserREST
Sep 23, 2026 10:14:32 PM com.sun.jersey.api.core.ScanningResourceConfig logClasses
INFO: Provider classes found:
class org.apache.ranger.common.RangerJAXBContextResolver
Sep 23, 2026 10:14:32 PM com.sun.jersey.spi.spring.container.servlet.SpringServlet getContext
INFO: Using default applicationContext
Sep 23, 2026 10:14:32 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, assetREST, of type org.apache.ranger.rest.AssetREST as a root resource class
Sep 23, 2026 10:14:32 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, metricsREST, of type org.apache.ranger.rest.MetricsREST as a root resource class
Sep 23, 2026 10:14:32 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, publicAPIs, of type org.apache.ranger.rest.PublicAPIs as a root resource class
Sep 23, 2026 10:14:32 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, publicAPIsv2, of type org.apache.ranger.rest.PublicAPIsv2 as a root resource class
Sep 23, 2026 10:14:32 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, roleREST, of type org.apache.ranger.rest.RoleREST as a root resource class
Sep 23, 2026 10:14:32 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, securityZoneREST, of type org.apache.ranger.rest.SecurityZoneREST as a root resource class
Sep 23, 2026 10:14:32 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, serviceREST, of type org.apache.ranger.rest.ServiceREST as a root resource class
Sep 23, 2026 10:14:32 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, tagREST, of type org.apache.ranger.rest.TagREST as a root resource class
Sep 23, 2026 10:14:32 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, userREST, of type org.apache.ranger.rest.UserREST as a root resource class
Sep 23, 2026 10:14:32 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, XAuditREST, of type org.apache.ranger.rest.XAuditREST as a root resource class
Sep 23, 2026 10:14:32 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, XKeyREST, of type org.apache.ranger.rest.XKeyREST as a root resource class
Sep 23, 2026 10:14:32 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, XUserREST, of type org.apache.ranger.rest.XUserREST as a root resource class
Sep 23, 2026 10:14:32 PM com.sun.jersey.server.impl.application.WebApplicationImpl _initiate
INFO: Initiating Jersey application, version 'Jersey: 1.19.3 10/24/2016 03:58 PM'
Sep 23, 2026 10:14:33 PM com.sun.jersey.spi.inject.Errors processErrorMessages
WARNING: The following warnings have been detected with resource and/or provider classes:
WARNING: A HTTP GET method, public void org.apache.ranger.rest.ServiceREST.getPoliciesInExcel(javax.servlet.http.HttpServletRequest,javax.servlet.http.HttpServletResponse), MUST return a non-void type.
WARNING: A HTTP GET method, public void org.apache.ranger.rest.ServiceREST.getPoliciesInJson(javax.servlet.http.HttpServletRequest,javax.servlet.http.HttpServletResponse,java.lang.Boolean), MUST return a non-void type.
WARNING: A HTTP GET method, public void org.apache.ranger.rest.ServiceREST.getPoliciesInCsv(javax.servlet.http.HttpServletRequest,javax.servlet.http.HttpServletResponse) throws java.io.IOException, MUST return a non-void type.
I20260923 22:14:34.303570 1567 mini_ranger.cc:261] Created Kudu service
WARNING: no policy specified for rangerkms/127.1.135.212@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "rangerkms/127.1.135.212@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal rangerkms/127.1.135.212@KRBTEST.COM with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskRiQe_a/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790201539513534-1567-0/krb5kdc/rangerkms_127.1.135.212@KRBTEST.COM.keytab.
Entry for principal rangerkms/127.1.135.212@KRBTEST.COM with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskRiQe_a/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790201539513534-1567-0/krb5kdc/rangerkms_127.1.135.212@KRBTEST.COM.keytab.
WARNING: no policy specified for HTTP/127.1.135.212@KRBTEST.COM; defaulting to no policy
add_principal: Principal or policy already exists while creating "HTTP/127.1.135.212@KRBTEST.COM".
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal HTTP/127.1.135.212@KRBTEST.COM with kvno 3, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskRiQe_a/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790201539513534-1567-0/krb5kdc/HTTP_127.1.135.212@KRBTEST.COM.keytab.
Entry for principal HTTP/127.1.135.212@KRBTEST.COM with kvno 3, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskRiQe_a/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790201539513534-1567-0/krb5kdc/HTTP_127.1.135.212@KRBTEST.COM.keytab.
WARNING: no policy specified for keyadmin@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "keyadmin@KRBTEST.COM" created.
Sep 23 22:14:34 dist-test-slave-s2fj krb5kdc[13301](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790201674, etypes {rep=17 tkt=17 ses=17}, keyadmin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for keyadmin@KRBTEST.COM:
I20260923 22:14:34.382886 1567 mini_ranger_kms.cc:208] Starting Ranger KMS...
I20260923 22:14:34.391592 1567 mini_ranger_kms.cc:78] Created minirangerkms Postgres user
I20260923 22:14:35.169852 1567 mini_ranger_kms.cc:81] Created rangerkms Postgres database
I20260923 22:14:35.169953 1567 mini_ranger_kms.cc:226] Starting Ranger KMS out of /tmp/dist-test-taskRiQe_a/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790201539513534-1567-0/minicluster-data/ranger-kms
I20260923 22:14:35.169975 1567 mini_ranger_kms.cc:227] Using postgres at 127.1.135.212:59003
2026-09-23 22:14:35,337 [I] DB FLAVOR :POSTGRES
2026-09-23 22:14:35,338 [I] --------- Verifying Ranger DB connection ---------
2026-09-23 22:14:35,338 [I] Checking connection
2026-09-23 22:14:35,338 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskRiQe_a/build/release/bin/postgresql.jar:/tmp/dist-test-taskRiQe_a/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790201539513534-1567-0/minicluster-data/ranger-kms/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.1.135.212:59003/rangerkms -u rangerkms -p '********' -noheader -trim -c \; -query "SELECT 1;"
2026-09-23 22:14:35,579 [I] connection success
2026-09-23 22:14:35,579 [I] --------- Verifying Ranger DB tables ---------
2026-09-23 22:14:35,579 [I] Verifying table ranger_masterkey in database rangerkms
2026-09-23 22:14:35,579 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskRiQe_a/build/release/bin/postgresql.jar:/tmp/dist-test-taskRiQe_a/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790201539513534-1567-0/minicluster-data/ranger-kms/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.1.135.212:59003/rangerkms -u rangerkms -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='rangerkms' and table_name = 'ranger_masterkey') as temp;"
2026-09-23 22:14:35,811 [I] Table ranger_masterkey does not exist in database rangerkms
2026-09-23 22:14:35,812 [I] --------- Importing Ranger Core DB Schema ---------
2026-09-23 22:14:35,812 [I] Importing db schema to database rangerkms from file: kms_core_db_postgres.sql
2026-09-23 22:14:35,812 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskRiQe_a/build/release/bin/postgresql.jar:/tmp/dist-test-taskRiQe_a/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790201539513534-1567-0/minicluster-data/ranger-kms/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.1.135.212:59003/rangerkms -u rangerkms -p '********' -noheader -trim -c \; -input /tmp/dist-test-taskRiQe_a/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790201539513534-1567-0/minicluster-data/ranger-kms/db/postgres/kms_core_db_postgres.sql
2026-09-23 22:14:36,067 [I] kms_core_db_postgres.sql DB schema imported successfully
I20260923 22:14:36.502108 1567 mini_ranger_kms.cc:310] Created kms service
I20260923 22:14:36.673955 1567 mini_ranger_kms.cc:326] Created kudu user
I20260923 22:14:36.716440 1567 mini_ranger_kms.cc:343] Created rangerkms user
Sep 23 22:14:36 dist-test-slave-s2fj krb5kdc[13301](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790201674, etypes {rep=17 tkt=17 ses=17}, keyadmin@KRBTEST.COM for HTTP/127.1.135.212@KRBTEST.COM
I20260923 22:14:37.079726 1567 mini_ranger_kms.cc:384] Added ranger policy
I20260923 22:14:37.079939 1567 mini_ranger_kms.cc:238] Using RangerKMS classpath: /tmp/dist-test-taskRiQe_a/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790201539513534-1567-0/minicluster-data/ranger-kms:/tmp/dist-test-taskRiQe_a/build/release/bin/postgresql.jar:/tmp/dist-test-taskRiQe_a/thirdparty/src/ranger-2.1.0-kms/ews/webapp/WEB-INF/classes/lib/*:/tmp/dist-test-taskRiQe_a/thirdparty/src/ranger-2.1.0-kms/ews/webapp/lib/*:/usr/lib/jvm/java-1.8.0-openjdk-amd64/lib/*
I20260923 22:14:37.079959 1567 mini_ranger_kms.cc:240] Using host: 127.1.135.212
I20260923 22:14:37.083330 1567 mini_ranger_kms.cc:276] Ranger KMS PID: 13694
I20260923 22:14:37.083408 1567 mini_ranger_kms.cc:277] Ranger KMS URL: http://127.1.135.212:41391
Sep 23, 2026 10:14:38 PM org.apache.ranger.server.tomcat.EmbeddedServer getKeyManagers
WARNING: Config 'ranger.keystore.file' or 'ranger.service.https.attrib.keystore.file' is not found or contains blank value
Sep 23, 2026 10:14:38 PM org.apache.ranger.server.tomcat.EmbeddedServer getTrustManagers
WARNING: Config 'ranger.truststore.file' is not found or contains blank value!
Sep 23, 2026 10:14:38 PM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Webapp file =/tmp/dist-test-taskRiQe_a/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790201539513534-1567-0/minicluster-data/ranger-kms/ews/webapp, webAppName = /kms
Sep 23, 2026 10:14:38 PM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Adding webapp [/kms] = path [/tmp/dist-test-taskRiQe_a/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790201539513534-1567-0/minicluster-data/ranger-kms/ews/webapp] .....
Sep 23, 2026 10:14:38 PM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Finished init of webapp [/kms] = path [/tmp/dist-test-taskRiQe_a/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790201539513534-1567-0/minicluster-data/ranger-kms/ews/webapp].
Sep 23, 2026 10:14:38 PM org.apache.ranger.server.tomcat.EmbeddedServer startServer
INFO: Server Name : minirangerkms
I20260923 22:14:39.298894 1567 mini_ranger_kms.cc:207] Time spent starting Ranger KMS: real 4.916s user 0.000s sys 0.153s
I20260923 22:14:39.298990 1567 mini_ranger_kms.cc:397] {"name":"kuduclusterkey","cipher":"AES/CTR/NoPadding","length":128,"description":"kuduclusterkey"}
I20260923 22:14:39.299033 1567 mini_ranger_kms.cc:401] 127.1.135.212:41391/kms/v1/keys
Sep 23 22:14:39 dist-test-slave-s2fj krb5kdc[13301](info): AS_REQ (1 etypes {17}) 127.0.0.1: ISSUE: authtime 1790201679, etypes {rep=17 tkt=17 ses=17}, rangerkms/127.1.135.212@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Sep 23 22:14:39 dist-test-slave-s2fj krb5kdc[13301](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790201679, etypes {rep=17 tkt=17 ses=17}, rangerkms/127.1.135.212@KRBTEST.COM for HTTP/127.1.135.212@KRBTEST.COM
Sep 23 22:14:40 dist-test-slave-s2fj krb5kdc[13301](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790201679, etypes {rep=17 tkt=17 ses=17}, rangerkms/127.1.135.212@KRBTEST.COM for HTTP/127.1.135.212@KRBTEST.COM
Sep 23 22:14:42 dist-test-slave-s2fj krb5kdc[13301](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790201682, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
WARNING: no policy specified for kudu/127.1.135.254@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.1.135.254@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.1.135.254 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskRiQe_a/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790201539513534-1567-0/krb5kdc/kudu_127.1.135.254.keytab.
Entry for principal kudu/127.1.135.254 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskRiQe_a/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790201539513534-1567-0/krb5kdc/kudu_127.1.135.254.keytab.
I20260923 22:14:42.337133 1567 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskRiQe_a/build/release/bin/kudu
/tmp/dist-test-taskRiQe_a/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskRiQe_a/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790201539513534-1567-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskRiQe_a/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790201539513534-1567-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskRiQe_a/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790201539513534-1567-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskRiQe_a/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790201539513534-1567-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.1.135.254:37257
--webserver_interface=127.1.135.254
--webserver_port=0
--keytab_file=/tmp/dist-test-taskRiQe_a/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790201539513534-1567-0/krb5kdc/kudu_127.1.135.254.keytab
--principal=kudu/127.1.135.254
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.1.135.212:45915
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.1.135.254:37257
--ranger_config_path=/tmp/dist-test-taskRiQe_a/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790201539513534-1567-0/minicluster-data/ranger-client
--trusted_user_acl=test-admin
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.1.135.212:41391/kms
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskRiQe_a/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790201539513534-1567-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskRiQe_a/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790201539513534-1567-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskRiQe_a/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790201539513534-1567-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260923 22:14:42.425132 13745 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260923 22:14:42.425316 13745 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260923 22:14:42.425336 13745 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260923 22:14:42.426472 13745 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260923 22:14:42.426512 13745 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260923 22:14:42.426527 13745 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260923 22:14:42.426538 13745 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260923 22:14:42.426587 13745 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260923 22:14:42.428057 13745 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.1.135.212:45915
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-taskRiQe_a/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790201539513534-1567-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskRiQe_a/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790201539513534-1567-0/minicluster-data/master-0/wal
--ranger_kms_url=127.1.135.212:41391/kms
--trusted_user_acl=<redacted>
--ipki_ca_key_size=768
--master_addresses=127.1.135.254:37257
--ranger_config_path=/tmp/dist-test-taskRiQe_a/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790201539513534-1567-0/minicluster-data/ranger-client
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskRiQe_a/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790201539513534-1567-0/krb5kdc/kudu_127.1.135.254.keytab
--principal=kudu/127.1.135.254
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.1.135.254:37257
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskRiQe_a/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790201539513534-1567-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.1.135.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.13745
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskRiQe_a/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790201539513534-1567-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 23 Sep 2026 20:53:37 UTC on ccycloud.quasar-rkviyx.root.comops.site
build id 4668083
I20260923 22:14:42.428236 13745 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260923 22:14:42.428459 13745 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260923 22:14:42.433004 13751 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260923 22:14:42.433004 13750 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260923 22:14:42.433038 13753 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260923 22:14:42.433207 13745 server_base.cc:1034] running on GCE node
I20260923 22:14:42.433410 13745 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260923 22:14:42.433653 13745 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260923 22:14:42.434832 13745 hybrid_clock.cc:648] HybridClock initialized: now 1790201682434816 us; error 38 us; skew 500 ppm
Sep 23 22:14:42 dist-test-slave-s2fj krb5kdc[13301](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790201682, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.135.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260923 22:14:42.444438 13745 init.cc:377] Logged in from keytab as kudu/127.1.135.254@KRBTEST.COM (short username kudu)
I20260923 22:14:42.444816 13745 webserver.cc:460] Webserver started at http://127.1.135.254:41145/ using document root <none> and password file <none>
I20260923 22:14:42.444983 13745 fs_manager.cc:359] Metadata directory not provided
I20260923 22:14:42.445016 13745 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260923 22:14:42.445101 13745 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
Sep 23 22:14:42 dist-test-slave-s2fj krb5kdc[13301](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790201682, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.135.254@KRBTEST.COM for HTTP/127.1.135.212@KRBTEST.COM
I20260923 22:14:42.494611 13745 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskRiQe_a/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790201539513534-1567-0/minicluster-data/master-0/data/instance:
uuid: "d7b9fc6eb32b4fb5a1e6920bf231523d"
format_stamp: "Formatted at 2026-09-23 22:14:42 on dist-test-slave-s2fj"
server_key: "910f35218cce50034486a37a864f6ba0"
server_key_iv: "1cae74aa93f84b27277249f9c20c8066"
server_key_version: "kuduclusterkey@0"
I20260923 22:14:42.495041 13745 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskRiQe_a/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790201539513534-1567-0/minicluster-data/master-0/wal/instance:
uuid: "d7b9fc6eb32b4fb5a1e6920bf231523d"
format_stamp: "Formatted at 2026-09-23 22:14:42 on dist-test-slave-s2fj"
server_key: "910f35218cce50034486a37a864f6ba0"
server_key_iv: "1cae74aa93f84b27277249f9c20c8066"
server_key_version: "kuduclusterkey@0"
I20260923 22:14:42.496771 13745 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.001s sys 0.000s
I20260923 22:14:42.510156 13760 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260923 22:14:42.510327 13745 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.000s
I20260923 22:14:42.510408 13745 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskRiQe_a/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790201539513534-1567-0/minicluster-data/master-0/data,/tmp/dist-test-taskRiQe_a/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790201539513534-1567-0/minicluster-data/master-0/wal
uuid: "d7b9fc6eb32b4fb5a1e6920bf231523d"
format_stamp: "Formatted at 2026-09-23 22:14:42 on dist-test-slave-s2fj"
server_key: "910f35218cce50034486a37a864f6ba0"
server_key_iv: "1cae74aa93f84b27277249f9c20c8066"
server_key_version: "kuduclusterkey@0"
I20260923 22:14:42.510490 13745 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskRiQe_a/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790201539513534-1567-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskRiQe_a/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790201539513534-1567-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskRiQe_a/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790201539513534-1567-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260923 22:14:42.534018 13745 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260923 22:14:42.536065 13745 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260923 22:14:42.536252 13745 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260923 22:14:42.541198 13745 rpc_server.cc:307] RPC server started. Bound to: 127.1.135.254:37257
I20260923 22:14:42.541218 13812 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.1.135.254:37257 every 8 connection(s)
I20260923 22:14:42.541698 13745 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskRiQe_a/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790201539513534-1567-0/minicluster-data/master-0/data/info.pb
I20260923 22:14:42.542388 13813 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260923 22:14:42.542645 1567 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskRiQe_a/build/release/bin/kudu as pid 13745
I20260923 22:14:42.542760 1567 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskRiQe_a/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790201539513534-1567-0/minicluster-data/master-0/wal/instance
I20260923 22:14:42.542966 1567 external_mini_cluster.cc:1428] Setting key bb251f0ba6e47a296eac8950ac65418a
I20260923 22:14:42.545794 13813 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P d7b9fc6eb32b4fb5a1e6920bf231523d: Bootstrap starting.
I20260923 22:14:42.546473 13813 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P d7b9fc6eb32b4fb5a1e6920bf231523d: Neither blocks nor log segments found. Creating new log.
I20260923 22:14:42.546787 13813 log.cc:824] T 00000000000000000000000000000000 P d7b9fc6eb32b4fb5a1e6920bf231523d: Log is configured to *not* fsync() on all Append() calls
I20260923 22:14:42.547472 13813 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P d7b9fc6eb32b4fb5a1e6920bf231523d: No bootstrap required, opened a new log
I20260923 22:14:42.549964 13813 raft_consensus.cc:348] T 00000000000000000000000000000000 P d7b9fc6eb32b4fb5a1e6920bf231523d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d7b9fc6eb32b4fb5a1e6920bf231523d" member_type: VOTER last_known_addr { host: "127.1.135.254" port: 37257 } }
I20260923 22:14:42.550217 13813 raft_consensus.cc:374] T 00000000000000000000000000000000 P d7b9fc6eb32b4fb5a1e6920bf231523d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
Sep 23 22:14:42 dist-test-slave-s2fj krb5kdc[13301](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790201682, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.1.135.254@KRBTEST.COM
I20260923 22:14:42.550343 13813 raft_consensus.cc:729] T 00000000000000000000000000000000 P d7b9fc6eb32b4fb5a1e6920bf231523d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d7b9fc6eb32b4fb5a1e6920bf231523d, State: Initialized, Role: FOLLOWER
I20260923 22:14:42.550495 13813 consensus_queue.cc:260] T 00000000000000000000000000000000 P d7b9fc6eb32b4fb5a1e6920bf231523d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d7b9fc6eb32b4fb5a1e6920bf231523d" member_type: VOTER last_known_addr { host: "127.1.135.254" port: 37257 } }
I20260923 22:14:42.550575 13813 raft_consensus.cc:388] T 00000000000000000000000000000000 P d7b9fc6eb32b4fb5a1e6920bf231523d [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260923 22:14:42.550603 13813 raft_consensus.cc:482] T 00000000000000000000000000000000 P d7b9fc6eb32b4fb5a1e6920bf231523d [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260923 22:14:42.550668 13813 raft_consensus.cc:3037] T 00000000000000000000000000000000 P d7b9fc6eb32b4fb5a1e6920bf231523d [term 0 FOLLOWER]: Advancing to term 1
I20260923 22:14:42.551290 13813 raft_consensus.cc:504] T 00000000000000000000000000000000 P d7b9fc6eb32b4fb5a1e6920bf231523d [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d7b9fc6eb32b4fb5a1e6920bf231523d" member_type: VOTER last_known_addr { host: "127.1.135.254" port: 37257 } }
I20260923 22:14:42.551425 13813 leader_election.cc:302] T 00000000000000000000000000000000 P d7b9fc6eb32b4fb5a1e6920bf231523d [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: d7b9fc6eb32b4fb5a1e6920bf231523d; no voters:
I20260923 22:14:42.551594 13813 leader_election.cc:288] T 00000000000000000000000000000000 P d7b9fc6eb32b4fb5a1e6920bf231523d [CANDIDATE]: Term 1 election: Requested vote from peers
I20260923 22:14:42.551885 13818 raft_consensus.cc:2781] T 00000000000000000000000000000000 P d7b9fc6eb32b4fb5a1e6920bf231523d [term 1 FOLLOWER]: Leader election won for term 1
I20260923 22:14:42.551888 13813 sys_catalog.cc:564] T 00000000000000000000000000000000 P d7b9fc6eb32b4fb5a1e6920bf231523d [sys.catalog]: configured and running, proceeding with master startup.
I20260923 22:14:42.552088 13818 raft_consensus.cc:686] T 00000000000000000000000000000000 P d7b9fc6eb32b4fb5a1e6920bf231523d [term 1 LEADER]: Becoming Leader. State: Replica: d7b9fc6eb32b4fb5a1e6920bf231523d, State: Running, Role: LEADER
I20260923 22:14:42.552225 13818 consensus_queue.cc:237] T 00000000000000000000000000000000 P d7b9fc6eb32b4fb5a1e6920bf231523d [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d7b9fc6eb32b4fb5a1e6920bf231523d" member_type: VOTER last_known_addr { host: "127.1.135.254" port: 37257 } }
I20260923 22:14:42.552577 13818 sys_catalog.cc:455] T 00000000000000000000000000000000 P d7b9fc6eb32b4fb5a1e6920bf231523d [sys.catalog]: SysCatalogTable state changed. Reason: New leader d7b9fc6eb32b4fb5a1e6920bf231523d. Latest consensus state: current_term: 1 leader_uuid: "d7b9fc6eb32b4fb5a1e6920bf231523d" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d7b9fc6eb32b4fb5a1e6920bf231523d" member_type: VOTER last_known_addr { host: "127.1.135.254" port: 37257 } } }
I20260923 22:14:42.552691 13818 sys_catalog.cc:458] T 00000000000000000000000000000000 P d7b9fc6eb32b4fb5a1e6920bf231523d [sys.catalog]: This master's current role is: LEADER
I20260923 22:14:42.553278 13827 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260923 22:14:42.553783 13827 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260923 22:14:42.553136 13819 sys_catalog.cc:455] T 00000000000000000000000000000000 P d7b9fc6eb32b4fb5a1e6920bf231523d [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "d7b9fc6eb32b4fb5a1e6920bf231523d" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d7b9fc6eb32b4fb5a1e6920bf231523d" member_type: VOTER last_known_addr { host: "127.1.135.254" port: 37257 } } }
I20260923 22:14:42.554021 13819 sys_catalog.cc:458] T 00000000000000000000000000000000 P d7b9fc6eb32b4fb5a1e6920bf231523d [sys.catalog]: This master's current role is: LEADER
I20260923 22:14:42.553503 13813 ranger_client.cc:318] Using new properties file: /tmp/dist-test-taskRiQe_a/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790201539513534-1567-0/minicluster-data/master-0/logs/kudu-ranger-subprocess-log4j2.properties
I20260923 22:14:42.554652 13816 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 22:14:42.543500 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40818 (local address 127.1.135.254:37257)
0923 22:14:42.544199 (+ 699us) server_negotiation.cc:206] Beginning negotiation
0923 22:14:42.544209 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0923 22:14:42.544244 (+ 35us) server_negotiation.cc:407] Connection header received
0923 22:14:42.544789 (+ 545us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 22:14:42.544797 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 22:14:42.544859 (+ 62us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 22:14:42.545004 (+ 145us) server_negotiation.cc:226] Negotiated authn=SASL
0923 22:14:42.545805 (+ 801us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 22:14:42.547096 (+ 1291us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 22:14:42.547919 (+ 823us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 22:14:42.548402 (+ 483us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 22:14:42.548528 (+ 126us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 22:14:42.550835 (+ 2307us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0923 22:14:42.550839 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0923 22:14:42.550839 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0923 22:14:42.550850 (+ 11us) server_negotiation.cc:922] calling sasl_server_start()
0923 22:14:42.552792 (+ 1942us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0923 22:14:42.553183 (+ 391us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0923 22:14:42.553186 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0923 22:14:42.553186 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0923 22:14:42.553303 (+ 117us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0923 22:14:42.553502 (+ 199us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0923 22:14:42.553503 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0923 22:14:42.553503 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0923 22:14:42.553805 (+ 302us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0923 22:14:42.553952 (+ 147us) server_negotiation.cc:1035] Waiting for connection context
0923 22:14:42.554258 (+ 306us) server_negotiation.cc:299] Negotiation successful
0923 22:14:42.554366 (+ 108us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":633,"thread_start_us":85,"threads_started":1}
I20260923 22:14:42.560499 13827 catalog_manager.cc:1347] Generated new cluster ID: e3b6d4c0251f47b88a23664e5aa2d502
I20260923 22:14:42.560596 13827 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260923 22:14:42.575431 13827 catalog_manager.cc:1370] Generated new certificate authority record
I20260923 22:14:42.576246 13827 catalog_manager.cc:1504] Loading token signing keys...
I20260923 22:14:42.583124 13827 catalog_manager.cc:5868] T 00000000000000000000000000000000 P d7b9fc6eb32b4fb5a1e6920bf231523d: Generated new TSK 0
I20260923 22:14:42.583356 13827 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.1.135.193@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.1.135.193@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.1.135.193 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskRiQe_a/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790201539513534-1567-0/krb5kdc/kudu_127.1.135.193.keytab.
Entry for principal kudu/127.1.135.193 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskRiQe_a/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790201539513534-1567-0/krb5kdc/kudu_127.1.135.193.keytab.
I20260923 22:14:43.245213 1567 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskRiQe_a/build/release/bin/kudu
/tmp/dist-test-taskRiQe_a/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskRiQe_a/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790201539513534-1567-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskRiQe_a/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790201539513534-1567-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskRiQe_a/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790201539513534-1567-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskRiQe_a/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790201539513534-1567-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.1.135.212:41391/kms
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.1.135.193:0
--local_ip_for_outbound_sockets=127.1.135.193
--webserver_interface=127.1.135.193
--webserver_port=0
--tserver_master_addrs=127.1.135.254:37257
--keytab_file=/tmp/dist-test-taskRiQe_a/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790201539513534-1567-0/krb5kdc/kudu_127.1.135.193.keytab
--principal=kudu/127.1.135.193
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.1.135.212:45915
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskRiQe_a/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790201539513534-1567-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskRiQe_a/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790201539513534-1567-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskRiQe_a/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790201539513534-1567-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260923 22:14:43.347072 13858 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260923 22:14:43.347316 13858 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260923 22:14:43.347347 13858 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260923 22:14:43.348860 13858 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260923 22:14:43.348912 13858 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260923 22:14:43.348946 13858 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.1.135.193
I20260923 22:14:43.350968 13858 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.1.135.212:45915
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-taskRiQe_a/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790201539513534-1567-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskRiQe_a/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790201539513534-1567-0/minicluster-data/ts-0/wal
--ranger_kms_url=127.1.135.212:41391/kms
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskRiQe_a/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790201539513534-1567-0/krb5kdc/kudu_127.1.135.193.keytab
--principal=kudu/127.1.135.193
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.1.135.193:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskRiQe_a/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790201539513534-1567-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.1.135.193
--webserver_port=0
--tserver_master_addrs=127.1.135.254:37257
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.13858
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.1.135.193
--log_dir=/tmp/dist-test-taskRiQe_a/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790201539513534-1567-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 23 Sep 2026 20:53:37 UTC on ccycloud.quasar-rkviyx.root.comops.site
build id 4668083
I20260923 22:14:43.351181 13858 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260923 22:14:43.351437 13858 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260923 22:14:43.357445 13864 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260923 22:14:43.357476 13866 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260923 22:14:43.357558 13858 server_base.cc:1034] running on GCE node
W20260923 22:14:43.357604 13863 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260923 22:14:43.357882 13858 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260923 22:14:43.358160 13858 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260923 22:14:43.359329 13858 hybrid_clock.cc:648] HybridClock initialized: now 1790201683359305 us; error 40 us; skew 500 ppm
Sep 23 22:14:43 dist-test-slave-s2fj krb5kdc[13301](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790201683, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.135.193@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260923 22:14:43.362418 13858 init.cc:377] Logged in from keytab as kudu/127.1.135.193@KRBTEST.COM (short username kudu)
I20260923 22:14:43.362906 13858 webserver.cc:460] Webserver started at http://127.1.135.193:34343/ using document root <none> and password file <none>
I20260923 22:14:43.363139 13858 fs_manager.cc:359] Metadata directory not provided
I20260923 22:14:43.363188 13858 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260923 22:14:43.363323 13858 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
Sep 23 22:14:43 dist-test-slave-s2fj krb5kdc[13301](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790201683, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.135.193@KRBTEST.COM for HTTP/127.1.135.212@KRBTEST.COM
I20260923 22:14:43.376240 13858 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskRiQe_a/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790201539513534-1567-0/minicluster-data/ts-0/data/instance:
uuid: "0ff4f9455d6444269b402aa13dc889e6"
format_stamp: "Formatted at 2026-09-23 22:14:43 on dist-test-slave-s2fj"
server_key: "3efb6651219b1fef04ac8b8f3b62b42c"
server_key_iv: "bb8382eae421842cc683d782ca3b66d5"
server_key_version: "kuduclusterkey@0"
I20260923 22:14:43.376720 13858 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskRiQe_a/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790201539513534-1567-0/minicluster-data/ts-0/wal/instance:
uuid: "0ff4f9455d6444269b402aa13dc889e6"
format_stamp: "Formatted at 2026-09-23 22:14:43 on dist-test-slave-s2fj"
server_key: "3efb6651219b1fef04ac8b8f3b62b42c"
server_key_iv: "bb8382eae421842cc683d782ca3b66d5"
server_key_version: "kuduclusterkey@0"
I20260923 22:14:43.378392 13858 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.001s sys 0.000s
I20260923 22:14:43.392176 13875 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260923 22:14:43.392365 13858 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260923 22:14:43.392439 13858 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskRiQe_a/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790201539513534-1567-0/minicluster-data/ts-0/data,/tmp/dist-test-taskRiQe_a/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790201539513534-1567-0/minicluster-data/ts-0/wal
uuid: "0ff4f9455d6444269b402aa13dc889e6"
format_stamp: "Formatted at 2026-09-23 22:14:43 on dist-test-slave-s2fj"
server_key: "3efb6651219b1fef04ac8b8f3b62b42c"
server_key_iv: "bb8382eae421842cc683d782ca3b66d5"
server_key_version: "kuduclusterkey@0"
I20260923 22:14:43.392503 13858 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskRiQe_a/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790201539513534-1567-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskRiQe_a/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790201539513534-1567-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskRiQe_a/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790201539513534-1567-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260923 22:14:43.400772 13858 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260923 22:14:43.403100 13858 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260923 22:14:43.403276 13858 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260923 22:14:43.403523 13858 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260923 22:14:43.403887 13858 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260923 22:14:43.403970 13858 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260923 22:14:43.404091 13858 ts_tablet_manager.cc:610] Registered 0 tablets
I20260923 22:14:43.404170 13858 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260923 22:14:43.411271 13858 rpc_server.cc:307] RPC server started. Bound to: 127.1.135.193:34647
I20260923 22:14:43.411334 13988 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.1.135.193:34647 every 8 connection(s)
I20260923 22:14:43.411723 13858 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskRiQe_a/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790201539513534-1567-0/minicluster-data/ts-0/data/info.pb
I20260923 22:14:43.412117 1567 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskRiQe_a/build/release/bin/kudu as pid 13858
I20260923 22:14:43.412214 1567 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskRiQe_a/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790201539513534-1567-0/minicluster-data/ts-0/wal/instance
I20260923 22:14:43.412420 1567 external_mini_cluster.cc:1428] Setting key 14d14c7b0bb135c52e86a1a511489e06
Sep 23 22:14:43 dist-test-slave-s2fj krb5kdc[13301](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790201683, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.135.193@KRBTEST.COM for kudu/127.1.135.254@KRBTEST.COM
WARNING: no policy specified for kudu/127.1.135.194@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.1.135.194@KRBTEST.COM" created.
I20260923 22:14:43.425902 13992 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 22:14:43.413104 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.135.193:33677 (local address 127.1.135.254:37257)
0923 22:14:43.413278 (+ 174us) server_negotiation.cc:206] Beginning negotiation
0923 22:14:43.413279 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0923 22:14:43.413506 (+ 227us) server_negotiation.cc:407] Connection header received
0923 22:14:43.414332 (+ 826us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 22:14:43.414334 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 22:14:43.414351 (+ 17us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 22:14:43.414438 (+ 87us) server_negotiation.cc:226] Negotiated authn=SASL
0923 22:14:43.416827 (+ 2389us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 22:14:43.417643 (+ 816us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 22:14:43.419158 (+ 1515us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 22:14:43.419477 (+ 319us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 22:14:43.419528 (+ 51us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 22:14:43.422200 (+ 2672us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0923 22:14:43.422202 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0923 22:14:43.422202 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0923 22:14:43.422209 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0923 22:14:43.424182 (+ 1973us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0923 22:14:43.424678 (+ 496us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0923 22:14:43.424680 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0923 22:14:43.424680 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0923 22:14:43.424782 (+ 102us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0923 22:14:43.425080 (+ 298us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0923 22:14:43.425081 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0923 22:14:43.425081 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0923 22:14:43.425278 (+ 197us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0923 22:14:43.425348 (+ 70us) server_negotiation.cc:1035] Waiting for connection context
0923 22:14:43.425701 (+ 353us) server_negotiation.cc:299] Negotiation successful
0923 22:14:43.425789 (+ 88us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":129,"thread_start_us":87,"threads_started":1}
I20260923 22:14:43.426415 13991 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 22:14:43.413002 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.135.254:37257 (local address 127.1.135.193:33677)
0923 22:14:43.413348 (+ 346us) negotiation.cc:107] Waiting for socket to connect
0923 22:14:43.413399 (+ 51us) client_negotiation.cc:174] Beginning negotiation
0923 22:14:43.414134 (+ 735us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0923 22:14:43.416184 (+ 2050us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0923 22:14:43.416189 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0923 22:14:43.416439 (+ 250us) client_negotiation.cc:189] Negotiated authn=SASL
0923 22:14:43.416710 (+ 271us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 22:14:43.416716 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 22:14:43.417763 (+ 1047us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 22:14:43.417764 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 22:14:43.419034 (+ 1270us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 22:14:43.419036 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 22:14:43.419646 (+ 610us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 22:14:43.419647 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 22:14:43.419785 (+ 138us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 22:14:43.420416 (+ 631us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0923 22:14:43.420426 (+ 10us) client_negotiation.cc:656] calling sasl_client_start()
0923 22:14:43.422072 (+ 1646us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0923 22:14:43.424265 (+ 2193us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0923 22:14:43.424271 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0923 22:14:43.424273 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0923 22:14:43.424561 (+ 288us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0923 22:14:43.424861 (+ 300us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0923 22:14:43.424862 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0923 22:14:43.424862 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0923 22:14:43.424994 (+ 132us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0923 22:14:43.425380 (+ 386us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0923 22:14:43.425386 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0923 22:14:43.425539 (+ 153us) client_negotiation.cc:769] Sending connection context
0923 22:14:43.426027 (+ 488us) client_negotiation.cc:240] Negotiation successful
0923 22:14:43.426191 (+ 164us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":300,"thread_start_us":80,"threads_started":1}
I20260923 22:14:43.426949 13989 heartbeater.cc:347] Connected to a master server at 127.1.135.254:37257
I20260923 22:14:43.427054 13989 heartbeater.cc:464] Registering TS with master...
I20260923 22:14:43.427240 13989 heartbeater.cc:511] Master 127.1.135.254:37257 requested a full tablet report, sending...
I20260923 22:14:43.427872 13777 ts_manager.cc:194] Registered new tserver with Master: 0ff4f9455d6444269b402aa13dc889e6 (127.1.135.193:34647)
I20260923 22:14:43.428772 13777 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.1.135.193@KRBTEST.COM'} at 127.1.135.193:33677
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.1.135.194 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskRiQe_a/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790201539513534-1567-0/krb5kdc/kudu_127.1.135.194.keytab.
Entry for principal kudu/127.1.135.194 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskRiQe_a/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790201539513534-1567-0/krb5kdc/kudu_127.1.135.194.keytab.
I20260923 22:14:43.444794 1567 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskRiQe_a/build/release/bin/kudu
/tmp/dist-test-taskRiQe_a/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskRiQe_a/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790201539513534-1567-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskRiQe_a/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790201539513534-1567-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskRiQe_a/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790201539513534-1567-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskRiQe_a/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790201539513534-1567-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.1.135.212:41391/kms
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.1.135.194:0
--local_ip_for_outbound_sockets=127.1.135.194
--webserver_interface=127.1.135.194
--webserver_port=0
--tserver_master_addrs=127.1.135.254:37257
--keytab_file=/tmp/dist-test-taskRiQe_a/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790201539513534-1567-0/krb5kdc/kudu_127.1.135.194.keytab
--principal=kudu/127.1.135.194
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.1.135.212:45915
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskRiQe_a/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790201539513534-1567-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskRiQe_a/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790201539513534-1567-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskRiQe_a/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790201539513534-1567-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260923 22:14:43.560668 13995 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260923 22:14:43.560984 13995 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260923 22:14:43.561043 13995 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260923 22:14:43.562696 13995 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260923 22:14:43.562752 13995 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260923 22:14:43.562793 13995 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.1.135.194
Sep 23 22:14:43 dist-test-slave-s2fj krb5kdc[13301](info): AS_REQ (1 etypes {17}) 127.0.0.1: ISSUE: authtime 1790201683, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.135.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260923 22:14:43.564953 13995 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.1.135.212:45915
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-taskRiQe_a/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790201539513534-1567-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskRiQe_a/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790201539513534-1567-0/minicluster-data/ts-1/wal
--ranger_kms_url=127.1.135.212:41391/kms
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskRiQe_a/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790201539513534-1567-0/krb5kdc/kudu_127.1.135.194.keytab
--principal=kudu/127.1.135.194
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.1.135.194:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskRiQe_a/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790201539513534-1567-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.1.135.194
--webserver_port=0
--tserver_master_addrs=127.1.135.254:37257
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.13995
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.1.135.194
--log_dir=/tmp/dist-test-taskRiQe_a/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790201539513534-1567-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 23 Sep 2026 20:53:37 UTC on ccycloud.quasar-rkviyx.root.comops.site
build id 4668083
I20260923 22:14:43.565197 13995 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260923 22:14:43.565526 13995 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260923 22:14:43.571529 14002 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260923 22:14:43.571815 14005 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260923 22:14:43.571892 13995 server_base.cc:1034] running on GCE node
W20260923 22:14:43.572099 14003 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260923 22:14:43.572379 13995 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260923 22:14:43.572664 13995 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260923 22:14:43.573834 13995 hybrid_clock.cc:648] HybridClock initialized: now 1790201683573815 us; error 32 us; skew 500 ppm
Sep 23 22:14:43 dist-test-slave-s2fj krb5kdc[13301](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790201683, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.135.194@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260923 22:14:43.577020 13995 init.cc:377] Logged in from keytab as kudu/127.1.135.194@KRBTEST.COM (short username kudu)
I20260923 22:14:43.577479 13995 webserver.cc:460] Webserver started at http://127.1.135.194:41919/ using document root <none> and password file <none>
I20260923 22:14:43.577726 13995 fs_manager.cc:359] Metadata directory not provided
I20260923 22:14:43.577777 13995 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260923 22:14:43.577886 13995 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
Sep 23 22:14:43 dist-test-slave-s2fj krb5kdc[13301](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790201683, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.135.194@KRBTEST.COM for HTTP/127.1.135.212@KRBTEST.COM
I20260923 22:14:43.594354 13995 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskRiQe_a/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790201539513534-1567-0/minicluster-data/ts-1/data/instance:
uuid: "ff2dd7276bc647e3b2cee0d9ec03c525"
format_stamp: "Formatted at 2026-09-23 22:14:43 on dist-test-slave-s2fj"
server_key: "8cd22ac1702477cb05bd74b66b822202"
server_key_iv: "b80e508194ee99ce453e173f2b4cdba9"
server_key_version: "kuduclusterkey@0"
I20260923 22:14:43.594798 13995 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskRiQe_a/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790201539513534-1567-0/minicluster-data/ts-1/wal/instance:
uuid: "ff2dd7276bc647e3b2cee0d9ec03c525"
format_stamp: "Formatted at 2026-09-23 22:14:43 on dist-test-slave-s2fj"
server_key: "8cd22ac1702477cb05bd74b66b822202"
server_key_iv: "b80e508194ee99ce453e173f2b4cdba9"
server_key_version: "kuduclusterkey@0"
I20260923 22:14:43.604532 13995 fs_manager.cc:691] Time spent creating directory manager: real 0.010s user 0.001s sys 0.000s
I20260923 22:14:43.619910 14012 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260923 22:14:43.620136 13995 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260923 22:14:43.620222 13995 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskRiQe_a/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790201539513534-1567-0/minicluster-data/ts-1/data,/tmp/dist-test-taskRiQe_a/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790201539513534-1567-0/minicluster-data/ts-1/wal
uuid: "ff2dd7276bc647e3b2cee0d9ec03c525"
format_stamp: "Formatted at 2026-09-23 22:14:43 on dist-test-slave-s2fj"
server_key: "8cd22ac1702477cb05bd74b66b822202"
server_key_iv: "b80e508194ee99ce453e173f2b4cdba9"
server_key_version: "kuduclusterkey@0"
I20260923 22:14:43.620293 13995 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskRiQe_a/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790201539513534-1567-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskRiQe_a/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790201539513534-1567-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskRiQe_a/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790201539513534-1567-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260923 22:14:43.633111 13995 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260923 22:14:43.636791 13995 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260923 22:14:43.636965 13995 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260923 22:14:43.637180 13995 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260923 22:14:43.637537 13995 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260923 22:14:43.637610 13995 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260923 22:14:43.637665 13995 ts_tablet_manager.cc:610] Registered 0 tablets
I20260923 22:14:43.637708 13995 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260923 22:14:43.643883 13995 rpc_server.cc:307] RPC server started. Bound to: 127.1.135.194:39053
I20260923 22:14:43.644346 13995 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskRiQe_a/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790201539513534-1567-0/minicluster-data/ts-1/data/info.pb
I20260923 22:14:43.647027 14125 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.1.135.194:39053 every 8 connection(s)
I20260923 22:14:43.648566 1567 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskRiQe_a/build/release/bin/kudu as pid 13995
I20260923 22:14:43.648689 1567 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskRiQe_a/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790201539513534-1567-0/minicluster-data/ts-1/wal/instance
I20260923 22:14:43.648877 1567 external_mini_cluster.cc:1428] Setting key a6f800eb5a0e5de12f975e9c41a80828
Sep 23 22:14:43 dist-test-slave-s2fj krb5kdc[13301](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790201683, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.135.194@KRBTEST.COM for kudu/127.1.135.254@KRBTEST.COM
I20260923 22:14:43.655951 13992 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 22:14:43.645135 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.135.194:55713 (local address 127.1.135.254:37257)
0923 22:14:43.645214 (+ 79us) server_negotiation.cc:206] Beginning negotiation
0923 22:14:43.645216 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0923 22:14:43.646033 (+ 817us) server_negotiation.cc:407] Connection header received
0923 22:14:43.646756 (+ 723us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 22:14:43.646757 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 22:14:43.646777 (+ 20us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 22:14:43.646855 (+ 78us) server_negotiation.cc:226] Negotiated authn=SASL
0923 22:14:43.647570 (+ 715us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 22:14:43.648056 (+ 486us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 22:14:43.649554 (+ 1498us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 22:14:43.649869 (+ 315us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 22:14:43.649946 (+ 77us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 22:14:43.652668 (+ 2722us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0923 22:14:43.652670 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0923 22:14:43.652670 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0923 22:14:43.652677 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0923 22:14:43.654417 (+ 1740us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0923 22:14:43.654916 (+ 499us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0923 22:14:43.654917 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0923 22:14:43.654917 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0923 22:14:43.654975 (+ 58us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0923 22:14:43.655241 (+ 266us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0923 22:14:43.655242 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0923 22:14:43.655242 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0923 22:14:43.655363 (+ 121us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0923 22:14:43.655404 (+ 41us) server_negotiation.cc:1035] Waiting for connection context
0923 22:14:43.655766 (+ 362us) server_negotiation.cc:299] Negotiation successful
0923 22:14:43.655853 (+ 87us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":47}
I20260923 22:14:43.656765 14129 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 22:14:43.645118 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.135.254:37257 (local address 127.1.135.194:55713)
0923 22:14:43.645900 (+ 782us) negotiation.cc:107] Waiting for socket to connect
0923 22:14:43.645946 (+ 46us) client_negotiation.cc:174] Beginning negotiation
0923 22:14:43.646613 (+ 667us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0923 22:14:43.646948 (+ 335us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0923 22:14:43.646952 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0923 22:14:43.647209 (+ 257us) client_negotiation.cc:189] Negotiated authn=SASL
0923 22:14:43.647475 (+ 266us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 22:14:43.647482 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 22:14:43.648193 (+ 711us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 22:14:43.648194 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 22:14:43.649456 (+ 1262us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 22:14:43.649459 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 22:14:43.649983 (+ 524us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 22:14:43.649984 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 22:14:43.650119 (+ 135us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 22:14:43.650653 (+ 534us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0923 22:14:43.650663 (+ 10us) client_negotiation.cc:656] calling sasl_client_start()
0923 22:14:43.652544 (+ 1881us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0923 22:14:43.654504 (+ 1960us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0923 22:14:43.654510 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0923 22:14:43.654512 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0923 22:14:43.654809 (+ 297us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0923 22:14:43.655048 (+ 239us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0923 22:14:43.655049 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0923 22:14:43.655050 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0923 22:14:43.655166 (+ 116us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0923 22:14:43.655447 (+ 281us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0923 22:14:43.655452 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0923 22:14:43.655597 (+ 145us) client_negotiation.cc:769] Sending connection context
0923 22:14:43.656377 (+ 780us) client_negotiation.cc:240] Negotiation successful
0923 22:14:43.656541 (+ 164us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":720,"thread_start_us":655,"threads_started":1}
I20260923 22:14:43.660077 14127 heartbeater.cc:347] Connected to a master server at 127.1.135.254:37257
I20260923 22:14:43.660181 14127 heartbeater.cc:464] Registering TS with master...
I20260923 22:14:43.660356 14127 heartbeater.cc:511] Master 127.1.135.254:37257 requested a full tablet report, sending...
I20260923 22:14:43.660781 13777 ts_manager.cc:194] Registered new tserver with Master: ff2dd7276bc647e3b2cee0d9ec03c525 (127.1.135.194:39053)
I20260923 22:14:43.661265 13777 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.1.135.194@KRBTEST.COM'} at 127.1.135.194:55713
WARNING: no policy specified for kudu/127.1.135.195@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.1.135.195@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.1.135.195 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskRiQe_a/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790201539513534-1567-0/krb5kdc/kudu_127.1.135.195.keytab.
Entry for principal kudu/127.1.135.195 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskRiQe_a/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790201539513534-1567-0/krb5kdc/kudu_127.1.135.195.keytab.
I20260923 22:14:43.699829 1567 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskRiQe_a/build/release/bin/kudu
/tmp/dist-test-taskRiQe_a/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskRiQe_a/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790201539513534-1567-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskRiQe_a/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790201539513534-1567-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskRiQe_a/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790201539513534-1567-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskRiQe_a/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790201539513534-1567-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.1.135.212:41391/kms
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.1.135.195:0
--local_ip_for_outbound_sockets=127.1.135.195
--webserver_interface=127.1.135.195
--webserver_port=0
--tserver_master_addrs=127.1.135.254:37257
--keytab_file=/tmp/dist-test-taskRiQe_a/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790201539513534-1567-0/krb5kdc/kudu_127.1.135.195.keytab
--principal=kudu/127.1.135.195
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.1.135.212:45915
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskRiQe_a/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790201539513534-1567-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskRiQe_a/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790201539513534-1567-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskRiQe_a/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790201539513534-1567-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260923 22:14:43.817809 14132 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260923 22:14:43.818159 14132 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260923 22:14:43.818254 14132 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260923 22:14:43.819952 14132 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260923 22:14:43.820055 14132 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260923 22:14:43.820168 14132 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.1.135.195
I20260923 22:14:43.822324 14132 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.1.135.212:45915
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-taskRiQe_a/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790201539513534-1567-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskRiQe_a/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790201539513534-1567-0/minicluster-data/ts-2/wal
--ranger_kms_url=127.1.135.212:41391/kms
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskRiQe_a/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790201539513534-1567-0/krb5kdc/kudu_127.1.135.195.keytab
--principal=kudu/127.1.135.195
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.1.135.195:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskRiQe_a/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790201539513534-1567-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.1.135.195
--webserver_port=0
--tserver_master_addrs=127.1.135.254:37257
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.14132
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.1.135.195
--log_dir=/tmp/dist-test-taskRiQe_a/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790201539513534-1567-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 23 Sep 2026 20:53:37 UTC on ccycloud.quasar-rkviyx.root.comops.site
build id 4668083
I20260923 22:14:43.822775 14132 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260923 22:14:43.823112 14132 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260923 22:14:43.829787 14138 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260923 22:14:43.830106 14137 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260923 22:14:43.830315 14140 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260923 22:14:43.831275 14132 server_base.cc:1034] running on GCE node
I20260923 22:14:43.831566 14132 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260923 22:14:43.831845 14132 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260923 22:14:43.839217 14132 hybrid_clock.cc:648] HybridClock initialized: now 1790201683839193 us; error 38 us; skew 500 ppm
Sep 23 22:14:43 dist-test-slave-s2fj krb5kdc[13301](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790201683, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.135.195@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260923 22:14:43.842348 14132 init.cc:377] Logged in from keytab as kudu/127.1.135.195@KRBTEST.COM (short username kudu)
I20260923 22:14:43.842828 14132 webserver.cc:460] Webserver started at http://127.1.135.195:42429/ using document root <none> and password file <none>
I20260923 22:14:43.843086 14132 fs_manager.cc:359] Metadata directory not provided
I20260923 22:14:43.843145 14132 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260923 22:14:43.843264 14132 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
Sep 23 22:14:43 dist-test-slave-s2fj krb5kdc[13301](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790201683, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.135.195@KRBTEST.COM for HTTP/127.1.135.212@KRBTEST.COM
I20260923 22:14:43.861786 14132 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskRiQe_a/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790201539513534-1567-0/minicluster-data/ts-2/data/instance:
uuid: "226dc984ad3b4589883715494ff740a4"
format_stamp: "Formatted at 2026-09-23 22:14:43 on dist-test-slave-s2fj"
server_key: "86c506ca04f9bc65cb455358c82082d2"
server_key_iv: "1bbbde8b89707045af175e5b3f630045"
server_key_version: "kuduclusterkey@0"
I20260923 22:14:43.862187 14132 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskRiQe_a/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790201539513534-1567-0/minicluster-data/ts-2/wal/instance:
uuid: "226dc984ad3b4589883715494ff740a4"
format_stamp: "Formatted at 2026-09-23 22:14:43 on dist-test-slave-s2fj"
server_key: "86c506ca04f9bc65cb455358c82082d2"
server_key_iv: "1bbbde8b89707045af175e5b3f630045"
server_key_version: "kuduclusterkey@0"
I20260923 22:14:43.863708 14132 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.001s sys 0.002s
I20260923 22:14:43.881313 14147 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260923 22:14:43.881584 14132 fs_manager.cc:725] Time spent opening block manager: real 0.004s user 0.001s sys 0.000s
I20260923 22:14:43.881699 14132 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskRiQe_a/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790201539513534-1567-0/minicluster-data/ts-2/data,/tmp/dist-test-taskRiQe_a/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790201539513534-1567-0/minicluster-data/ts-2/wal
uuid: "226dc984ad3b4589883715494ff740a4"
format_stamp: "Formatted at 2026-09-23 22:14:43 on dist-test-slave-s2fj"
server_key: "86c506ca04f9bc65cb455358c82082d2"
server_key_iv: "1bbbde8b89707045af175e5b3f630045"
server_key_version: "kuduclusterkey@0"
I20260923 22:14:43.881796 14132 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskRiQe_a/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790201539513534-1567-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskRiQe_a/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790201539513534-1567-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskRiQe_a/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790201539513534-1567-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260923 22:14:43.893785 14132 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260923 22:14:43.901845 14132 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260923 22:14:43.902037 14132 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260923 22:14:43.902311 14132 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260923 22:14:43.902817 14132 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260923 22:14:43.902909 14132 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260923 22:14:43.902979 14132 ts_tablet_manager.cc:610] Registered 0 tablets
I20260923 22:14:43.903054 14132 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260923 22:14:43.910017 14132 rpc_server.cc:307] RPC server started. Bound to: 127.1.135.195:43765
I20260923 22:14:43.910548 14132 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskRiQe_a/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790201539513534-1567-0/minicluster-data/ts-2/data/info.pb
I20260923 22:14:43.913465 14260 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.1.135.195:43765 every 8 connection(s)
I20260923 22:14:43.916458 1567 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskRiQe_a/build/release/bin/kudu as pid 14132
I20260923 22:14:43.916579 1567 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskRiQe_a/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790201539513534-1567-0/minicluster-data/ts-2/wal/instance
I20260923 22:14:43.916744 1567 external_mini_cluster.cc:1428] Setting key acef2ce02ed3964fe16f7972e20aa8f8
Sep 23 22:14:43 dist-test-slave-s2fj krb5kdc[13301](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790201683, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.135.195@KRBTEST.COM for kudu/127.1.135.254@KRBTEST.COM
I20260923 22:14:43.925429 13992 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 22:14:43.913058 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.135.195:55815 (local address 127.1.135.254:37257)
0923 22:14:43.913105 (+ 47us) server_negotiation.cc:206] Beginning negotiation
0923 22:14:43.913107 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0923 22:14:43.914296 (+ 1189us) server_negotiation.cc:407] Connection header received
0923 22:14:43.915055 (+ 759us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 22:14:43.915057 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 22:14:43.915070 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 22:14:43.915102 (+ 32us) server_negotiation.cc:226] Negotiated authn=SASL
0923 22:14:43.916003 (+ 901us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 22:14:43.916787 (+ 784us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 22:14:43.918965 (+ 2178us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 22:14:43.919253 (+ 288us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 22:14:43.919302 (+ 49us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 22:14:43.922057 (+ 2755us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0923 22:14:43.922059 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0923 22:14:43.922059 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0923 22:14:43.922066 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0923 22:14:43.923835 (+ 1769us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0923 22:14:43.924372 (+ 537us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0923 22:14:43.924373 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0923 22:14:43.924374 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0923 22:14:43.924428 (+ 54us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0923 22:14:43.924717 (+ 289us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0923 22:14:43.924718 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0923 22:14:43.924718 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0923 22:14:43.924846 (+ 128us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0923 22:14:43.924889 (+ 43us) server_negotiation.cc:1035] Waiting for connection context
0923 22:14:43.925258 (+ 369us) server_negotiation.cc:299] Negotiation successful
0923 22:14:43.925336 (+ 78us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":22}
I20260923 22:14:43.926162 14263 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 22:14:43.912704 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.135.254:37257 (local address 127.1.135.195:55815)
0923 22:14:43.914113 (+ 1409us) negotiation.cc:107] Waiting for socket to connect
0923 22:14:43.914209 (+ 96us) client_negotiation.cc:174] Beginning negotiation
0923 22:14:43.914896 (+ 687us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0923 22:14:43.915343 (+ 447us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0923 22:14:43.915348 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0923 22:14:43.915620 (+ 272us) client_negotiation.cc:189] Negotiated authn=SASL
0923 22:14:43.915890 (+ 270us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 22:14:43.915896 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 22:14:43.917572 (+ 1676us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 22:14:43.917573 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 22:14:43.918870 (+ 1297us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 22:14:43.918873 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 22:14:43.919428 (+ 555us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 22:14:43.919429 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 22:14:43.919570 (+ 141us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 22:14:43.920292 (+ 722us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0923 22:14:43.920302 (+ 10us) client_negotiation.cc:656] calling sasl_client_start()
0923 22:14:43.921936 (+ 1634us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0923 22:14:43.923932 (+ 1996us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0923 22:14:43.923938 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0923 22:14:43.923940 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0923 22:14:43.924284 (+ 344us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0923 22:14:43.924502 (+ 218us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0923 22:14:43.924502 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0923 22:14:43.924503 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0923 22:14:43.924634 (+ 131us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0923 22:14:43.924936 (+ 302us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0923 22:14:43.924943 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0923 22:14:43.925090 (+ 147us) client_negotiation.cc:769] Sending connection context
0923 22:14:43.925769 (+ 679us) client_negotiation.cc:240] Negotiation successful
0923 22:14:43.925922 (+ 153us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1368,"thread_start_us":76,"threads_started":1}
I20260923 22:14:43.926822 14261 heartbeater.cc:347] Connected to a master server at 127.1.135.254:37257
I20260923 22:14:43.926913 14261 heartbeater.cc:464] Registering TS with master...
I20260923 22:14:43.927091 14261 heartbeater.cc:511] Master 127.1.135.254:37257 requested a full tablet report, sending...
I20260923 22:14:43.927512 13777 ts_manager.cc:194] Registered new tserver with Master: 226dc984ad3b4589883715494ff740a4 (127.1.135.195:43765)
I20260923 22:14:43.927985 13777 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.1.135.195@KRBTEST.COM'} at 127.1.135.195:55815
I20260923 22:14:43.928920 1567 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260923 22:14:43.929627 1567 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskRiQe_a/build/release/bin/kudu with pid 13858
I20260923 22:14:43.935983 1567 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskRiQe_a/build/release/bin/kudu with pid 13995
I20260923 22:14:43.943141 1567 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskRiQe_a/build/release/bin/kudu with pid 14132
I20260923 22:14:43.949647 1567 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskRiQe_a/build/release/bin/kudu with pid 13745
I20260923 22:14:43.953835 1567 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskRiQe_a/build/release/bin/kudu
/tmp/dist-test-taskRiQe_a/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskRiQe_a/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790201539513534-1567-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskRiQe_a/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790201539513534-1567-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskRiQe_a/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790201539513534-1567-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskRiQe_a/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790201539513534-1567-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.1.135.254:37257
--webserver_interface=127.1.135.254
--webserver_port=41145
--keytab_file=/tmp/dist-test-taskRiQe_a/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790201539513534-1567-0/krb5kdc/kudu_127.1.135.254.keytab
--principal=kudu/127.1.135.254
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.1.135.212:45915
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.1.135.254:37257
--ranger_config_path=/tmp/dist-test-taskRiQe_a/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790201539513534-1567-0/minicluster-data/ranger-client
--trusted_user_acl=test-admin
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.1.135.212:41391/kms
--rpc_trace_negotiation
--ranger_kms_url=invalid.host:1234/kms,127.1.135.212:41391/kms with env {KRB5CCNAME=/tmp/dist-test-taskRiQe_a/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790201539513534-1567-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskRiQe_a/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790201539513534-1567-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskRiQe_a/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790201539513534-1567-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260923 22:14:44.033258 14264 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260923 22:14:44.033433 14264 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260923 22:14:44.033452 14264 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260923 22:14:44.034492 14264 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260923 22:14:44.034528 14264 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260923 22:14:44.034541 14264 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260923 22:14:44.034554 14264 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260923 22:14:44.034564 14264 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260923 22:14:44.035914 14264 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.1.135.212:45915
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-taskRiQe_a/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790201539513534-1567-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskRiQe_a/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790201539513534-1567-0/minicluster-data/master-0/wal
--ranger_kms_url=invalid.host:1234/kms,127.1.135.212:41391/kms
--trusted_user_acl=<redacted>
--ipki_ca_key_size=768
--master_addresses=127.1.135.254:37257
--ranger_config_path=/tmp/dist-test-taskRiQe_a/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790201539513534-1567-0/minicluster-data/ranger-client
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskRiQe_a/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790201539513534-1567-0/krb5kdc/kudu_127.1.135.254.keytab
--principal=kudu/127.1.135.254
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.1.135.254:37257
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskRiQe_a/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790201539513534-1567-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.1.135.254
--webserver_port=41145
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.14264
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskRiQe_a/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790201539513534-1567-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 23 Sep 2026 20:53:37 UTC on ccycloud.quasar-rkviyx.root.comops.site
build id 4668083
I20260923 22:14:44.036128 14264 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260923 22:14:44.036348 14264 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260923 22:14:44.040768 14270 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260923 22:14:44.040894 14272 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260923 22:14:44.040784 14269 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260923 22:14:44.040993 14264 server_base.cc:1034] running on GCE node
I20260923 22:14:44.041195 14264 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260923 22:14:44.041555 14264 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260923 22:14:44.042711 14264 hybrid_clock.cc:648] HybridClock initialized: now 1790201684042700 us; error 27 us; skew 500 ppm
Sep 23 22:14:44 dist-test-slave-s2fj krb5kdc[13301](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790201684, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.135.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260923 22:14:44.045185 14264 init.cc:377] Logged in from keytab as kudu/127.1.135.254@KRBTEST.COM (short username kudu)
I20260923 22:14:44.045643 14264 webserver.cc:460] Webserver started at http://127.1.135.254:41145/ using document root <none> and password file <none>
I20260923 22:14:44.045881 14264 fs_manager.cc:359] Metadata directory not provided
I20260923 22:14:44.045928 14264 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
Sep 23 22:14:44 dist-test-slave-s2fj krb5kdc[13301](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790201684, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.135.254@KRBTEST.COM for HTTP/127.1.135.212@KRBTEST.COM
I20260923 22:14:44.089998 14264 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.001s sys 0.000s
I20260923 22:14:44.090762 14280 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260923 22:14:44.090953 14264 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260923 22:14:44.091055 14264 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskRiQe_a/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790201539513534-1567-0/minicluster-data/master-0/data,/tmp/dist-test-taskRiQe_a/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790201539513534-1567-0/minicluster-data/master-0/wal
uuid: "d7b9fc6eb32b4fb5a1e6920bf231523d"
format_stamp: "Formatted at 2026-09-23 22:14:42 on dist-test-slave-s2fj"
server_key: "910f35218cce50034486a37a864f6ba0"
server_key_iv: "1cae74aa93f84b27277249f9c20c8066"
server_key_version: "kuduclusterkey@0"
I20260923 22:14:44.091485 14264 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskRiQe_a/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790201539513534-1567-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskRiQe_a/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790201539513534-1567-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskRiQe_a/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790201539513534-1567-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260923 22:14:44.103922 14264 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260923 22:14:44.105705 14264 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260923 22:14:44.105854 14264 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260923 22:14:44.110487 14264 rpc_server.cc:307] RPC server started. Bound to: 127.1.135.254:37257
I20260923 22:14:44.110512 14332 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.1.135.254:37257 every 8 connection(s)
I20260923 22:14:44.110952 14264 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskRiQe_a/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790201539513534-1567-0/minicluster-data/master-0/data/info.pb
I20260923 22:14:44.112150 14333 sys_catalog.cc:263] Verifying existing consensus state
I20260923 22:14:44.112756 14333 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P d7b9fc6eb32b4fb5a1e6920bf231523d: Bootstrap starting.
I20260923 22:14:44.119985 1567 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskRiQe_a/build/release/bin/kudu as pid 14264
I20260923 22:14:44.120344 1567 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskRiQe_a/build/release/bin/kudu
/tmp/dist-test-taskRiQe_a/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskRiQe_a/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790201539513534-1567-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskRiQe_a/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790201539513534-1567-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskRiQe_a/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790201539513534-1567-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskRiQe_a/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790201539513534-1567-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.1.135.212:41391/kms
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.1.135.193:34647
--local_ip_for_outbound_sockets=127.1.135.193
--tserver_master_addrs=127.1.135.254:37257
--webserver_port=34343
--webserver_interface=127.1.135.193
--keytab_file=/tmp/dist-test-taskRiQe_a/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790201539513534-1567-0/krb5kdc/kudu_127.1.135.193.keytab
--principal=kudu/127.1.135.193
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.1.135.212:45915
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--ranger_kms_url=invalid.host:1234/kms,127.1.135.212:41391/kms with env {KRB5CCNAME=/tmp/dist-test-taskRiQe_a/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790201539513534-1567-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskRiQe_a/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790201539513534-1567-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskRiQe_a/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790201539513534-1567-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260923 22:14:44.120533 14333 log.cc:824] T 00000000000000000000000000000000 P d7b9fc6eb32b4fb5a1e6920bf231523d: Log is configured to *not* fsync() on all Append() calls
I20260923 22:14:44.122550 14333 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P d7b9fc6eb32b4fb5a1e6920bf231523d: Bootstrap replayed 1/1 log segments. Stats: ops{read=4 overwritten=0 applied=4 ignored=0} inserts{seen=3 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260923 22:14:44.122874 14333 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P d7b9fc6eb32b4fb5a1e6920bf231523d: Bootstrap complete.
I20260923 22:14:44.125324 14333 raft_consensus.cc:348] T 00000000000000000000000000000000 P d7b9fc6eb32b4fb5a1e6920bf231523d [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d7b9fc6eb32b4fb5a1e6920bf231523d" member_type: VOTER last_known_addr { host: "127.1.135.254" port: 37257 } }
I20260923 22:14:44.125556 14333 raft_consensus.cc:729] T 00000000000000000000000000000000 P d7b9fc6eb32b4fb5a1e6920bf231523d [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: d7b9fc6eb32b4fb5a1e6920bf231523d, State: Initialized, Role: FOLLOWER
I20260923 22:14:44.125762 14333 consensus_queue.cc:260] T 00000000000000000000000000000000 P d7b9fc6eb32b4fb5a1e6920bf231523d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d7b9fc6eb32b4fb5a1e6920bf231523d" member_type: VOTER last_known_addr { host: "127.1.135.254" port: 37257 } }
I20260923 22:14:44.125855 14333 raft_consensus.cc:388] T 00000000000000000000000000000000 P d7b9fc6eb32b4fb5a1e6920bf231523d [term 1 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260923 22:14:44.125918 14333 raft_consensus.cc:482] T 00000000000000000000000000000000 P d7b9fc6eb32b4fb5a1e6920bf231523d [term 1 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260923 22:14:44.125974 14333 raft_consensus.cc:3037] T 00000000000000000000000000000000 P d7b9fc6eb32b4fb5a1e6920bf231523d [term 1 FOLLOWER]: Advancing to term 2
I20260923 22:14:44.126916 14333 raft_consensus.cc:504] T 00000000000000000000000000000000 P d7b9fc6eb32b4fb5a1e6920bf231523d [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d7b9fc6eb32b4fb5a1e6920bf231523d" member_type: VOTER last_known_addr { host: "127.1.135.254" port: 37257 } }
I20260923 22:14:44.127028 14333 leader_election.cc:302] T 00000000000000000000000000000000 P d7b9fc6eb32b4fb5a1e6920bf231523d [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: d7b9fc6eb32b4fb5a1e6920bf231523d; no voters:
I20260923 22:14:44.127198 14333 leader_election.cc:288] T 00000000000000000000000000000000 P d7b9fc6eb32b4fb5a1e6920bf231523d [CANDIDATE]: Term 2 election: Requested vote from peers
I20260923 22:14:44.127230 14337 raft_consensus.cc:2781] T 00000000000000000000000000000000 P d7b9fc6eb32b4fb5a1e6920bf231523d [term 2 FOLLOWER]: Leader election won for term 2
I20260923 22:14:44.127422 14337 raft_consensus.cc:686] T 00000000000000000000000000000000 P d7b9fc6eb32b4fb5a1e6920bf231523d [term 2 LEADER]: Becoming Leader. State: Replica: d7b9fc6eb32b4fb5a1e6920bf231523d, State: Running, Role: LEADER
I20260923 22:14:44.127511 14333 sys_catalog.cc:564] T 00000000000000000000000000000000 P d7b9fc6eb32b4fb5a1e6920bf231523d [sys.catalog]: configured and running, proceeding with master startup.
I20260923 22:14:44.127554 14337 consensus_queue.cc:237] T 00000000000000000000000000000000 P d7b9fc6eb32b4fb5a1e6920bf231523d [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 2, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d7b9fc6eb32b4fb5a1e6920bf231523d" member_type: VOTER last_known_addr { host: "127.1.135.254" port: 37257 } }
I20260923 22:14:44.127871 14337 sys_catalog.cc:455] T 00000000000000000000000000000000 P d7b9fc6eb32b4fb5a1e6920bf231523d [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 2 leader_uuid: "d7b9fc6eb32b4fb5a1e6920bf231523d" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d7b9fc6eb32b4fb5a1e6920bf231523d" member_type: VOTER last_known_addr { host: "127.1.135.254" port: 37257 } } }
I20260923 22:14:44.128002 14337 sys_catalog.cc:458] T 00000000000000000000000000000000 P d7b9fc6eb32b4fb5a1e6920bf231523d [sys.catalog]: This master's current role is: LEADER
I20260923 22:14:44.128196 14337 sys_catalog.cc:455] T 00000000000000000000000000000000 P d7b9fc6eb32b4fb5a1e6920bf231523d [sys.catalog]: SysCatalogTable state changed. Reason: New leader d7b9fc6eb32b4fb5a1e6920bf231523d. Latest consensus state: current_term: 2 leader_uuid: "d7b9fc6eb32b4fb5a1e6920bf231523d" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d7b9fc6eb32b4fb5a1e6920bf231523d" member_type: VOTER last_known_addr { host: "127.1.135.254" port: 37257 } } }
I20260923 22:14:44.128307 14337 sys_catalog.cc:458] T 00000000000000000000000000000000 P d7b9fc6eb32b4fb5a1e6920bf231523d [sys.catalog]: This master's current role is: LEADER
I20260923 22:14:44.128927 14333 ranger_client.cc:318] Using overwritten properties file: /tmp/dist-test-taskRiQe_a/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790201539513534-1567-0/minicluster-data/master-0/logs/kudu-ranger-subprocess-log4j2.properties
I20260923 22:14:44.131435 14345 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260923 22:14:44.132268 14345 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260923 22:14:44.132493 14345 catalog_manager.cc:1259] Loaded cluster ID: e3b6d4c0251f47b88a23664e5aa2d502
I20260923 22:14:44.132581 14345 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260923 22:14:44.133764 14345 catalog_manager.cc:1504] Loading token signing keys...
I20260923 22:14:44.133983 14345 catalog_manager.cc:5879] T 00000000000000000000000000000000 P d7b9fc6eb32b4fb5a1e6920bf231523d: Loaded TSK: 0
I20260923 22:14:44.134382 14345 catalog_manager.cc:1514] Initializing in-progress tserver states...
W20260923 22:14:44.229599 14336 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260923 22:14:44.229810 14336 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260923 22:14:44.229849 14336 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260923 22:14:44.230976 14336 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260923 22:14:44.231051 14336 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260923 22:14:44.231096 14336 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.1.135.193
I20260923 22:14:44.233145 14336 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.1.135.212:45915
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-taskRiQe_a/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790201539513534-1567-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskRiQe_a/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790201539513534-1567-0/minicluster-data/ts-0/wal
--ranger_kms_url=invalid.host:1234/kms,127.1.135.212:41391/kms
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskRiQe_a/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790201539513534-1567-0/krb5kdc/kudu_127.1.135.193.keytab
--principal=kudu/127.1.135.193
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.1.135.193:34647
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskRiQe_a/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790201539513534-1567-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.1.135.193
--webserver_port=34343
--tserver_master_addrs=127.1.135.254:37257
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.14336
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.1.135.193
--log_dir=/tmp/dist-test-taskRiQe_a/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790201539513534-1567-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 23 Sep 2026 20:53:37 UTC on ccycloud.quasar-rkviyx.root.comops.site
build id 4668083
I20260923 22:14:44.233587 14336 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260923 22:14:44.233870 14336 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260923 22:14:44.239977 14371 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260923 22:14:44.240046 14370 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260923 22:14:44.240242 14373 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260923 22:14:44.240490 14336 server_base.cc:1034] running on GCE node
I20260923 22:14:44.240713 14336 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260923 22:14:44.240998 14336 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260923 22:14:44.242154 14336 hybrid_clock.cc:648] HybridClock initialized: now 1790201684242144 us; error 28 us; skew 500 ppm
Sep 23 22:14:44 dist-test-slave-s2fj krb5kdc[13301](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790201684, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.135.193@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260923 22:14:44.244704 14336 init.cc:377] Logged in from keytab as kudu/127.1.135.193@KRBTEST.COM (short username kudu)
I20260923 22:14:44.245190 14336 webserver.cc:460] Webserver started at http://127.1.135.193:34343/ using document root <none> and password file <none>
I20260923 22:14:44.245391 14336 fs_manager.cc:359] Metadata directory not provided
I20260923 22:14:44.245453 14336 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
Sep 23 22:14:44 dist-test-slave-s2fj krb5kdc[13301](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790201684, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.135.193@KRBTEST.COM for HTTP/127.1.135.212@KRBTEST.COM
I20260923 22:14:44.269712 14336 fs_manager.cc:709] Time spent opening directory manager: real 0.004s user 0.001s sys 0.000s
I20260923 22:14:44.275317 14381 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260923 22:14:44.275511 14336 fs_manager.cc:725] Time spent opening block manager: real 0.005s user 0.000s sys 0.000s
I20260923 22:14:44.275580 14336 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskRiQe_a/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790201539513534-1567-0/minicluster-data/ts-0/data,/tmp/dist-test-taskRiQe_a/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790201539513534-1567-0/minicluster-data/ts-0/wal
uuid: "0ff4f9455d6444269b402aa13dc889e6"
format_stamp: "Formatted at 2026-09-23 22:14:43 on dist-test-slave-s2fj"
server_key: "3efb6651219b1fef04ac8b8f3b62b42c"
server_key_iv: "bb8382eae421842cc683d782ca3b66d5"
server_key_version: "kuduclusterkey@0"
I20260923 22:14:44.275995 14336 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskRiQe_a/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790201539513534-1567-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskRiQe_a/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790201539513534-1567-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskRiQe_a/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790201539513534-1567-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260923 22:14:44.306450 14336 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260923 22:14:44.308712 14336 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260923 22:14:44.308841 14336 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260923 22:14:44.309038 14336 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260923 22:14:44.309334 14336 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260923 22:14:44.309374 14336 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260923 22:14:44.309408 14336 ts_tablet_manager.cc:610] Registered 0 tablets
I20260923 22:14:44.309505 14336 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260923 22:14:44.316336 14336 rpc_server.cc:307] RPC server started. Bound to: 127.1.135.193:34647
I20260923 22:14:44.316924 14336 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskRiQe_a/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790201539513534-1567-0/minicluster-data/ts-0/data/info.pb
I20260923 22:14:44.324393 14494 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.1.135.193:34647 every 8 connection(s)
I20260923 22:14:44.326958 1567 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskRiQe_a/build/release/bin/kudu as pid 14336
I20260923 22:14:44.327256 1567 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskRiQe_a/build/release/bin/kudu
/tmp/dist-test-taskRiQe_a/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskRiQe_a/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790201539513534-1567-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskRiQe_a/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790201539513534-1567-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskRiQe_a/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790201539513534-1567-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskRiQe_a/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790201539513534-1567-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.1.135.212:41391/kms
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.1.135.194:39053
--local_ip_for_outbound_sockets=127.1.135.194
--tserver_master_addrs=127.1.135.254:37257
--webserver_port=41919
--webserver_interface=127.1.135.194
--keytab_file=/tmp/dist-test-taskRiQe_a/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790201539513534-1567-0/krb5kdc/kudu_127.1.135.194.keytab
--principal=kudu/127.1.135.194
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.1.135.212:45915
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--ranger_kms_url=invalid.host:1234/kms,127.1.135.212:41391/kms with env {KRB5CCNAME=/tmp/dist-test-taskRiQe_a/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790201539513534-1567-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskRiQe_a/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790201539513534-1567-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskRiQe_a/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790201539513534-1567-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
Sep 23 22:14:44 dist-test-slave-s2fj krb5kdc[13301](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790201684, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.135.193@KRBTEST.COM for kudu/127.1.135.254@KRBTEST.COM
I20260923 22:14:44.338661 14498 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 22:14:44.325001 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.135.193:46273 (local address 127.1.135.254:37257)
0923 22:14:44.326828 (+ 1827us) server_negotiation.cc:206] Beginning negotiation
0923 22:14:44.326838 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0923 22:14:44.327774 (+ 936us) server_negotiation.cc:407] Connection header received
0923 22:14:44.327901 (+ 127us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 22:14:44.327906 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 22:14:44.327954 (+ 48us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 22:14:44.328053 (+ 99us) server_negotiation.cc:226] Negotiated authn=SASL
0923 22:14:44.328807 (+ 754us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 22:14:44.330094 (+ 1287us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 22:14:44.331555 (+ 1461us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 22:14:44.332144 (+ 589us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 22:14:44.332253 (+ 109us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 22:14:44.334738 (+ 2485us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0923 22:14:44.334742 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0923 22:14:44.334742 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0923 22:14:44.334753 (+ 11us) server_negotiation.cc:922] calling sasl_server_start()
0923 22:14:44.336691 (+ 1938us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0923 22:14:44.337215 (+ 524us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0923 22:14:44.337218 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0923 22:14:44.337219 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0923 22:14:44.337320 (+ 101us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0923 22:14:44.337606 (+ 286us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0923 22:14:44.337607 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0923 22:14:44.337607 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0923 22:14:44.337887 (+ 280us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0923 22:14:44.337946 (+ 59us) server_negotiation.cc:1035] Waiting for connection context
0923 22:14:44.338351 (+ 405us) server_negotiation.cc:299] Negotiation successful
0923 22:14:44.338468 (+ 117us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":1765,"thread_start_us":78,"threads_started":1}
I20260923 22:14:44.339670 14497 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 22:14:44.324964 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.135.254:37257 (local address 127.1.135.193:46273)
0923 22:14:44.326917 (+ 1953us) negotiation.cc:107] Waiting for socket to connect
0923 22:14:44.326960 (+ 43us) client_negotiation.cc:174] Beginning negotiation
0923 22:14:44.327635 (+ 675us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0923 22:14:44.328217 (+ 582us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0923 22:14:44.328220 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0923 22:14:44.328418 (+ 198us) client_negotiation.cc:189] Negotiated authn=SASL
0923 22:14:44.328675 (+ 257us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 22:14:44.328682 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 22:14:44.330208 (+ 1526us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 22:14:44.330209 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 22:14:44.331436 (+ 1227us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 22:14:44.331439 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 22:14:44.332370 (+ 931us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 22:14:44.332372 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 22:14:44.332494 (+ 122us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 22:14:44.333094 (+ 600us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0923 22:14:44.333101 (+ 7us) client_negotiation.cc:656] calling sasl_client_start()
0923 22:14:44.334623 (+ 1522us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0923 22:14:44.336786 (+ 2163us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0923 22:14:44.336792 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0923 22:14:44.336795 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0923 22:14:44.337095 (+ 300us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0923 22:14:44.337397 (+ 302us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0923 22:14:44.337398 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0923 22:14:44.337398 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0923 22:14:44.337531 (+ 133us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0923 22:14:44.337997 (+ 466us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0923 22:14:44.338002 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0923 22:14:44.338147 (+ 145us) client_negotiation.cc:769] Sending connection context
0923 22:14:44.339274 (+ 1127us) client_negotiation.cc:240] Negotiation successful
0923 22:14:44.339455 (+ 181us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1914,"thread_start_us":104,"threads_started":1}
I20260923 22:14:44.340670 14495 heartbeater.cc:347] Connected to a master server at 127.1.135.254:37257
I20260923 22:14:44.340770 14495 heartbeater.cc:464] Registering TS with master...
I20260923 22:14:44.340960 14495 heartbeater.cc:511] Master 127.1.135.254:37257 requested a full tablet report, sending...
W20260923 22:14:44.341780 14495 heartbeater.cc:644] Failed to heartbeat to 127.1.135.254:37257 (0 consecutive failures): Service unavailable: Catalog manager is not initialized. State: Starting
I20260923 22:14:44.341842 14495 heartbeater.cc:464] Registering TS with master...
I20260923 22:14:44.341897 14495 heartbeater.cc:511] Master 127.1.135.254:37257 requested a full tablet report, sending...
I20260923 22:14:44.342346 14495 heartbeater.cc:464] Registering TS with master...
I20260923 22:14:44.342414 14495 heartbeater.cc:511] Master 127.1.135.254:37257 requested a full tablet report, sending...
W20260923 22:14:44.342731 14495 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
W20260923 22:14:44.445442 14499 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260923 22:14:44.445684 14499 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260923 22:14:44.445719 14499 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260923 22:14:44.446882 14499 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260923 22:14:44.446987 14499 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260923 22:14:44.447048 14499 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.1.135.194
I20260923 22:14:44.448590 14499 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.1.135.212:45915
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-taskRiQe_a/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790201539513534-1567-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskRiQe_a/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790201539513534-1567-0/minicluster-data/ts-1/wal
--ranger_kms_url=invalid.host:1234/kms,127.1.135.212:41391/kms
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskRiQe_a/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790201539513534-1567-0/krb5kdc/kudu_127.1.135.194.keytab
--principal=kudu/127.1.135.194
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.1.135.194:39053
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskRiQe_a/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790201539513534-1567-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.1.135.194
--webserver_port=41919
--tserver_master_addrs=127.1.135.254:37257
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.14499
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.1.135.194
--log_dir=/tmp/dist-test-taskRiQe_a/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790201539513534-1567-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 23 Sep 2026 20:53:37 UTC on ccycloud.quasar-rkviyx.root.comops.site
build id 4668083
I20260923 22:14:44.448853 14499 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260923 22:14:44.449061 14499 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260923 22:14:44.453944 14507 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260923 22:14:44.453955 14505 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260923 22:14:44.454080 14504 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260923 22:14:44.454579 14499 server_base.cc:1034] running on GCE node
I20260923 22:14:44.454799 14499 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260923 22:14:44.455025 14499 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260923 22:14:44.456147 14499 hybrid_clock.cc:648] HybridClock initialized: now 1790201684456128 us; error 36 us; skew 500 ppm
Sep 23 22:14:44 dist-test-slave-s2fj krb5kdc[13301](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790201684, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.135.194@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260923 22:14:44.459144 14499 init.cc:377] Logged in from keytab as kudu/127.1.135.194@KRBTEST.COM (short username kudu)
I20260923 22:14:44.459707 14499 webserver.cc:460] Webserver started at http://127.1.135.194:41919/ using document root <none> and password file <none>
I20260923 22:14:44.459959 14499 fs_manager.cc:359] Metadata directory not provided
I20260923 22:14:44.460103 14499 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
Sep 23 22:14:44 dist-test-slave-s2fj krb5kdc[13301](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790201684, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.135.194@KRBTEST.COM for HTTP/127.1.135.212@KRBTEST.COM
I20260923 22:14:44.480242 14499 fs_manager.cc:709] Time spent opening directory manager: real 0.005s user 0.001s sys 0.000s
I20260923 22:14:44.481278 14515 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260923 22:14:44.481465 14499 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.000s
I20260923 22:14:44.481549 14499 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskRiQe_a/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790201539513534-1567-0/minicluster-data/ts-1/data,/tmp/dist-test-taskRiQe_a/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790201539513534-1567-0/minicluster-data/ts-1/wal
uuid: "ff2dd7276bc647e3b2cee0d9ec03c525"
format_stamp: "Formatted at 2026-09-23 22:14:43 on dist-test-slave-s2fj"
server_key: "8cd22ac1702477cb05bd74b66b822202"
server_key_iv: "b80e508194ee99ce453e173f2b4cdba9"
server_key_version: "kuduclusterkey@0"
I20260923 22:14:44.481992 14499 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskRiQe_a/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790201539513534-1567-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskRiQe_a/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790201539513534-1567-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskRiQe_a/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790201539513534-1567-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260923 22:14:44.495972 14499 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260923 22:14:44.498176 14499 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260923 22:14:44.498332 14499 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260923 22:14:44.498562 14499 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260923 22:14:44.498889 14499 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260923 22:14:44.498937 14499 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260923 22:14:44.498997 14499 ts_tablet_manager.cc:610] Registered 0 tablets
I20260923 22:14:44.499023 14499 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260923 22:14:44.505928 14499 rpc_server.cc:307] RPC server started. Bound to: 127.1.135.194:39053
I20260923 22:14:44.506016 14628 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.1.135.194:39053 every 8 connection(s)
I20260923 22:14:44.506330 14499 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskRiQe_a/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790201539513534-1567-0/minicluster-data/ts-1/data/info.pb
Sep 23 22:14:44 dist-test-slave-s2fj krb5kdc[13301](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790201684, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.135.194@KRBTEST.COM for kudu/127.1.135.254@KRBTEST.COM
I20260923 22:14:44.515594 14498 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 22:14:44.507420 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.135.194:47105 (local address 127.1.135.254:37257)
0923 22:14:44.507490 (+ 70us) server_negotiation.cc:206] Beginning negotiation
0923 22:14:44.507493 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0923 22:14:44.507729 (+ 236us) server_negotiation.cc:407] Connection header received
0923 22:14:44.508385 (+ 656us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 22:14:44.508386 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 22:14:44.508401 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 22:14:44.508447 (+ 46us) server_negotiation.cc:226] Negotiated authn=SASL
0923 22:14:44.508976 (+ 529us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 22:14:44.509477 (+ 501us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 22:14:44.510450 (+ 973us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 22:14:44.510687 (+ 237us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 22:14:44.510753 (+ 66us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 22:14:44.512629 (+ 1876us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0923 22:14:44.512630 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0923 22:14:44.512630 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0923 22:14:44.512638 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0923 22:14:44.514186 (+ 1548us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0923 22:14:44.514593 (+ 407us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0923 22:14:44.514595 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0923 22:14:44.514595 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0923 22:14:44.514662 (+ 67us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0923 22:14:44.514976 (+ 314us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0923 22:14:44.514977 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0923 22:14:44.514978 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0923 22:14:44.515092 (+ 114us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0923 22:14:44.515169 (+ 77us) server_negotiation.cc:1035] Waiting for connection context
0923 22:14:44.515422 (+ 253us) server_negotiation.cc:299] Negotiation successful
0923 22:14:44.515506 (+ 84us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":41}
I20260923 22:14:44.515787 14631 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 22:14:44.507109 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.135.254:37257 (local address 127.1.135.194:47105)
0923 22:14:44.507599 (+ 490us) negotiation.cc:107] Waiting for socket to connect
0923 22:14:44.507637 (+ 38us) client_negotiation.cc:174] Beginning negotiation
0923 22:14:44.508271 (+ 634us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0923 22:14:44.508526 (+ 255us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0923 22:14:44.508529 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0923 22:14:44.508698 (+ 169us) client_negotiation.cc:189] Negotiated authn=SASL
0923 22:14:44.508886 (+ 188us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 22:14:44.508890 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 22:14:44.509569 (+ 679us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 22:14:44.509570 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 22:14:44.510344 (+ 774us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 22:14:44.510346 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 22:14:44.510789 (+ 443us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 22:14:44.510791 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 22:14:44.510903 (+ 112us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 22:14:44.511345 (+ 442us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0923 22:14:44.511352 (+ 7us) client_negotiation.cc:656] calling sasl_client_start()
0923 22:14:44.512539 (+ 1187us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0923 22:14:44.514268 (+ 1729us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0923 22:14:44.514273 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0923 22:14:44.514275 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0923 22:14:44.514474 (+ 199us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0923 22:14:44.514764 (+ 290us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0923 22:14:44.514765 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0923 22:14:44.514765 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0923 22:14:44.514873 (+ 108us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0923 22:14:44.515165 (+ 292us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0923 22:14:44.515171 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0923 22:14:44.515293 (+ 122us) client_negotiation.cc:769] Sending connection context
0923 22:14:44.515402 (+ 109us) client_negotiation.cc:240] Negotiation successful
0923 22:14:44.515535 (+ 133us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":426,"thread_start_us":111,"threads_started":1}
I20260923 22:14:44.515973 1567 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskRiQe_a/build/release/bin/kudu as pid 14499
I20260923 22:14:44.516319 14629 heartbeater.cc:347] Connected to a master server at 127.1.135.254:37257
I20260923 22:14:44.516333 1567 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskRiQe_a/build/release/bin/kudu
/tmp/dist-test-taskRiQe_a/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskRiQe_a/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790201539513534-1567-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskRiQe_a/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790201539513534-1567-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskRiQe_a/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790201539513534-1567-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskRiQe_a/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790201539513534-1567-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.1.135.212:41391/kms
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.1.135.195:43765
--local_ip_for_outbound_sockets=127.1.135.195
--tserver_master_addrs=127.1.135.254:37257
--webserver_port=42429
--webserver_interface=127.1.135.195
--keytab_file=/tmp/dist-test-taskRiQe_a/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790201539513534-1567-0/krb5kdc/kudu_127.1.135.195.keytab
--principal=kudu/127.1.135.195
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.1.135.212:45915
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--ranger_kms_url=invalid.host:1234/kms,127.1.135.212:41391/kms with env {KRB5CCNAME=/tmp/dist-test-taskRiQe_a/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790201539513534-1567-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskRiQe_a/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790201539513534-1567-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskRiQe_a/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790201539513534-1567-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260923 22:14:44.516451 14629 heartbeater.cc:464] Registering TS with master...
I20260923 22:14:44.516630 14629 heartbeater.cc:511] Master 127.1.135.254:37257 requested a full tablet report, sending...
W20260923 22:14:44.517226 14629 heartbeater.cc:644] Failed to heartbeat to 127.1.135.254:37257 (0 consecutive failures): Service unavailable: Catalog manager is not initialized. State: Starting
I20260923 22:14:44.517303 14629 heartbeater.cc:464] Registering TS with master...
I20260923 22:14:44.517369 14629 heartbeater.cc:511] Master 127.1.135.254:37257 requested a full tablet report, sending...
I20260923 22:14:44.517663 14629 heartbeater.cc:464] Registering TS with master...
I20260923 22:14:44.517729 14629 heartbeater.cc:511] Master 127.1.135.254:37257 requested a full tablet report, sending...
W20260923 22:14:44.518034 14629 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
W20260923 22:14:44.628360 14632 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260923 22:14:44.628575 14632 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260923 22:14:44.628610 14632 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260923 22:14:44.629956 14632 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260923 22:14:44.630020 14632 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260923 22:14:44.630065 14632 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.1.135.195
I20260923 22:14:44.632006 14632 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.1.135.212:45915
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-taskRiQe_a/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790201539513534-1567-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskRiQe_a/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790201539513534-1567-0/minicluster-data/ts-2/wal
--ranger_kms_url=invalid.host:1234/kms,127.1.135.212:41391/kms
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskRiQe_a/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790201539513534-1567-0/krb5kdc/kudu_127.1.135.195.keytab
--principal=kudu/127.1.135.195
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.1.135.195:43765
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskRiQe_a/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790201539513534-1567-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.1.135.195
--webserver_port=42429
--tserver_master_addrs=127.1.135.254:37257
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.14632
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.1.135.195
--log_dir=/tmp/dist-test-taskRiQe_a/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790201539513534-1567-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 23 Sep 2026 20:53:37 UTC on ccycloud.quasar-rkviyx.root.comops.site
build id 4668083
I20260923 22:14:44.632282 14632 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260923 22:14:44.632520 14632 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260923 22:14:44.637549 14641 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260923 22:14:44.637522 14639 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260923 22:14:44.637960 14632 server_base.cc:1034] running on GCE node
W20260923 22:14:44.638119 14638 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260923 22:14:44.638345 14632 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260923 22:14:44.638592 14632 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260923 22:14:44.639761 14632 hybrid_clock.cc:648] HybridClock initialized: now 1790201684639748 us; error 32 us; skew 500 ppm
Sep 23 22:14:44 dist-test-slave-s2fj krb5kdc[13301](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790201684, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.135.195@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260923 22:14:44.642328 14632 init.cc:377] Logged in from keytab as kudu/127.1.135.195@KRBTEST.COM (short username kudu)
I20260923 22:14:44.642794 14632 webserver.cc:460] Webserver started at http://127.1.135.195:42429/ using document root <none> and password file <none>
I20260923 22:14:44.643054 14632 fs_manager.cc:359] Metadata directory not provided
I20260923 22:14:44.643134 14632 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
Sep 23 22:14:44 dist-test-slave-s2fj krb5kdc[13301](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790201684, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.135.195@KRBTEST.COM for HTTP/127.1.135.212@KRBTEST.COM
I20260923 22:14:44.660418 14632 fs_manager.cc:709] Time spent opening directory manager: real 0.004s user 0.001s sys 0.000s
I20260923 22:14:44.661252 14649 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260923 22:14:44.661442 14632 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.000s
I20260923 22:14:44.661604 14632 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskRiQe_a/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790201539513534-1567-0/minicluster-data/ts-2/data,/tmp/dist-test-taskRiQe_a/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790201539513534-1567-0/minicluster-data/ts-2/wal
uuid: "226dc984ad3b4589883715494ff740a4"
format_stamp: "Formatted at 2026-09-23 22:14:43 on dist-test-slave-s2fj"
server_key: "86c506ca04f9bc65cb455358c82082d2"
server_key_iv: "1bbbde8b89707045af175e5b3f630045"
server_key_version: "kuduclusterkey@0"
I20260923 22:14:44.661909 14632 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskRiQe_a/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790201539513534-1567-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskRiQe_a/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790201539513534-1567-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskRiQe_a/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790201539513534-1567-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260923 22:14:44.685410 14632 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260923 22:14:44.687915 14632 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260923 22:14:44.688144 14632 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260923 22:14:44.688439 14632 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260923 22:14:44.688838 14632 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260923 22:14:44.688923 14632 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260923 22:14:44.688997 14632 ts_tablet_manager.cc:610] Registered 0 tablets
I20260923 22:14:44.689054 14632 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260923 22:14:44.695267 14632 rpc_server.cc:307] RPC server started. Bound to: 127.1.135.195:43765
I20260923 22:14:44.695422 14762 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.1.135.195:43765 every 8 connection(s)
I20260923 22:14:44.695901 14632 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskRiQe_a/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790201539513534-1567-0/minicluster-data/ts-2/data/info.pb
I20260923 22:14:44.697715 1567 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskRiQe_a/build/release/bin/kudu as pid 14632
Sep 23 22:14:44 dist-test-slave-s2fj krb5kdc[13301](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790201684, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.135.195@KRBTEST.COM for kudu/127.1.135.254@KRBTEST.COM
I20260923 22:14:44.705986 14498 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 22:14:44.696627 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.135.195:40139 (local address 127.1.135.254:37257)
0923 22:14:44.696708 (+ 81us) server_negotiation.cc:206] Beginning negotiation
0923 22:14:44.696711 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0923 22:14:44.696917 (+ 206us) server_negotiation.cc:407] Connection header received
0923 22:14:44.697575 (+ 658us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 22:14:44.697576 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 22:14:44.697588 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 22:14:44.697654 (+ 66us) server_negotiation.cc:226] Negotiated authn=SASL
0923 22:14:44.698550 (+ 896us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 22:14:44.699293 (+ 743us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 22:14:44.700813 (+ 1520us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 22:14:44.701080 (+ 267us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 22:14:44.701161 (+ 81us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 22:14:44.703115 (+ 1954us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0923 22:14:44.703116 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0923 22:14:44.703117 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0923 22:14:44.703123 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0923 22:14:44.704578 (+ 1455us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0923 22:14:44.705001 (+ 423us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0923 22:14:44.705003 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0923 22:14:44.705004 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0923 22:14:44.705067 (+ 63us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0923 22:14:44.705345 (+ 278us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0923 22:14:44.705345 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0923 22:14:44.705346 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0923 22:14:44.705473 (+ 127us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0923 22:14:44.705542 (+ 69us) server_negotiation.cc:1035] Waiting for connection context
0923 22:14:44.705827 (+ 285us) server_negotiation.cc:299] Negotiation successful
0923 22:14:44.705901 (+ 74us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":51}
I20260923 22:14:44.706426 14765 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 22:14:44.696592 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.135.254:37257 (local address 127.1.135.195:40139)
0923 22:14:44.696754 (+ 162us) negotiation.cc:107] Waiting for socket to connect
0923 22:14:44.696815 (+ 61us) client_negotiation.cc:174] Beginning negotiation
0923 22:14:44.697460 (+ 645us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0923 22:14:44.697733 (+ 273us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0923 22:14:44.697738 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0923 22:14:44.697962 (+ 224us) client_negotiation.cc:189] Negotiated authn=SASL
0923 22:14:44.698406 (+ 444us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 22:14:44.698430 (+ 24us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 22:14:44.699569 (+ 1139us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 22:14:44.699570 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 22:14:44.700712 (+ 1142us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 22:14:44.700714 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 22:14:44.701187 (+ 473us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 22:14:44.701189 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 22:14:44.701302 (+ 113us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 22:14:44.701707 (+ 405us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0923 22:14:44.701714 (+ 7us) client_negotiation.cc:656] calling sasl_client_start()
0923 22:14:44.703001 (+ 1287us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0923 22:14:44.704655 (+ 1654us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0923 22:14:44.704661 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0923 22:14:44.704663 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0923 22:14:44.704905 (+ 242us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0923 22:14:44.705139 (+ 234us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0923 22:14:44.705140 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0923 22:14:44.705140 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0923 22:14:44.705272 (+ 132us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0923 22:14:44.705549 (+ 277us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0923 22:14:44.705555 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0923 22:14:44.705682 (+ 127us) client_negotiation.cc:769] Sending connection context
0923 22:14:44.706088 (+ 406us) client_negotiation.cc:240] Negotiation successful
0923 22:14:44.706225 (+ 137us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":121,"thread_start_us":90,"threads_started":1}
I20260923 22:14:44.707029 14763 heartbeater.cc:347] Connected to a master server at 127.1.135.254:37257
I20260923 22:14:44.707123 14763 heartbeater.cc:464] Registering TS with master...
I20260923 22:14:44.707306 14763 heartbeater.cc:511] Master 127.1.135.254:37257 requested a full tablet report, sending...
W20260923 22:14:44.707926 14763 heartbeater.cc:644] Failed to heartbeat to 127.1.135.254:37257 (0 consecutive failures): Service unavailable: Catalog manager is not initialized. State: Starting
I20260923 22:14:44.707996 14763 heartbeater.cc:464] Registering TS with master...
I20260923 22:14:44.708055 14763 heartbeater.cc:511] Master 127.1.135.254:37257 requested a full tablet report, sending...
I20260923 22:14:44.708530 14763 heartbeater.cc:464] Registering TS with master...
I20260923 22:14:44.708601 14763 heartbeater.cc:511] Master 127.1.135.254:37257 requested a full tablet report, sending...
W20260923 22:14:44.708928 14763 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
I20260923 22:14:44.711473 14767 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 22:14:44.698106 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33912 (local address 127.1.135.254:37257)
0923 22:14:44.700854 (+ 2748us) server_negotiation.cc:206] Beginning negotiation
0923 22:14:44.700856 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0923 22:14:44.700861 (+ 5us) server_negotiation.cc:407] Connection header received
0923 22:14:44.700875 (+ 14us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 22:14:44.700875 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 22:14:44.700885 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 22:14:44.700935 (+ 50us) server_negotiation.cc:226] Negotiated authn=SASL
0923 22:14:44.704850 (+ 3915us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 22:14:44.705492 (+ 642us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 22:14:44.706772 (+ 1280us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 22:14:44.707130 (+ 358us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 22:14:44.707190 (+ 60us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 22:14:44.708928 (+ 1738us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0923 22:14:44.708929 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0923 22:14:44.708929 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0923 22:14:44.708936 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0923 22:14:44.710335 (+ 1399us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0923 22:14:44.710717 (+ 382us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0923 22:14:44.710718 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0923 22:14:44.710719 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0923 22:14:44.710774 (+ 55us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0923 22:14:44.710972 (+ 198us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0923 22:14:44.710973 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0923 22:14:44.710973 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0923 22:14:44.711095 (+ 122us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0923 22:14:44.711268 (+ 173us) server_negotiation.cc:1035] Waiting for connection context
0923 22:14:44.711302 (+ 34us) server_negotiation.cc:299] Negotiation successful
0923 22:14:44.711374 (+ 72us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":2731,"thread_start_us":63,"threads_started":1}
Sep 23 22:14:45 dist-test-slave-s2fj krb5kdc[13301](info): AS_REQ (1 etypes {17}) 127.0.0.1: ISSUE: authtime 1790201685, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.135.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260923 22:14:45.342916 14495 heartbeater.cc:464] Registering TS with master...
I20260923 22:14:45.343070 14495 heartbeater.cc:511] Master 127.1.135.254:37257 requested a full tablet report, sending...
I20260923 22:14:45.343479 14297 ts_manager.cc:194] Registered new tserver with Master: 0ff4f9455d6444269b402aa13dc889e6 (127.1.135.193:34647)
I20260923 22:14:45.344548 14297 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.1.135.193@KRBTEST.COM'} at 127.1.135.193:46273
Sep 23 22:14:45 dist-test-slave-s2fj krb5kdc[13301](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790201685, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.135.254@KRBTEST.COM for HTTP/127.1.135.212@KRBTEST.COM
I20260923 22:14:45.518180 14629 heartbeater.cc:464] Registering TS with master...
I20260923 22:14:45.518333 14629 heartbeater.cc:511] Master 127.1.135.254:37257 requested a full tablet report, sending...
I20260923 22:14:45.522744 14297 ts_manager.cc:194] Registered new tserver with Master: ff2dd7276bc647e3b2cee0d9ec03c525 (127.1.135.194:39053)
I20260923 22:14:45.523298 14297 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.1.135.194@KRBTEST.COM'} at 127.1.135.194:47105
I20260923 22:14:45.709097 14763 heartbeater.cc:464] Registering TS with master...
I20260923 22:14:45.709239 14763 heartbeater.cc:511] Master 127.1.135.254:37257 requested a full tablet report, sending...
I20260923 22:14:45.709513 14297 ts_manager.cc:194] Registered new tserver with Master: 226dc984ad3b4589883715494ff740a4 (127.1.135.195:43765)
I20260923 22:14:45.710026 14297 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.1.135.195@KRBTEST.COM'} at 127.1.135.195:40139
I20260923 22:14:45.710793 1567 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260923 22:14:45.719667 14784 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 22:14:45.711667 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33922 (local address 127.1.135.254:37257)
0923 22:14:45.711923 (+ 256us) server_negotiation.cc:206] Beginning negotiation
0923 22:14:45.711924 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0923 22:14:45.712092 (+ 168us) server_negotiation.cc:407] Connection header received
0923 22:14:45.712119 (+ 27us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 22:14:45.712120 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 22:14:45.712133 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 22:14:45.712197 (+ 64us) server_negotiation.cc:226] Negotiated authn=SASL
0923 22:14:45.712913 (+ 716us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 22:14:45.713668 (+ 755us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 22:14:45.714761 (+ 1093us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 22:14:45.715071 (+ 310us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 22:14:45.715137 (+ 66us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 22:14:45.716661 (+ 1524us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0923 22:14:45.716663 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0923 22:14:45.716663 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0923 22:14:45.716671 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0923 22:14:45.718405 (+ 1734us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0923 22:14:45.718800 (+ 395us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0923 22:14:45.718801 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0923 22:14:45.718802 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0923 22:14:45.718865 (+ 63us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0923 22:14:45.719085 (+ 220us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0923 22:14:45.719086 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0923 22:14:45.719086 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0923 22:14:45.719224 (+ 138us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0923 22:14:45.719328 (+ 104us) server_negotiation.cc:1035] Waiting for connection context
0923 22:14:45.719469 (+ 141us) server_negotiation.cc:299] Negotiation successful
0923 22:14:45.719553 (+ 84us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":234,"thread_start_us":61,"threads_started":1}
I20260923 22:14:45.721731 14297 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:33922:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260923 22:14:45.724043 14297 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260923 22:14:45.732306 14791 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 22:14:45.727604 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41694 (local address 127.1.135.194:39053)
0923 22:14:45.727763 (+ 159us) server_negotiation.cc:206] Beginning negotiation
0923 22:14:45.727768 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0923 22:14:45.728447 (+ 679us) server_negotiation.cc:407] Connection header received
0923 22:14:45.728539 (+ 92us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 22:14:45.728542 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 22:14:45.728557 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 22:14:45.728636 (+ 79us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0923 22:14:45.728783 (+ 147us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 22:14:45.729762 (+ 979us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 22:14:45.731145 (+ 1383us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 22:14:45.731834 (+ 689us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 22:14:45.731877 (+ 43us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 22:14:45.731928 (+ 51us) server_negotiation.cc:1035] Waiting for connection context
0923 22:14:45.732197 (+ 269us) server_negotiation.cc:299] Negotiation successful
0923 22:14:45.732236 (+ 39us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":132,"thread_start_us":71,"threads_started":1}
I20260923 22:14:45.732697 14794 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 22:14:45.727511 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.135.194:39053 (local address 127.0.0.1:41694)
0923 22:14:45.728300 (+ 789us) negotiation.cc:107] Waiting for socket to connect
0923 22:14:45.728372 (+ 72us) client_negotiation.cc:174] Beginning negotiation
0923 22:14:45.728442 (+ 70us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0923 22:14:45.728668 (+ 226us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0923 22:14:45.728671 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0923 22:14:45.728672 (+ 1us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0923 22:14:45.728729 (+ 57us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 22:14:45.728731 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 22:14:45.729862 (+ 1131us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 22:14:45.729863 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 22:14:45.731052 (+ 1189us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 22:14:45.731055 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 22:14:45.731985 (+ 930us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 22:14:45.731986 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 22:14:45.732085 (+ 99us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 22:14:45.732099 (+ 14us) client_negotiation.cc:769] Sending connection context
0923 22:14:45.732575 (+ 476us) client_negotiation.cc:240] Negotiation successful
0923 22:14:45.732633 (+ 58us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":508,"thread_start_us":73,"threads_started":1}
I20260923 22:14:45.733587 14563 tablet_service.cc:1467] Processing CreateTablet for tablet 3465c9c0b91141b885540c8a1bd526de (DEFAULT_TABLE table=test-table [id=0b389b74dc0a45a1bd0841bb939615e1]), partition=RANGE (key) PARTITION UNBOUNDED
I20260923 22:14:45.733942 14563 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 3465c9c0b91141b885540c8a1bd526de. 1 dirs total, 0 dirs full, 0 dirs failed
I20260923 22:14:45.734869 14792 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 22:14:45.727899 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.135.195:43765 (local address 127.0.0.1:57472)
0923 22:14:45.728808 (+ 909us) negotiation.cc:107] Waiting for socket to connect
0923 22:14:45.728816 (+ 8us) client_negotiation.cc:174] Beginning negotiation
0923 22:14:45.729077 (+ 261us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0923 22:14:45.729333 (+ 256us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0923 22:14:45.729334 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0923 22:14:45.729336 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0923 22:14:45.729380 (+ 44us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 22:14:45.729382 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 22:14:45.731397 (+ 2015us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 22:14:45.731398 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 22:14:45.733654 (+ 2256us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 22:14:45.733657 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 22:14:45.734657 (+ 1000us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 22:14:45.734658 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 22:14:45.734721 (+ 63us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 22:14:45.734724 (+ 3us) client_negotiation.cc:769] Sending connection context
0923 22:14:45.734772 (+ 48us) client_negotiation.cc:240] Negotiation successful
0923 22:14:45.734807 (+ 35us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":897,"thread_start_us":78,"threads_started":1}
I20260923 22:14:45.735389 14795 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 22:14:45.728044 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57472 (local address 127.1.135.195:43765)
0923 22:14:45.728179 (+ 135us) server_negotiation.cc:206] Beginning negotiation
0923 22:14:45.728183 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0923 22:14:45.728861 (+ 678us) server_negotiation.cc:407] Connection header received
0923 22:14:45.729173 (+ 312us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 22:14:45.729176 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 22:14:45.729190 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 22:14:45.729236 (+ 46us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0923 22:14:45.730280 (+ 1044us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 22:14:45.731294 (+ 1014us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 22:14:45.733745 (+ 2451us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 22:14:45.734495 (+ 750us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 22:14:45.734550 (+ 55us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 22:14:45.734597 (+ 47us) server_negotiation.cc:1035] Waiting for connection context
0923 22:14:45.735250 (+ 653us) server_negotiation.cc:299] Negotiation successful
0923 22:14:45.735296 (+ 46us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":113,"thread_start_us":62,"threads_started":1}
I20260923 22:14:45.735668 14790 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 22:14:45.727747 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.135.193:34647 (local address 127.0.0.1:41596)
0923 22:14:45.728561 (+ 814us) negotiation.cc:107] Waiting for socket to connect
0923 22:14:45.728569 (+ 8us) client_negotiation.cc:174] Beginning negotiation
0923 22:14:45.728608 (+ 39us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0923 22:14:45.729435 (+ 827us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0923 22:14:45.729436 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0923 22:14:45.729437 (+ 1us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0923 22:14:45.729468 (+ 31us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 22:14:45.729469 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 22:14:45.733427 (+ 3958us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 22:14:45.733428 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 22:14:45.734437 (+ 1009us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 22:14:45.734439 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 22:14:45.735450 (+ 1011us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 22:14:45.735451 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 22:14:45.735510 (+ 59us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 22:14:45.735522 (+ 12us) client_negotiation.cc:769] Sending connection context
0923 22:14:45.735572 (+ 50us) client_negotiation.cc:240] Negotiation successful
0923 22:14:45.735606 (+ 34us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":799,"thread_start_us":67,"threads_started":1}
I20260923 22:14:45.735886 14793 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 22:14:45.727828 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41596 (local address 127.1.135.193:34647)
0923 22:14:45.728241 (+ 413us) server_negotiation.cc:206] Beginning negotiation
0923 22:14:45.728245 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0923 22:14:45.728891 (+ 646us) server_negotiation.cc:407] Connection header received
0923 22:14:45.728927 (+ 36us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 22:14:45.728929 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 22:14:45.728945 (+ 16us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 22:14:45.729008 (+ 63us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0923 22:14:45.729539 (+ 531us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 22:14:45.733326 (+ 3787us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 22:14:45.734547 (+ 1221us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 22:14:45.735261 (+ 714us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 22:14:45.735349 (+ 88us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 22:14:45.735414 (+ 65us) server_negotiation.cc:1035] Waiting for connection context
0923 22:14:45.735756 (+ 342us) server_negotiation.cc:299] Negotiation successful
0923 22:14:45.735798 (+ 42us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":386,"thread_start_us":52,"threads_started":1}
I20260923 22:14:45.735952 14678 tablet_service.cc:1467] Processing CreateTablet for tablet 3465c9c0b91141b885540c8a1bd526de (DEFAULT_TABLE table=test-table [id=0b389b74dc0a45a1bd0841bb939615e1]), partition=RANGE (key) PARTITION UNBOUNDED
I20260923 22:14:45.736313 14678 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 3465c9c0b91141b885540c8a1bd526de. 1 dirs total, 0 dirs full, 0 dirs failed
I20260923 22:14:45.738448 14428 tablet_service.cc:1467] Processing CreateTablet for tablet 3465c9c0b91141b885540c8a1bd526de (DEFAULT_TABLE table=test-table [id=0b389b74dc0a45a1bd0841bb939615e1]), partition=RANGE (key) PARTITION UNBOUNDED
I20260923 22:14:45.738749 14428 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 3465c9c0b91141b885540c8a1bd526de. 1 dirs total, 0 dirs full, 0 dirs failed
I20260923 22:14:45.739960 14797 tablet_bootstrap.cc:492] T 3465c9c0b91141b885540c8a1bd526de P 226dc984ad3b4589883715494ff740a4: Bootstrap starting.
I20260923 22:14:45.740607 14797 tablet_bootstrap.cc:654] T 3465c9c0b91141b885540c8a1bd526de P 226dc984ad3b4589883715494ff740a4: Neither blocks nor log segments found. Creating new log.
I20260923 22:14:45.741318 14797 log.cc:824] T 3465c9c0b91141b885540c8a1bd526de P 226dc984ad3b4589883715494ff740a4: Log is configured to *not* fsync() on all Append() calls
I20260923 22:14:45.741896 14799 tablet_bootstrap.cc:492] T 3465c9c0b91141b885540c8a1bd526de P 0ff4f9455d6444269b402aa13dc889e6: Bootstrap starting.
I20260923 22:14:45.742254 14796 tablet_bootstrap.cc:492] T 3465c9c0b91141b885540c8a1bd526de P ff2dd7276bc647e3b2cee0d9ec03c525: Bootstrap starting.
I20260923 22:14:45.742455 14799 tablet_bootstrap.cc:654] T 3465c9c0b91141b885540c8a1bd526de P 0ff4f9455d6444269b402aa13dc889e6: Neither blocks nor log segments found. Creating new log.
I20260923 22:14:45.742784 14799 log.cc:824] T 3465c9c0b91141b885540c8a1bd526de P 0ff4f9455d6444269b402aa13dc889e6: Log is configured to *not* fsync() on all Append() calls
I20260923 22:14:45.742914 14796 tablet_bootstrap.cc:654] T 3465c9c0b91141b885540c8a1bd526de P ff2dd7276bc647e3b2cee0d9ec03c525: Neither blocks nor log segments found. Creating new log.
I20260923 22:14:45.743237 14796 log.cc:824] T 3465c9c0b91141b885540c8a1bd526de P ff2dd7276bc647e3b2cee0d9ec03c525: Log is configured to *not* fsync() on all Append() calls
I20260923 22:14:45.743533 14799 tablet_bootstrap.cc:492] T 3465c9c0b91141b885540c8a1bd526de P 0ff4f9455d6444269b402aa13dc889e6: No bootstrap required, opened a new log
I20260923 22:14:45.743604 14799 ts_tablet_manager.cc:1397] T 3465c9c0b91141b885540c8a1bd526de P 0ff4f9455d6444269b402aa13dc889e6: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20260923 22:14:45.745429 14797 tablet_bootstrap.cc:492] T 3465c9c0b91141b885540c8a1bd526de P 226dc984ad3b4589883715494ff740a4: No bootstrap required, opened a new log
I20260923 22:14:45.745486 14797 ts_tablet_manager.cc:1397] T 3465c9c0b91141b885540c8a1bd526de P 226dc984ad3b4589883715494ff740a4: Time spent bootstrapping tablet: real 0.006s user 0.002s sys 0.000s
I20260923 22:14:45.746016 14799 raft_consensus.cc:348] T 3465c9c0b91141b885540c8a1bd526de P 0ff4f9455d6444269b402aa13dc889e6 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ff2dd7276bc647e3b2cee0d9ec03c525" member_type: VOTER last_known_addr { host: "127.1.135.194" port: 39053 } } peers { permanent_uuid: "0ff4f9455d6444269b402aa13dc889e6" member_type: VOTER last_known_addr { host: "127.1.135.193" port: 34647 } } peers { permanent_uuid: "226dc984ad3b4589883715494ff740a4" member_type: VOTER last_known_addr { host: "127.1.135.195" port: 43765 } }
I20260923 22:14:45.746192 14799 raft_consensus.cc:374] T 3465c9c0b91141b885540c8a1bd526de P 0ff4f9455d6444269b402aa13dc889e6 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260923 22:14:45.746228 14799 raft_consensus.cc:729] T 3465c9c0b91141b885540c8a1bd526de P 0ff4f9455d6444269b402aa13dc889e6 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 0ff4f9455d6444269b402aa13dc889e6, State: Initialized, Role: FOLLOWER
I20260923 22:14:45.746378 14799 consensus_queue.cc:260] T 3465c9c0b91141b885540c8a1bd526de P 0ff4f9455d6444269b402aa13dc889e6 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ff2dd7276bc647e3b2cee0d9ec03c525" member_type: VOTER last_known_addr { host: "127.1.135.194" port: 39053 } } peers { permanent_uuid: "0ff4f9455d6444269b402aa13dc889e6" member_type: VOTER last_known_addr { host: "127.1.135.193" port: 34647 } } peers { permanent_uuid: "226dc984ad3b4589883715494ff740a4" member_type: VOTER last_known_addr { host: "127.1.135.195" port: 43765 } }
I20260923 22:14:45.746632 14799 ts_tablet_manager.cc:1428] T 3465c9c0b91141b885540c8a1bd526de P 0ff4f9455d6444269b402aa13dc889e6: Time spent starting tablet: real 0.003s user 0.003s sys 0.000s
I20260923 22:14:45.746742 14495 heartbeater.cc:503] Master 127.1.135.254:37257 was elected leader, sending a full tablet report...
I20260923 22:14:45.747421 14797 raft_consensus.cc:348] T 3465c9c0b91141b885540c8a1bd526de P 226dc984ad3b4589883715494ff740a4 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ff2dd7276bc647e3b2cee0d9ec03c525" member_type: VOTER last_known_addr { host: "127.1.135.194" port: 39053 } } peers { permanent_uuid: "0ff4f9455d6444269b402aa13dc889e6" member_type: VOTER last_known_addr { host: "127.1.135.193" port: 34647 } } peers { permanent_uuid: "226dc984ad3b4589883715494ff740a4" member_type: VOTER last_known_addr { host: "127.1.135.195" port: 43765 } }
I20260923 22:14:45.747547 14797 raft_consensus.cc:374] T 3465c9c0b91141b885540c8a1bd526de P 226dc984ad3b4589883715494ff740a4 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260923 22:14:45.747574 14797 raft_consensus.cc:729] T 3465c9c0b91141b885540c8a1bd526de P 226dc984ad3b4589883715494ff740a4 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 226dc984ad3b4589883715494ff740a4, State: Initialized, Role: FOLLOWER
I20260923 22:14:45.747665 14797 consensus_queue.cc:260] T 3465c9c0b91141b885540c8a1bd526de P 226dc984ad3b4589883715494ff740a4 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ff2dd7276bc647e3b2cee0d9ec03c525" member_type: VOTER last_known_addr { host: "127.1.135.194" port: 39053 } } peers { permanent_uuid: "0ff4f9455d6444269b402aa13dc889e6" member_type: VOTER last_known_addr { host: "127.1.135.193" port: 34647 } } peers { permanent_uuid: "226dc984ad3b4589883715494ff740a4" member_type: VOTER last_known_addr { host: "127.1.135.195" port: 43765 } }
I20260923 22:14:45.747886 14797 ts_tablet_manager.cc:1428] T 3465c9c0b91141b885540c8a1bd526de P 226dc984ad3b4589883715494ff740a4: Time spent starting tablet: real 0.002s user 0.002s sys 0.000s
I20260923 22:14:45.748196 14763 heartbeater.cc:503] Master 127.1.135.254:37257 was elected leader, sending a full tablet report...
I20260923 22:14:45.748994 14796 tablet_bootstrap.cc:492] T 3465c9c0b91141b885540c8a1bd526de P ff2dd7276bc647e3b2cee0d9ec03c525: No bootstrap required, opened a new log
I20260923 22:14:45.749068 14796 ts_tablet_manager.cc:1397] T 3465c9c0b91141b885540c8a1bd526de P ff2dd7276bc647e3b2cee0d9ec03c525: Time spent bootstrapping tablet: real 0.007s user 0.001s sys 0.000s
I20260923 22:14:45.751263 14796 raft_consensus.cc:348] T 3465c9c0b91141b885540c8a1bd526de P ff2dd7276bc647e3b2cee0d9ec03c525 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ff2dd7276bc647e3b2cee0d9ec03c525" member_type: VOTER last_known_addr { host: "127.1.135.194" port: 39053 } } peers { permanent_uuid: "0ff4f9455d6444269b402aa13dc889e6" member_type: VOTER last_known_addr { host: "127.1.135.193" port: 34647 } } peers { permanent_uuid: "226dc984ad3b4589883715494ff740a4" member_type: VOTER last_known_addr { host: "127.1.135.195" port: 43765 } }
I20260923 22:14:45.751402 14796 raft_consensus.cc:374] T 3465c9c0b91141b885540c8a1bd526de P ff2dd7276bc647e3b2cee0d9ec03c525 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260923 22:14:45.751430 14796 raft_consensus.cc:729] T 3465c9c0b91141b885540c8a1bd526de P ff2dd7276bc647e3b2cee0d9ec03c525 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: ff2dd7276bc647e3b2cee0d9ec03c525, State: Initialized, Role: FOLLOWER
I20260923 22:14:45.751515 14796 consensus_queue.cc:260] T 3465c9c0b91141b885540c8a1bd526de P ff2dd7276bc647e3b2cee0d9ec03c525 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ff2dd7276bc647e3b2cee0d9ec03c525" member_type: VOTER last_known_addr { host: "127.1.135.194" port: 39053 } } peers { permanent_uuid: "0ff4f9455d6444269b402aa13dc889e6" member_type: VOTER last_known_addr { host: "127.1.135.193" port: 34647 } } peers { permanent_uuid: "226dc984ad3b4589883715494ff740a4" member_type: VOTER last_known_addr { host: "127.1.135.195" port: 43765 } }
I20260923 22:14:45.751773 14629 heartbeater.cc:503] Master 127.1.135.254:37257 was elected leader, sending a full tablet report...
I20260923 22:14:45.751727 14796 ts_tablet_manager.cc:1428] T 3465c9c0b91141b885540c8a1bd526de P ff2dd7276bc647e3b2cee0d9ec03c525: Time spent starting tablet: real 0.003s user 0.003s sys 0.000s
W20260923 22:14:45.757777 14630 tablet.cc:2369] T 3465c9c0b91141b885540c8a1bd526de P ff2dd7276bc647e3b2cee0d9ec03c525: Can't schedule compaction. Clean time has not been advanced past its initial value.
Sep 23 22:14:45 dist-test-slave-s2fj krb5kdc[13301](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790201685, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.135.254@KRBTEST.COM for HTTP/127.1.135.212@KRBTEST.COM
I20260923 22:14:45.820684 14803 raft_consensus.cc:482] T 3465c9c0b91141b885540c8a1bd526de P 226dc984ad3b4589883715494ff740a4 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260923 22:14:45.820849 14803 raft_consensus.cc:504] T 3465c9c0b91141b885540c8a1bd526de P 226dc984ad3b4589883715494ff740a4 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ff2dd7276bc647e3b2cee0d9ec03c525" member_type: VOTER last_known_addr { host: "127.1.135.194" port: 39053 } } peers { permanent_uuid: "0ff4f9455d6444269b402aa13dc889e6" member_type: VOTER last_known_addr { host: "127.1.135.193" port: 34647 } } peers { permanent_uuid: "226dc984ad3b4589883715494ff740a4" member_type: VOTER last_known_addr { host: "127.1.135.195" port: 43765 } }
I20260923 22:14:45.821250 14803 leader_election.cc:288] T 3465c9c0b91141b885540c8a1bd526de P 226dc984ad3b4589883715494ff740a4 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers ff2dd7276bc647e3b2cee0d9ec03c525 (127.1.135.194:39053), 0ff4f9455d6444269b402aa13dc889e6 (127.1.135.193:34647)
W20260923 22:14:45.826224 14496 tablet.cc:2369] T 3465c9c0b91141b885540c8a1bd526de P 0ff4f9455d6444269b402aa13dc889e6: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260923 22:14:45.826916 14793 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 22:14:45.821696 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.135.195:54441 (local address 127.1.135.193:34647)
0923 22:14:45.821893 (+ 197us) server_negotiation.cc:206] Beginning negotiation
0923 22:14:45.821895 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0923 22:14:45.821901 (+ 6us) server_negotiation.cc:407] Connection header received
0923 22:14:45.821925 (+ 24us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 22:14:45.821926 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 22:14:45.821934 (+ 8us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 22:14:45.821995 (+ 61us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0923 22:14:45.822564 (+ 569us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 22:14:45.823294 (+ 730us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 22:14:45.825522 (+ 2228us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 22:14:45.826087 (+ 565us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 22:14:45.826137 (+ 50us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 22:14:45.826158 (+ 21us) server_negotiation.cc:1035] Waiting for connection context
0923 22:14:45.826815 (+ 657us) server_negotiation.cc:299] Negotiation successful
0923 22:14:45.826854 (+ 39us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":175}
I20260923 22:14:45.827293 14791 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 22:14:45.821637 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.135.195:39227 (local address 127.1.135.194:39053)
0923 22:14:45.821736 (+ 99us) server_negotiation.cc:206] Beginning negotiation
0923 22:14:45.821738 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0923 22:14:45.821746 (+ 8us) server_negotiation.cc:407] Connection header received
0923 22:14:45.821773 (+ 27us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 22:14:45.821774 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 22:14:45.821782 (+ 8us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 22:14:45.821839 (+ 57us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0923 22:14:45.822030 (+ 191us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 22:14:45.822737 (+ 707us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 22:14:45.826247 (+ 3510us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 22:14:45.826749 (+ 502us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 22:14:45.826794 (+ 45us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 22:14:45.826801 (+ 7us) server_negotiation.cc:1035] Waiting for connection context
0923 22:14:45.827195 (+ 394us) server_negotiation.cc:299] Negotiation successful
0923 22:14:45.827238 (+ 43us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":78}
I20260923 22:14:45.827411 14807 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 22:14:45.821433 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.135.193:34647 (local address 127.1.135.195:54441)
0923 22:14:45.821725 (+ 292us) negotiation.cc:107] Waiting for socket to connect
0923 22:14:45.821732 (+ 7us) client_negotiation.cc:174] Beginning negotiation
0923 22:14:45.821761 (+ 29us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0923 22:14:45.822404 (+ 643us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0923 22:14:45.822405 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0923 22:14:45.822407 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0923 22:14:45.822473 (+ 66us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 22:14:45.822475 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 22:14:45.823600 (+ 1125us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 22:14:45.823601 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 22:14:45.825420 (+ 1819us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 22:14:45.825423 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 22:14:45.826666 (+ 1243us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 22:14:45.826667 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 22:14:45.826733 (+ 66us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 22:14:45.826736 (+ 3us) client_negotiation.cc:769] Sending connection context
0923 22:14:45.827269 (+ 533us) client_negotiation.cc:240] Negotiation successful
0923 22:14:45.827308 (+ 39us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":280,"thread_start_us":89,"threads_started":1}
I20260923 22:14:45.827736 14808 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 22:14:45.821393 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.135.194:39053 (local address 127.1.135.195:39227)
0923 22:14:45.821599 (+ 206us) negotiation.cc:107] Waiting for socket to connect
0923 22:14:45.821609 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0923 22:14:45.821658 (+ 49us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0923 22:14:45.821843 (+ 185us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0923 22:14:45.821844 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0923 22:14:45.821845 (+ 1us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0923 22:14:45.821919 (+ 74us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 22:14:45.821921 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 22:14:45.823789 (+ 1868us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 22:14:45.823790 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 22:14:45.826150 (+ 2360us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 22:14:45.826153 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 22:14:45.827056 (+ 903us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 22:14:45.827057 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 22:14:45.827124 (+ 67us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 22:14:45.827127 (+ 3us) client_negotiation.cc:769] Sending connection context
0923 22:14:45.827627 (+ 500us) client_negotiation.cc:240] Negotiation successful
0923 22:14:45.827667 (+ 40us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":186,"thread_start_us":88,"threads_started":1}
I20260923 22:14:45.828253 14449 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "3465c9c0b91141b885540c8a1bd526de" candidate_uuid: "226dc984ad3b4589883715494ff740a4" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0ff4f9455d6444269b402aa13dc889e6" is_pre_election: true
I20260923 22:14:45.828406 14449 raft_consensus.cc:2454] T 3465c9c0b91141b885540c8a1bd526de P 0ff4f9455d6444269b402aa13dc889e6 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 226dc984ad3b4589883715494ff740a4 in term 0.
I20260923 22:14:45.828459 14583 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "3465c9c0b91141b885540c8a1bd526de" candidate_uuid: "226dc984ad3b4589883715494ff740a4" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ff2dd7276bc647e3b2cee0d9ec03c525" is_pre_election: true
I20260923 22:14:45.828595 14583 raft_consensus.cc:2454] T 3465c9c0b91141b885540c8a1bd526de P ff2dd7276bc647e3b2cee0d9ec03c525 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 226dc984ad3b4589883715494ff740a4 in term 0.
I20260923 22:14:45.828622 14651 leader_election.cc:302] T 3465c9c0b91141b885540c8a1bd526de P 226dc984ad3b4589883715494ff740a4 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0ff4f9455d6444269b402aa13dc889e6, 226dc984ad3b4589883715494ff740a4; no voters:
I20260923 22:14:45.828735 14803 raft_consensus.cc:2781] T 3465c9c0b91141b885540c8a1bd526de P 226dc984ad3b4589883715494ff740a4 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260923 22:14:45.828794 14803 raft_consensus.cc:482] T 3465c9c0b91141b885540c8a1bd526de P 226dc984ad3b4589883715494ff740a4 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260923 22:14:45.828816 14803 raft_consensus.cc:3037] T 3465c9c0b91141b885540c8a1bd526de P 226dc984ad3b4589883715494ff740a4 [term 0 FOLLOWER]: Advancing to term 1
I20260923 22:14:45.829696 14803 raft_consensus.cc:504] T 3465c9c0b91141b885540c8a1bd526de P 226dc984ad3b4589883715494ff740a4 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ff2dd7276bc647e3b2cee0d9ec03c525" member_type: VOTER last_known_addr { host: "127.1.135.194" port: 39053 } } peers { permanent_uuid: "0ff4f9455d6444269b402aa13dc889e6" member_type: VOTER last_known_addr { host: "127.1.135.193" port: 34647 } } peers { permanent_uuid: "226dc984ad3b4589883715494ff740a4" member_type: VOTER last_known_addr { host: "127.1.135.195" port: 43765 } }
I20260923 22:14:45.829821 14803 leader_election.cc:288] T 3465c9c0b91141b885540c8a1bd526de P 226dc984ad3b4589883715494ff740a4 [CANDIDATE]: Term 1 election: Requested vote from peers ff2dd7276bc647e3b2cee0d9ec03c525 (127.1.135.194:39053), 0ff4f9455d6444269b402aa13dc889e6 (127.1.135.193:34647)
I20260923 22:14:45.829959 14583 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "3465c9c0b91141b885540c8a1bd526de" candidate_uuid: "226dc984ad3b4589883715494ff740a4" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ff2dd7276bc647e3b2cee0d9ec03c525"
I20260923 22:14:45.830034 14583 raft_consensus.cc:3037] T 3465c9c0b91141b885540c8a1bd526de P ff2dd7276bc647e3b2cee0d9ec03c525 [term 0 FOLLOWER]: Advancing to term 1
I20260923 22:14:45.830185 14449 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "3465c9c0b91141b885540c8a1bd526de" candidate_uuid: "226dc984ad3b4589883715494ff740a4" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0ff4f9455d6444269b402aa13dc889e6"
I20260923 22:14:45.830248 14449 raft_consensus.cc:3037] T 3465c9c0b91141b885540c8a1bd526de P 0ff4f9455d6444269b402aa13dc889e6 [term 0 FOLLOWER]: Advancing to term 1
I20260923 22:14:45.830796 14583 raft_consensus.cc:2454] T 3465c9c0b91141b885540c8a1bd526de P ff2dd7276bc647e3b2cee0d9ec03c525 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 226dc984ad3b4589883715494ff740a4 in term 1.
I20260923 22:14:45.830976 14653 leader_election.cc:302] T 3465c9c0b91141b885540c8a1bd526de P 226dc984ad3b4589883715494ff740a4 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 226dc984ad3b4589883715494ff740a4, ff2dd7276bc647e3b2cee0d9ec03c525; no voters:
I20260923 22:14:45.831040 14449 raft_consensus.cc:2454] T 3465c9c0b91141b885540c8a1bd526de P 0ff4f9455d6444269b402aa13dc889e6 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 226dc984ad3b4589883715494ff740a4 in term 1.
I20260923 22:14:45.831131 14803 raft_consensus.cc:2781] T 3465c9c0b91141b885540c8a1bd526de P 226dc984ad3b4589883715494ff740a4 [term 1 FOLLOWER]: Leader election won for term 1
I20260923 22:14:45.831248 14803 raft_consensus.cc:686] T 3465c9c0b91141b885540c8a1bd526de P 226dc984ad3b4589883715494ff740a4 [term 1 LEADER]: Becoming Leader. State: Replica: 226dc984ad3b4589883715494ff740a4, State: Running, Role: LEADER
I20260923 22:14:45.831318 14803 consensus_queue.cc:237] T 3465c9c0b91141b885540c8a1bd526de P 226dc984ad3b4589883715494ff740a4 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ff2dd7276bc647e3b2cee0d9ec03c525" member_type: VOTER last_known_addr { host: "127.1.135.194" port: 39053 } } peers { permanent_uuid: "0ff4f9455d6444269b402aa13dc889e6" member_type: VOTER last_known_addr { host: "127.1.135.193" port: 34647 } } peers { permanent_uuid: "226dc984ad3b4589883715494ff740a4" member_type: VOTER last_known_addr { host: "127.1.135.195" port: 43765 } }
I20260923 22:14:45.832087 14296 catalog_manager.cc:5495] T 3465c9c0b91141b885540c8a1bd526de P 226dc984ad3b4589883715494ff740a4 reported cstate change: term changed from 0 to 1, leader changed from <none> to 226dc984ad3b4589883715494ff740a4 (127.1.135.195). New cstate: current_term: 1 leader_uuid: "226dc984ad3b4589883715494ff740a4" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ff2dd7276bc647e3b2cee0d9ec03c525" member_type: VOTER last_known_addr { host: "127.1.135.194" port: 39053 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "0ff4f9455d6444269b402aa13dc889e6" member_type: VOTER last_known_addr { host: "127.1.135.193" port: 34647 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "226dc984ad3b4589883715494ff740a4" member_type: VOTER last_known_addr { host: "127.1.135.195" port: 43765 } health_report { overall_health: HEALTHY } } }
I20260923 22:14:45.853679 14795 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 22:14:45.850382 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57476 (local address 127.1.135.195:43765)
0923 22:14:45.850420 (+ 38us) server_negotiation.cc:206] Beginning negotiation
0923 22:14:45.850422 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0923 22:14:45.850430 (+ 8us) server_negotiation.cc:407] Connection header received
0923 22:14:45.850475 (+ 45us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 22:14:45.850476 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 22:14:45.850494 (+ 18us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 22:14:45.850553 (+ 59us) server_negotiation.cc:226] Negotiated authn=TOKEN
0923 22:14:45.850753 (+ 200us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 22:14:45.851435 (+ 682us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 22:14:45.852659 (+ 1224us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 22:14:45.852948 (+ 289us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 22:14:45.853008 (+ 60us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 22:14:45.853196 (+ 188us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0923 22:14:45.853318 (+ 122us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0923 22:14:45.853379 (+ 61us) server_negotiation.cc:1035] Waiting for connection context
0923 22:14:45.853570 (+ 191us) server_negotiation.cc:299] Negotiation successful
0923 22:14:45.853587 (+ 17us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":20}
I20260923 22:14:45.855587 14449 raft_consensus.cc:1261] T 3465c9c0b91141b885540c8a1bd526de P 0ff4f9455d6444269b402aa13dc889e6 [term 1 FOLLOWER]: Refusing update from remote peer 226dc984ad3b4589883715494ff740a4: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260923 22:14:45.855846 14583 raft_consensus.cc:1261] T 3465c9c0b91141b885540c8a1bd526de P ff2dd7276bc647e3b2cee0d9ec03c525 [term 1 FOLLOWER]: Refusing update from remote peer 226dc984ad3b4589883715494ff740a4: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260923 22:14:45.856276 14803 consensus_queue.cc:1035] T 3465c9c0b91141b885540c8a1bd526de P 226dc984ad3b4589883715494ff740a4 [LEADER]: Connected to new peer: Peer: permanent_uuid: "ff2dd7276bc647e3b2cee0d9ec03c525" member_type: VOTER last_known_addr { host: "127.1.135.194" port: 39053 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260923 22:14:45.856407 14803 consensus_queue.cc:1035] T 3465c9c0b91141b885540c8a1bd526de P 226dc984ad3b4589883715494ff740a4 [LEADER]: Connected to new peer: Peer: permanent_uuid: "0ff4f9455d6444269b402aa13dc889e6" member_type: VOTER last_known_addr { host: "127.1.135.193" port: 34647 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260923 22:14:45.860661 14812 mvcc.cc:205] Tried to move back new op lower bound from 7332666105261572096 to 7332666105165594624. Current Snapshot: MvccSnapshot[applied={T|T < 7332666105261572096}]
I20260923 22:14:45.862685 14815 mvcc.cc:205] Tried to move back new op lower bound from 7332666105261572096 to 7332666105165594624. Current Snapshot: MvccSnapshot[applied={T|T < 7332666105261572096}]
I20260923 22:14:45.864524 14813 mvcc.cc:205] Tried to move back new op lower bound from 7332666105261572096 to 7332666105165594624. Current Snapshot: MvccSnapshot[applied={T|T < 7332666105261572096}]
I20260923 22:14:45.867062 14297 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:33922:
table { table_name: "test-table" } modify_external_catalogs: true
I20260923 22:14:45.867142 14297 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:33922:
table { table_name: "test-table" } modify_external_catalogs: true
I20260923 22:14:45.867792 14297 catalog_manager.cc:5782] T 00000000000000000000000000000000 P d7b9fc6eb32b4fb5a1e6920bf231523d: Sending DeleteTablet for 3 replicas of tablet 3465c9c0b91141b885540c8a1bd526de
I20260923 22:14:45.868117 14563 tablet_service.cc:1514] Processing DeleteTablet for tablet 3465c9c0b91141b885540c8a1bd526de with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-23 22:14:45 UTC) from {username='kudu', principal='kudu/127.1.135.254@KRBTEST.COM'} at 127.0.0.1:41694
I20260923 22:14:45.868310 14678 tablet_service.cc:1514] Processing DeleteTablet for tablet 3465c9c0b91141b885540c8a1bd526de with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-23 22:14:45 UTC) from {username='kudu', principal='kudu/127.1.135.254@KRBTEST.COM'} at 127.0.0.1:57472
I20260923 22:14:45.868310 14428 tablet_service.cc:1514] Processing DeleteTablet for tablet 3465c9c0b91141b885540c8a1bd526de with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-23 22:14:45 UTC) from {username='kudu', principal='kudu/127.1.135.254@KRBTEST.COM'} at 127.0.0.1:41596
I20260923 22:14:45.868700 1567 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskRiQe_a/build/release/bin/kudu with pid 14336
I20260923 22:14:45.880142 14823 tablet_replica.cc:330] T 3465c9c0b91141b885540c8a1bd526de P 226dc984ad3b4589883715494ff740a4: stopping tablet replica
I20260923 22:14:45.884116 14823 raft_consensus.cc:2229] T 3465c9c0b91141b885540c8a1bd526de P 226dc984ad3b4589883715494ff740a4 [term 1 LEADER]: Raft consensus shutting down.
I20260923 22:14:45.884341 14823 raft_consensus.cc:2258] T 3465c9c0b91141b885540c8a1bd526de P 226dc984ad3b4589883715494ff740a4 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260923 22:14:45.884543 14822 tablet_replica.cc:330] T 3465c9c0b91141b885540c8a1bd526de P ff2dd7276bc647e3b2cee0d9ec03c525: stopping tablet replica
I20260923 22:14:45.884977 14823 ts_tablet_manager.cc:1905] T 3465c9c0b91141b885540c8a1bd526de P 226dc984ad3b4589883715494ff740a4: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260923 22:14:45.886216 14822 raft_consensus.cc:2229] T 3465c9c0b91141b885540c8a1bd526de P ff2dd7276bc647e3b2cee0d9ec03c525 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260923 22:14:45.886394 14822 raft_consensus.cc:2258] T 3465c9c0b91141b885540c8a1bd526de P ff2dd7276bc647e3b2cee0d9ec03c525 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260923 22:14:45.887055 14823 ts_tablet_manager.cc:1918] T 3465c9c0b91141b885540c8a1bd526de P 226dc984ad3b4589883715494ff740a4: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260923 22:14:45.887476 14822 ts_tablet_manager.cc:1905] T 3465c9c0b91141b885540c8a1bd526de P ff2dd7276bc647e3b2cee0d9ec03c525: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260923 22:14:45.889060 14822 ts_tablet_manager.cc:1918] T 3465c9c0b91141b885540c8a1bd526de P ff2dd7276bc647e3b2cee0d9ec03c525: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260923 22:14:45.889207 14823 log.cc:1196] T 3465c9c0b91141b885540c8a1bd526de P 226dc984ad3b4589883715494ff740a4: Deleting WAL directory at /tmp/dist-test-taskRiQe_a/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790201539513534-1567-0/minicluster-data/ts-2/wal/wals/3465c9c0b91141b885540c8a1bd526de
I20260923 22:14:45.889523 14823 ts_tablet_manager.cc:1939] T 3465c9c0b91141b885540c8a1bd526de P 226dc984ad3b4589883715494ff740a4: Deleting consensus metadata
I20260923 22:14:45.890034 14282 catalog_manager.cc:4841] TS 226dc984ad3b4589883715494ff740a4 (127.1.135.195:43765): tablet 3465c9c0b91141b885540c8a1bd526de (table test-table [id=0b389b74dc0a45a1bd0841bb939615e1]) successfully deleted
I20260923 22:14:45.890234 14822 log.cc:1196] T 3465c9c0b91141b885540c8a1bd526de P ff2dd7276bc647e3b2cee0d9ec03c525: Deleting WAL directory at /tmp/dist-test-taskRiQe_a/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790201539513534-1567-0/minicluster-data/ts-1/wal/wals/3465c9c0b91141b885540c8a1bd526de
I20260923 22:14:45.890558 14822 ts_tablet_manager.cc:1939] T 3465c9c0b91141b885540c8a1bd526de P ff2dd7276bc647e3b2cee0d9ec03c525: Deleting consensus metadata
I20260923 22:14:45.890997 14284 catalog_manager.cc:4841] TS ff2dd7276bc647e3b2cee0d9ec03c525 (127.1.135.194:39053): tablet 3465c9c0b91141b885540c8a1bd526de (table test-table [id=0b389b74dc0a45a1bd0841bb939615e1]) successfully deleted
W20260923 22:14:45.892362 14282 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.1.135.193:34647 (error 108)
I20260923 22:14:45.894800 1567 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskRiQe_a/build/release/bin/kudu with pid 14499
I20260923 22:14:45.908205 1567 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskRiQe_a/build/release/bin/kudu with pid 14632
I20260923 22:14:45.911782 14792 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 22:14:45.911658 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.135.193:34647 (local address 127.0.0.1:41602)
0923 22:14:45.911700 (+ 42us) negotiation.cc:107] Waiting for socket to connect
0923 22:14:45.911745 (+ 45us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.135.193:34647: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":22}
W20260923 22:14:45.911991 14282 catalog_manager.cc:4584] TS 0ff4f9455d6444269b402aa13dc889e6 (127.1.135.193:34647): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet 3465c9c0b91141b885540c8a1bd526de: Network error: Client connection negotiation failed: client connection to 127.1.135.193:34647: connect: Connection refused (error 111)
I20260923 22:14:45.913743 1567 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskRiQe_a/build/release/bin/kudu with pid 14264
I20260923 22:14:45.920856 1567 mini_ranger_kms.cc:60] Stopping Ranger KMS...
I20260923 22:14:46.265118 1567 mini_ranger_kms.cc:62] Stopped Ranger KMS
I20260923 22:14:46.265205 1567 mini_ranger.cc:66] Stopping Ranger...
I20260923 22:14:46.611227 1567 mini_ranger.cc:68] Stopped Ranger
2026-09-23 22:14:46.611 UTC [13331] LOG: received smart shutdown request
2026-09-23 22:14:46.614 UTC [13331] LOG: background worker "logical replication launcher" (PID 13340) exited with exit code 1
2026-09-23 22:14:46.615 UTC [13335] LOG: shutting down
2026-09-23 22:14:46.763 UTC [13331] LOG: database system is shut down
2026-09-23T22:14:46Z chronyd exiting
[ OK ] SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers (45929 ms)
[ RUN ] SecurityITest.IPKICACert
Loading random data
Initializing database '/tmp/dist-test-taskRiQe_a/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790201539513534-1567-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Sep 23 22:14:46 dist-test-slave-s2fj krb5kdc[14834](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Sep 23 22:14:46 dist-test-slave-s2fj krb5kdc[14834](info): set up 2 sockets
Sep 23 22:14:46 dist-test-slave-s2fj krb5kdc[14834](info): commencing operation
krb5kdc: starting...
W20260923 22:14:47.891925 1567 mini_kdc.cc:121] Time spent starting KDC: real 1.001s user 0.003s sys 0.002s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskRiQe_a/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790201539513534-1567-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskRiQe_a/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790201539513534-1567-0/krb5kdc/test-user.keytab.
Sep 23 22:14:47 dist-test-slave-s2fj krb5kdc[14834](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790201687, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-09-23T22:14:47Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-09-23T22:14:47Z Disabled control of system clock
WARNING: no policy specified for kudu/127.1.135.254@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.1.135.254@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.1.135.254 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskRiQe_a/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790201539513534-1567-0/krb5kdc/kudu_127.1.135.254.keytab.
Entry for principal kudu/127.1.135.254 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskRiQe_a/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790201539513534-1567-0/krb5kdc/kudu_127.1.135.254.keytab.
I20260923 22:14:48.001365 1567 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskRiQe_a/build/release/bin/kudu
/tmp/dist-test-taskRiQe_a/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskRiQe_a/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790201539513534-1567-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskRiQe_a/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790201539513534-1567-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskRiQe_a/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790201539513534-1567-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskRiQe_a/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790201539513534-1567-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.1.135.254:39543
--webserver_interface=127.1.135.254
--webserver_port=0
--keytab_file=/tmp/dist-test-taskRiQe_a/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790201539513534-1567-0/krb5kdc/kudu_127.1.135.254.keytab
--principal=kudu/127.1.135.254
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.1.135.212:37859
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.1.135.254:39543,127.1.135.253:37899,127.1.135.252:36511
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskRiQe_a/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790201539513534-1567-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskRiQe_a/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790201539513534-1567-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskRiQe_a/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790201539513534-1567-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260923 22:14:48.077616 14847 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260923 22:14:48.077796 14847 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260923 22:14:48.077817 14847 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260923 22:14:48.078855 14847 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260923 22:14:48.078887 14847 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260923 22:14:48.078902 14847 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260923 22:14:48.078913 14847 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260923 22:14:48.078925 14847 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260923 22:14:48.080323 14847 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.1.135.212:37859
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskRiQe_a/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790201539513534-1567-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskRiQe_a/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790201539513534-1567-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.1.135.254:39543,127.1.135.253:37899,127.1.135.252:36511
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskRiQe_a/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790201539513534-1567-0/krb5kdc/kudu_127.1.135.254.keytab
--principal=kudu/127.1.135.254
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.1.135.254:39543
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskRiQe_a/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790201539513534-1567-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.1.135.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.14847
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskRiQe_a/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790201539513534-1567-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 23 Sep 2026 20:53:37 UTC on ccycloud.quasar-rkviyx.root.comops.site
build id 4668083
I20260923 22:14:48.080493 14847 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260923 22:14:48.080647 14847 file_cache.cc:493] Constructed file cache file cache with capacity 419430
I20260923 22:14:48.085248 14847 server_base.cc:1034] running on GCE node
W20260923 22:14:48.085233 14853 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260923 22:14:48.085222 14855 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260923 22:14:48.085228 14852 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260923 22:14:48.085692 14847 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260923 22:14:48.085970 14847 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260923 22:14:48.087102 14847 hybrid_clock.cc:648] HybridClock initialized: now 1790201688087097 us; error 33 us; skew 500 ppm
Sep 23 22:14:48 dist-test-slave-s2fj krb5kdc[14834](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790201688, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.135.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260923 22:14:48.094059 14847 init.cc:377] Logged in from keytab as kudu/127.1.135.254@KRBTEST.COM (short username kudu)
I20260923 22:14:48.094525 14847 webserver.cc:460] Webserver started at http://127.1.135.254:40927/ using document root <none> and password file <none>
I20260923 22:14:48.094740 14847 fs_manager.cc:359] Metadata directory not provided
I20260923 22:14:48.094806 14847 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260923 22:14:48.094951 14847 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260923 22:14:48.095958 14847 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskRiQe_a/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790201539513534-1567-0/minicluster-data/master-0/data/instance:
uuid: "d5923ad1b2bd4b5a997a7936eb1ca7f6"
format_stamp: "Formatted at 2026-09-23 22:14:48 on dist-test-slave-s2fj"
server_key: "8f9aa2fb865bc9db40e2bb91b01bbd39"
server_key_iv: "014fd4674356720bb3e48816a01c2f5c"
server_key_version: "encryptionkey@0"
I20260923 22:14:48.096362 14847 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskRiQe_a/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790201539513534-1567-0/minicluster-data/master-0/wal/instance:
uuid: "d5923ad1b2bd4b5a997a7936eb1ca7f6"
format_stamp: "Formatted at 2026-09-23 22:14:48 on dist-test-slave-s2fj"
server_key: "8f9aa2fb865bc9db40e2bb91b01bbd39"
server_key_iv: "014fd4674356720bb3e48816a01c2f5c"
server_key_version: "encryptionkey@0"
I20260923 22:14:48.097627 14847 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.001s sys 0.000s
I20260923 22:14:48.098315 14862 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260923 22:14:48.098471 14847 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260923 22:14:48.098569 14847 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskRiQe_a/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790201539513534-1567-0/minicluster-data/master-0/data,/tmp/dist-test-taskRiQe_a/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790201539513534-1567-0/minicluster-data/master-0/wal
uuid: "d5923ad1b2bd4b5a997a7936eb1ca7f6"
format_stamp: "Formatted at 2026-09-23 22:14:48 on dist-test-slave-s2fj"
server_key: "8f9aa2fb865bc9db40e2bb91b01bbd39"
server_key_iv: "014fd4674356720bb3e48816a01c2f5c"
server_key_version: "encryptionkey@0"
I20260923 22:14:48.098647 14847 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskRiQe_a/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790201539513534-1567-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskRiQe_a/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790201539513534-1567-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskRiQe_a/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790201539513534-1567-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260923 22:14:48.108651 14847 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260923 22:14:48.110744 14847 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260923 22:14:48.110893 14847 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260923 22:14:48.114905 14847 rpc_server.cc:307] RPC server started. Bound to: 127.1.135.254:39543
I20260923 22:14:48.114935 14914 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.1.135.254:39543 every 8 connection(s)
I20260923 22:14:48.115239 14847 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskRiQe_a/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790201539513534-1567-0/minicluster-data/master-0/data/info.pb
I20260923 22:14:48.115607 1567 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskRiQe_a/build/release/bin/kudu as pid 14847
I20260923 22:14:48.115736 1567 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskRiQe_a/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790201539513534-1567-0/minicluster-data/master-0/wal/instance
I20260923 22:14:48.115928 1567 external_mini_cluster.cc:1428] Setting key a5b088d1ac71e3f16ac891bb9a319713
I20260923 22:14:48.116170 14915 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260923 22:14:48.117767 14915 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.1.135.254" port: 39543 } has no permanent_uuid. Determining permanent_uuid...
WARNING: no policy specified for kudu/127.1.135.253@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.1.135.253@KRBTEST.COM" created.
Sep 23 22:14:48 dist-test-slave-s2fj krb5kdc[14834](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790201688, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.135.254@KRBTEST.COM for kudu/127.1.135.254@KRBTEST.COM
I20260923 22:14:48.130512 14918 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 22:14:48.118468 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.135.254:39543 (local address 127.0.0.1:40926)
0923 22:14:48.118611 (+ 143us) negotiation.cc:107] Waiting for socket to connect
0923 22:14:48.118644 (+ 33us) client_negotiation.cc:174] Beginning negotiation
0923 22:14:48.119068 (+ 424us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0923 22:14:48.119393 (+ 325us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0923 22:14:48.119395 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0923 22:14:48.119621 (+ 226us) client_negotiation.cc:189] Negotiated authn=SASL
0923 22:14:48.119777 (+ 156us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 22:14:48.119780 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 22:14:48.121370 (+ 1590us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 22:14:48.121371 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 22:14:48.121990 (+ 619us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 22:14:48.121991 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 22:14:48.122652 (+ 661us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 22:14:48.122654 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 22:14:48.122734 (+ 80us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 22:14:48.123246 (+ 512us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0923 22:14:48.123254 (+ 8us) client_negotiation.cc:656] calling sasl_client_start()
0923 22:14:48.127019 (+ 3765us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0923 22:14:48.129111 (+ 2092us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0923 22:14:48.129116 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0923 22:14:48.129117 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0923 22:14:48.129340 (+ 223us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0923 22:14:48.129616 (+ 276us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0923 22:14:48.129617 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0923 22:14:48.129617 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0923 22:14:48.129689 (+ 72us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0923 22:14:48.130140 (+ 451us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0923 22:14:48.130144 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0923 22:14:48.130193 (+ 49us) client_negotiation.cc:769] Sending connection context
0923 22:14:48.130260 (+ 67us) client_negotiation.cc:240] Negotiation successful
0923 22:14:48.130370 (+ 110us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":107,"thread_start_us":73,"threads_started":1}
I20260923 22:14:48.130642 14919 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 22:14:48.118482 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40926 (local address 127.1.135.254:39543)
0923 22:14:48.118641 (+ 159us) server_negotiation.cc:206] Beginning negotiation
0923 22:14:48.118649 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0923 22:14:48.118739 (+ 90us) server_negotiation.cc:407] Connection header received
0923 22:14:48.119248 (+ 509us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 22:14:48.119252 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 22:14:48.119275 (+ 23us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 22:14:48.119383 (+ 108us) server_negotiation.cc:226] Negotiated authn=SASL
0923 22:14:48.119877 (+ 494us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 22:14:48.121283 (+ 1406us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 22:14:48.122071 (+ 788us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 22:14:48.122562 (+ 491us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 22:14:48.122702 (+ 140us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 22:14:48.127099 (+ 4397us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0923 22:14:48.127105 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0923 22:14:48.127105 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0923 22:14:48.127111 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0923 22:14:48.129030 (+ 1919us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0923 22:14:48.129434 (+ 404us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0923 22:14:48.129438 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0923 22:14:48.129438 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0923 22:14:48.129542 (+ 104us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0923 22:14:48.129754 (+ 212us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0923 22:14:48.129755 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0923 22:14:48.129755 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0923 22:14:48.130065 (+ 310us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0923 22:14:48.130158 (+ 93us) server_negotiation.cc:1035] Waiting for connection context
0923 22:14:48.130319 (+ 161us) server_negotiation.cc:299] Negotiation successful
0923 22:14:48.130456 (+ 137us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":29,"server-negotiator.queue_time_us":120,"thread_start_us":72,"threads_started":1}
I20260923 22:14:48.131258 14915 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.1.135.253" port: 37899 } has no permanent_uuid. Determining permanent_uuid...
I20260923 22:14:48.131542 14918 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 22:14:48.131434 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.135.253:37899 (local address 127.0.0.1:55194)
0923 22:14:48.131473 (+ 39us) negotiation.cc:107] Waiting for socket to connect
0923 22:14:48.131507 (+ 34us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.135.253:37899: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":27}
W20260923 22:14:48.131706 14866 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.1.135.253:37899: connect: Connection refused (error 111)
I20260923 22:14:48.132174 14918 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 22:14:48.132036 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.135.253:37899 (local address 127.0.0.1:55206)
0923 22:14:48.132124 (+ 88us) negotiation.cc:107] Waiting for socket to connect
0923 22:14:48.132150 (+ 26us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.135.253:37899: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":75}
W20260923 22:14:48.132290 14915 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.1.135.253:37899: Network error: Client connection negotiation failed: client connection to 127.1.135.253:37899: connect: Connection refused (error 111)
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.1.135.253 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskRiQe_a/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790201539513534-1567-0/krb5kdc/kudu_127.1.135.253.keytab.
Entry for principal kudu/127.1.135.253 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskRiQe_a/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790201539513534-1567-0/krb5kdc/kudu_127.1.135.253.keytab.
I20260923 22:14:48.142306 1567 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskRiQe_a/build/release/bin/kudu
/tmp/dist-test-taskRiQe_a/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskRiQe_a/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790201539513534-1567-0/minicluster-data/master-1/wal
--fs_data_dirs=/tmp/dist-test-taskRiQe_a/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790201539513534-1567-0/minicluster-data/master-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskRiQe_a/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790201539513534-1567-0/minicluster-data/master-1/logs
--server_dump_info_path=/tmp/dist-test-taskRiQe_a/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790201539513534-1567-0/minicluster-data/master-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.1.135.253:37899
--webserver_interface=127.1.135.253
--webserver_port=0
--keytab_file=/tmp/dist-test-taskRiQe_a/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790201539513534-1567-0/krb5kdc/kudu_127.1.135.253.keytab
--principal=kudu/127.1.135.253
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.1.135.212:37859
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.1.135.254:39543,127.1.135.253:37899,127.1.135.252:36511
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskRiQe_a/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790201539513534-1567-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskRiQe_a/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790201539513534-1567-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskRiQe_a/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790201539513534-1567-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260923 22:14:48.181495 14915 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.1.135.253" port: 37899 } attempt: 1
I20260923 22:14:48.181931 14918 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 22:14:48.181806 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.135.253:37899 (local address 127.0.0.1:55214)
0923 22:14:48.181869 (+ 63us) negotiation.cc:107] Waiting for socket to connect
0923 22:14:48.181906 (+ 37us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.135.253:37899: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":41}
I20260923 22:14:48.182261 14918 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 22:14:48.182173 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.135.253:37899 (local address 127.0.0.1:55224)
0923 22:14:48.182216 (+ 43us) negotiation.cc:107] Waiting for socket to connect
0923 22:14:48.182238 (+ 22us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.135.253:37899: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":29}
W20260923 22:14:48.182343 14915 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.1.135.253:37899: Network error: Client connection negotiation failed: client connection to 127.1.135.253:37899: connect: Connection refused (error 111)
W20260923 22:14:48.220007 14922 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260923 22:14:48.220207 14922 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260923 22:14:48.220228 14922 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260923 22:14:48.221383 14922 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260923 22:14:48.221421 14922 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260923 22:14:48.221436 14922 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260923 22:14:48.221460 14922 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260923 22:14:48.221472 14922 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260923 22:14:48.223168 14922 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.1.135.212:37859
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskRiQe_a/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790201539513534-1567-0/minicluster-data/master-1/data
--fs_wal_dir=/tmp/dist-test-taskRiQe_a/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790201539513534-1567-0/minicluster-data/master-1/wal
--ipki_ca_key_size=768
--master_addresses=127.1.135.254:39543,127.1.135.253:37899,127.1.135.252:36511
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskRiQe_a/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790201539513534-1567-0/krb5kdc/kudu_127.1.135.253.keytab
--principal=kudu/127.1.135.253
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.1.135.253:37899
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskRiQe_a/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790201539513534-1567-0/minicluster-data/master-1/data/info.pb
--webserver_interface=127.1.135.253
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.14922
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskRiQe_a/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790201539513534-1567-0/minicluster-data/master-1/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 23 Sep 2026 20:53:37 UTC on ccycloud.quasar-rkviyx.root.comops.site
build id 4668083
I20260923 22:14:48.223412 14922 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260923 22:14:48.223630 14922 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260923 22:14:48.228098 14928 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260923 22:14:48.228142 14922 server_base.cc:1034] running on GCE node
W20260923 22:14:48.228143 14927 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260923 22:14:48.228098 14930 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260923 22:14:48.228534 14922 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260923 22:14:48.228849 14922 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260923 22:14:48.230039 14922 hybrid_clock.cc:648] HybridClock initialized: now 1790201688230022 us; error 27 us; skew 500 ppm
Sep 23 22:14:48 dist-test-slave-s2fj krb5kdc[14834](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790201688, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.135.253@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260923 22:14:48.232402 14922 init.cc:377] Logged in from keytab as kudu/127.1.135.253@KRBTEST.COM (short username kudu)
I20260923 22:14:48.232786 14922 webserver.cc:460] Webserver started at http://127.1.135.253:44221/ using document root <none> and password file <none>
I20260923 22:14:48.232987 14922 fs_manager.cc:359] Metadata directory not provided
I20260923 22:14:48.233060 14922 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260923 22:14:48.233204 14922 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260923 22:14:48.234138 14922 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskRiQe_a/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790201539513534-1567-0/minicluster-data/master-1/data/instance:
uuid: "5e991661dbe34bc1b470f3eadf12fb36"
format_stamp: "Formatted at 2026-09-23 22:14:48 on dist-test-slave-s2fj"
server_key: "e222852e0fd4493d85191b36bfb57f8d"
server_key_iv: "bf4f031633cfdf88207d76e1e92ab72d"
server_key_version: "encryptionkey@0"
I20260923 22:14:48.234475 14922 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskRiQe_a/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790201539513534-1567-0/minicluster-data/master-1/wal/instance:
uuid: "5e991661dbe34bc1b470f3eadf12fb36"
format_stamp: "Formatted at 2026-09-23 22:14:48 on dist-test-slave-s2fj"
server_key: "e222852e0fd4493d85191b36bfb57f8d"
server_key_iv: "bf4f031633cfdf88207d76e1e92ab72d"
server_key_version: "encryptionkey@0"
I20260923 22:14:48.235670 14922 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.001s
I20260923 22:14:48.236353 14937 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260923 22:14:48.236502 14922 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.000s
I20260923 22:14:48.236593 14922 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskRiQe_a/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790201539513534-1567-0/minicluster-data/master-1/data,/tmp/dist-test-taskRiQe_a/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790201539513534-1567-0/minicluster-data/master-1/wal
uuid: "5e991661dbe34bc1b470f3eadf12fb36"
format_stamp: "Formatted at 2026-09-23 22:14:48 on dist-test-slave-s2fj"
server_key: "e222852e0fd4493d85191b36bfb57f8d"
server_key_iv: "bf4f031633cfdf88207d76e1e92ab72d"
server_key_version: "encryptionkey@0"
I20260923 22:14:48.236702 14922 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskRiQe_a/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790201539513534-1567-0/minicluster-data/master-1/wal
metadata directory: /tmp/dist-test-taskRiQe_a/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790201539513534-1567-0/minicluster-data/master-1/wal
1 data directories: /tmp/dist-test-taskRiQe_a/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790201539513534-1567-0/minicluster-data/master-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260923 22:14:48.245823 14922 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260923 22:14:48.247366 14922 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260923 22:14:48.247506 14922 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260923 22:14:48.250481 14915 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.1.135.253" port: 37899 } attempt: 2
I20260923 22:14:48.250900 14918 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 22:14:48.250748 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.135.253:37899 (local address 127.0.0.1:55238)
0923 22:14:48.250823 (+ 75us) negotiation.cc:107] Waiting for socket to connect
0923 22:14:48.250864 (+ 41us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.135.253:37899: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":49}
I20260923 22:14:48.251267 14918 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 22:14:48.251175 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.135.253:37899 (local address 127.0.0.1:55244)
0923 22:14:48.251223 (+ 48us) negotiation.cc:107] Waiting for socket to connect
0923 22:14:48.251247 (+ 24us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.135.253:37899: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":34}
W20260923 22:14:48.251385 14915 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.1.135.253:37899: Network error: Client connection negotiation failed: client connection to 127.1.135.253:37899: connect: Connection refused (error 111)
I20260923 22:14:48.251518 14922 rpc_server.cc:307] RPC server started. Bound to: 127.1.135.253:37899
I20260923 22:14:48.251595 14989 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.1.135.253:37899 every 8 connection(s)
I20260923 22:14:48.251857 14922 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskRiQe_a/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790201539513534-1567-0/minicluster-data/master-1/data/info.pb
I20260923 22:14:48.252619 14990 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260923 22:14:48.253826 14990 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.1.135.254" port: 39543 } has no permanent_uuid. Determining permanent_uuid...
I20260923 22:14:48.256189 1567 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskRiQe_a/build/release/bin/kudu as pid 14922
I20260923 22:14:48.256305 1567 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskRiQe_a/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790201539513534-1567-0/minicluster-data/master-1/wal/instance
I20260923 22:14:48.256488 1567 external_mini_cluster.cc:1428] Setting key c808af0425fe6317af33311c959f55a7
Sep 23 22:14:48 dist-test-slave-s2fj krb5kdc[14834](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790201688, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.135.253@KRBTEST.COM for kudu/127.1.135.254@KRBTEST.COM
I20260923 22:14:48.263525 14919 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 22:14:48.254396 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40938 (local address 127.1.135.254:39543)
0923 22:14:48.254453 (+ 57us) server_negotiation.cc:206] Beginning negotiation
0923 22:14:48.254455 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0923 22:14:48.254652 (+ 197us) server_negotiation.cc:407] Connection header received
0923 22:14:48.255109 (+ 457us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 22:14:48.255110 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 22:14:48.255133 (+ 23us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 22:14:48.255188 (+ 55us) server_negotiation.cc:226] Negotiated authn=SASL
0923 22:14:48.255778 (+ 590us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 22:14:48.256514 (+ 736us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 22:14:48.258012 (+ 1498us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 22:14:48.258369 (+ 357us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 22:14:48.258447 (+ 78us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 22:14:48.260826 (+ 2379us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0923 22:14:48.260827 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0923 22:14:48.260828 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0923 22:14:48.260834 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0923 22:14:48.262150 (+ 1316us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0923 22:14:48.262615 (+ 465us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0923 22:14:48.262616 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0923 22:14:48.262616 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0923 22:14:48.262662 (+ 46us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0923 22:14:48.262921 (+ 259us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0923 22:14:48.262922 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0923 22:14:48.262922 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0923 22:14:48.263044 (+ 122us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0923 22:14:48.263102 (+ 58us) server_negotiation.cc:1035] Waiting for connection context
0923 22:14:48.263413 (+ 311us) server_negotiation.cc:299] Negotiation successful
0923 22:14:48.263473 (+ 60us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":38}
I20260923 22:14:48.263778 14992 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 22:14:48.254368 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.135.254:39543 (local address 127.0.0.1:40938)
0923 22:14:48.254530 (+ 162us) negotiation.cc:107] Waiting for socket to connect
0923 22:14:48.254566 (+ 36us) client_negotiation.cc:174] Beginning negotiation
0923 22:14:48.255026 (+ 460us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0923 22:14:48.255243 (+ 217us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0923 22:14:48.255246 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0923 22:14:48.255456 (+ 210us) client_negotiation.cc:189] Negotiated authn=SASL
0923 22:14:48.255641 (+ 185us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 22:14:48.255644 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 22:14:48.256630 (+ 986us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 22:14:48.256631 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 22:14:48.257915 (+ 1284us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 22:14:48.257918 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 22:14:48.258455 (+ 537us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 22:14:48.258455 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 22:14:48.258549 (+ 94us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 22:14:48.259055 (+ 506us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0923 22:14:48.259062 (+ 7us) client_negotiation.cc:656] calling sasl_client_start()
0923 22:14:48.260739 (+ 1677us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0923 22:14:48.262232 (+ 1493us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0923 22:14:48.262239 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0923 22:14:48.262241 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0923 22:14:48.262541 (+ 300us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0923 22:14:48.262727 (+ 186us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0923 22:14:48.262728 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0923 22:14:48.262729 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0923 22:14:48.262857 (+ 128us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0923 22:14:48.263120 (+ 263us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0923 22:14:48.263126 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0923 22:14:48.263277 (+ 151us) client_negotiation.cc:769] Sending connection context
0923 22:14:48.263397 (+ 120us) client_negotiation.cc:240] Negotiation successful
0923 22:14:48.263561 (+ 164us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":128,"thread_start_us":93,"threads_started":1}
I20260923 22:14:48.264343 14990 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.1.135.253" port: 37899 } has no permanent_uuid. Determining permanent_uuid...
WARNING: no policy specified for kudu/127.1.135.252@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.1.135.252@KRBTEST.COM" created.
Sep 23 22:14:48 dist-test-slave-s2fj krb5kdc[14834](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790201688, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.135.253@KRBTEST.COM for kudu/127.1.135.253@KRBTEST.COM
I20260923 22:14:48.271625 14992 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 22:14:48.264611 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.135.253:37899 (local address 127.0.0.1:55252)
0923 22:14:48.264789 (+ 178us) negotiation.cc:107] Waiting for socket to connect
0923 22:14:48.264800 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0923 22:14:48.264842 (+ 42us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0923 22:14:48.264990 (+ 148us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0923 22:14:48.264991 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0923 22:14:48.265126 (+ 135us) client_negotiation.cc:189] Negotiated authn=SASL
0923 22:14:48.265171 (+ 45us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 22:14:48.265172 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 22:14:48.266064 (+ 892us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 22:14:48.266065 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 22:14:48.266534 (+ 469us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 22:14:48.266535 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 22:14:48.267064 (+ 529us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 22:14:48.267065 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 22:14:48.267129 (+ 64us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 22:14:48.267557 (+ 428us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0923 22:14:48.267562 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0923 22:14:48.268759 (+ 1197us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0923 22:14:48.270711 (+ 1952us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0923 22:14:48.270712 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0923 22:14:48.270713 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0923 22:14:48.270859 (+ 146us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0923 22:14:48.271095 (+ 236us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0923 22:14:48.271096 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0923 22:14:48.271096 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0923 22:14:48.271141 (+ 45us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0923 22:14:48.271444 (+ 303us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0923 22:14:48.271444 (+ 0us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0923 22:14:48.271472 (+ 28us) client_negotiation.cc:769] Sending connection context
0923 22:14:48.271517 (+ 45us) client_negotiation.cc:240] Negotiation successful
0923 22:14:48.271575 (+ 58us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":163}
I20260923 22:14:48.271768 14994 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 22:14:48.264670 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55252 (local address 127.1.135.253:37899)
0923 22:14:48.264819 (+ 149us) server_negotiation.cc:206] Beginning negotiation
0923 22:14:48.264823 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0923 22:14:48.264846 (+ 23us) server_negotiation.cc:407] Connection header received
0923 22:14:48.264916 (+ 70us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 22:14:48.264917 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 22:14:48.264927 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 22:14:48.264987 (+ 60us) server_negotiation.cc:226] Negotiated authn=SASL
0923 22:14:48.265232 (+ 245us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 22:14:48.265988 (+ 756us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 22:14:48.266631 (+ 643us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 22:14:48.266934 (+ 303us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 22:14:48.267064 (+ 130us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 22:14:48.268835 (+ 1771us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0923 22:14:48.268843 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0923 22:14:48.268843 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0923 22:14:48.268850 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0923 22:14:48.270636 (+ 1786us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0923 22:14:48.270938 (+ 302us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0923 22:14:48.270941 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0923 22:14:48.270942 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0923 22:14:48.271001 (+ 59us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0923 22:14:48.271213 (+ 212us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0923 22:14:48.271213 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0923 22:14:48.271214 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0923 22:14:48.271379 (+ 165us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0923 22:14:48.271463 (+ 84us) server_negotiation.cc:1035] Waiting for connection context
0923 22:14:48.271552 (+ 89us) server_negotiation.cc:299] Negotiation successful
0923 22:14:48.271636 (+ 84us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":3,"server-negotiator.queue_time_us":124,"thread_start_us":67,"threads_started":1}
I20260923 22:14:48.272384 14990 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.1.135.252" port: 36511 } has no permanent_uuid. Determining permanent_uuid...
I20260923 22:14:48.272692 14992 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 22:14:48.272577 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.135.252:36511 (local address 127.0.0.1:56644)
0923 22:14:48.272628 (+ 51us) negotiation.cc:107] Waiting for socket to connect
0923 22:14:48.272665 (+ 37us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.135.252:36511: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":38}
W20260923 22:14:48.272846 14939 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.1.135.252:36511: connect: Connection refused (error 111)
I20260923 22:14:48.273241 14992 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 22:14:48.273146 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.135.252:36511 (local address 127.0.0.1:56660)
0923 22:14:48.273205 (+ 59us) negotiation.cc:107] Waiting for socket to connect
0923 22:14:48.273226 (+ 21us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.135.252:36511: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":48}
W20260923 22:14:48.273362 14990 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.1.135.252:36511: Network error: Client connection negotiation failed: client connection to 127.1.135.252:36511: connect: Connection refused (error 111)
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.1.135.252 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskRiQe_a/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790201539513534-1567-0/krb5kdc/kudu_127.1.135.252.keytab.
Entry for principal kudu/127.1.135.252 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskRiQe_a/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790201539513534-1567-0/krb5kdc/kudu_127.1.135.252.keytab.
I20260923 22:14:48.280972 1567 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskRiQe_a/build/release/bin/kudu
/tmp/dist-test-taskRiQe_a/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskRiQe_a/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790201539513534-1567-0/minicluster-data/master-2/wal
--fs_data_dirs=/tmp/dist-test-taskRiQe_a/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790201539513534-1567-0/minicluster-data/master-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskRiQe_a/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790201539513534-1567-0/minicluster-data/master-2/logs
--server_dump_info_path=/tmp/dist-test-taskRiQe_a/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790201539513534-1567-0/minicluster-data/master-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.1.135.252:36511
--webserver_interface=127.1.135.252
--webserver_port=0
--keytab_file=/tmp/dist-test-taskRiQe_a/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790201539513534-1567-0/krb5kdc/kudu_127.1.135.252.keytab
--principal=kudu/127.1.135.252
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.1.135.212:37859
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.1.135.254:39543,127.1.135.253:37899,127.1.135.252:36511
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskRiQe_a/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790201539513534-1567-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskRiQe_a/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790201539513534-1567-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskRiQe_a/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790201539513534-1567-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260923 22:14:48.322609 14990 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.1.135.252" port: 36511 } attempt: 1
I20260923 22:14:48.323179 14992 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 22:14:48.323045 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.135.252:36511 (local address 127.0.0.1:56662)
0923 22:14:48.323107 (+ 62us) negotiation.cc:107] Waiting for socket to connect
0923 22:14:48.323148 (+ 41us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.135.252:36511: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":39}
I20260923 22:14:48.323535 14992 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 22:14:48.323442 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.135.252:36511 (local address 127.0.0.1:56676)
0923 22:14:48.323489 (+ 47us) negotiation.cc:107] Waiting for socket to connect
0923 22:14:48.323514 (+ 25us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.135.252:36511: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":36}
W20260923 22:14:48.323709 14990 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.1.135.252:36511: Network error: Client connection negotiation failed: client connection to 127.1.135.252:36511: connect: Connection refused (error 111)
I20260923 22:14:48.342615 14915 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.1.135.253" port: 37899 } attempt: 3
Sep 23 22:14:48 dist-test-slave-s2fj krb5kdc[14834](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790201688, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.135.254@KRBTEST.COM for kudu/127.1.135.253@KRBTEST.COM
I20260923 22:14:48.349535 14918 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 22:14:48.342933 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.135.253:37899 (local address 127.0.0.1:55258)
0923 22:14:48.342999 (+ 66us) negotiation.cc:107] Waiting for socket to connect
0923 22:14:48.343009 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0923 22:14:48.343077 (+ 68us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0923 22:14:48.343235 (+ 158us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0923 22:14:48.343236 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0923 22:14:48.343446 (+ 210us) client_negotiation.cc:189] Negotiated authn=SASL
0923 22:14:48.343524 (+ 78us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 22:14:48.343526 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 22:14:48.344224 (+ 698us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 22:14:48.344225 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 22:14:48.345063 (+ 838us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 22:14:48.345066 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 22:14:48.345378 (+ 312us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 22:14:48.345379 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 22:14:48.345455 (+ 76us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 22:14:48.345829 (+ 374us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0923 22:14:48.345835 (+ 6us) client_negotiation.cc:656] calling sasl_client_start()
0923 22:14:48.347040 (+ 1205us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0923 22:14:48.348566 (+ 1526us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0923 22:14:48.348567 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0923 22:14:48.348567 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0923 22:14:48.348739 (+ 172us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0923 22:14:48.348932 (+ 193us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0923 22:14:48.348933 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0923 22:14:48.348933 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0923 22:14:48.348988 (+ 55us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0923 22:14:48.349229 (+ 241us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0923 22:14:48.349230 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0923 22:14:48.349273 (+ 43us) client_negotiation.cc:769] Sending connection context
0923 22:14:48.349357 (+ 84us) client_negotiation.cc:240] Negotiation successful
0923 22:14:48.349447 (+ 90us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":48}
I20260923 22:14:48.349534 14994 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 22:14:48.343006 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55258 (local address 127.1.135.253:37899)
0923 22:14:48.343098 (+ 92us) server_negotiation.cc:206] Beginning negotiation
0923 22:14:48.343100 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0923 22:14:48.343108 (+ 8us) server_negotiation.cc:407] Connection header received
0923 22:14:48.343130 (+ 22us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 22:14:48.343130 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 22:14:48.343140 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 22:14:48.343196 (+ 56us) server_negotiation.cc:226] Negotiated authn=SASL
0923 22:14:48.343599 (+ 403us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 22:14:48.344148 (+ 549us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 22:14:48.345141 (+ 993us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 22:14:48.345314 (+ 173us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 22:14:48.345354 (+ 40us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 22:14:48.347129 (+ 1775us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0923 22:14:48.347131 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0923 22:14:48.347132 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0923 22:14:48.347139 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0923 22:14:48.348466 (+ 1327us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0923 22:14:48.348809 (+ 343us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0923 22:14:48.348810 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0923 22:14:48.348811 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0923 22:14:48.348866 (+ 55us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0923 22:14:48.349057 (+ 191us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0923 22:14:48.349058 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0923 22:14:48.349058 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0923 22:14:48.349162 (+ 104us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0923 22:14:48.349236 (+ 74us) server_negotiation.cc:1035] Waiting for connection context
0923 22:14:48.349379 (+ 143us) server_negotiation.cc:299] Negotiation successful
0923 22:14:48.349448 (+ 69us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":59}
I20260923 22:14:48.349957 14915 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.1.135.252" port: 36511 } has no permanent_uuid. Determining permanent_uuid...
I20260923 22:14:48.350279 14918 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 22:14:48.350186 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.135.252:36511 (local address 127.0.0.1:56688)
0923 22:14:48.350232 (+ 46us) negotiation.cc:107] Waiting for socket to connect
0923 22:14:48.350258 (+ 26us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.135.252:36511: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":32}
I20260923 22:14:48.350615 14918 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 22:14:48.350522 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.135.252:36511 (local address 127.0.0.1:56698)
0923 22:14:48.350567 (+ 45us) negotiation.cc:107] Waiting for socket to connect
0923 22:14:48.350596 (+ 29us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.135.252:36511: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":32}
W20260923 22:14:48.350703 14915 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.1.135.252:36511: Network error: Client connection negotiation failed: client connection to 127.1.135.252:36511: connect: Connection refused (error 111)
W20260923 22:14:48.359498 14997 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260923 22:14:48.359658 14997 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260923 22:14:48.359679 14997 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260923 22:14:48.360821 14997 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260923 22:14:48.360862 14997 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260923 22:14:48.360875 14997 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260923 22:14:48.360910 14997 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260923 22:14:48.360936 14997 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260923 22:14:48.362308 14997 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.1.135.212:37859
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskRiQe_a/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790201539513534-1567-0/minicluster-data/master-2/data
--fs_wal_dir=/tmp/dist-test-taskRiQe_a/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790201539513534-1567-0/minicluster-data/master-2/wal
--ipki_ca_key_size=768
--master_addresses=127.1.135.254:39543,127.1.135.253:37899,127.1.135.252:36511
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskRiQe_a/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790201539513534-1567-0/krb5kdc/kudu_127.1.135.252.keytab
--principal=kudu/127.1.135.252
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.1.135.252:36511
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskRiQe_a/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790201539513534-1567-0/minicluster-data/master-2/data/info.pb
--webserver_interface=127.1.135.252
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.14997
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskRiQe_a/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790201539513534-1567-0/minicluster-data/master-2/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 23 Sep 2026 20:53:37 UTC on ccycloud.quasar-rkviyx.root.comops.site
build id 4668083
I20260923 22:14:48.362514 14997 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260923 22:14:48.362707 14997 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260923 22:14:48.367146 15005 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260923 22:14:48.367154 15002 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260923 22:14:48.367154 15003 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260923 22:14:48.367579 14997 server_base.cc:1034] running on GCE node
I20260923 22:14:48.367834 14997 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260923 22:14:48.368171 14997 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260923 22:14:48.369328 14997 hybrid_clock.cc:648] HybridClock initialized: now 1790201688369310 us; error 28 us; skew 500 ppm
Sep 23 22:14:48 dist-test-slave-s2fj krb5kdc[14834](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790201688, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.135.252@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260923 22:14:48.371558 14997 init.cc:377] Logged in from keytab as kudu/127.1.135.252@KRBTEST.COM (short username kudu)
I20260923 22:14:48.371919 14997 webserver.cc:460] Webserver started at http://127.1.135.252:36339/ using document root <none> and password file <none>
I20260923 22:14:48.372149 14997 fs_manager.cc:359] Metadata directory not provided
I20260923 22:14:48.372206 14997 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260923 22:14:48.372298 14997 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260923 22:14:48.373211 14997 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskRiQe_a/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790201539513534-1567-0/minicluster-data/master-2/data/instance:
uuid: "972f82227cf14c55b0c7527c0e670a82"
format_stamp: "Formatted at 2026-09-23 22:14:48 on dist-test-slave-s2fj"
server_key: "b16dde4f2c6346ea2ae784fa0658d4be"
server_key_iv: "ea51769f8aaa7b2eaea4ea60c81d1b63"
server_key_version: "encryptionkey@0"
I20260923 22:14:48.373512 14997 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskRiQe_a/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790201539513534-1567-0/minicluster-data/master-2/wal/instance:
uuid: "972f82227cf14c55b0c7527c0e670a82"
format_stamp: "Formatted at 2026-09-23 22:14:48 on dist-test-slave-s2fj"
server_key: "b16dde4f2c6346ea2ae784fa0658d4be"
server_key_iv: "ea51769f8aaa7b2eaea4ea60c81d1b63"
server_key_version: "encryptionkey@0"
I20260923 22:14:48.374652 14997 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.002s
I20260923 22:14:48.375249 15012 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260923 22:14:48.375393 14997 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.001s
I20260923 22:14:48.375468 14997 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskRiQe_a/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790201539513534-1567-0/minicluster-data/master-2/data,/tmp/dist-test-taskRiQe_a/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790201539513534-1567-0/minicluster-data/master-2/wal
uuid: "972f82227cf14c55b0c7527c0e670a82"
format_stamp: "Formatted at 2026-09-23 22:14:48 on dist-test-slave-s2fj"
server_key: "b16dde4f2c6346ea2ae784fa0658d4be"
server_key_iv: "ea51769f8aaa7b2eaea4ea60c81d1b63"
server_key_version: "encryptionkey@0"
I20260923 22:14:48.375516 14997 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskRiQe_a/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790201539513534-1567-0/minicluster-data/master-2/wal
metadata directory: /tmp/dist-test-taskRiQe_a/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790201539513534-1567-0/minicluster-data/master-2/wal
1 data directories: /tmp/dist-test-taskRiQe_a/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790201539513534-1567-0/minicluster-data/master-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260923 22:14:48.381909 14915 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.1.135.252" port: 36511 } attempt: 1
I20260923 22:14:48.382246 14918 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 22:14:48.382134 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.135.252:36511 (local address 127.0.0.1:56714)
0923 22:14:48.382185 (+ 51us) negotiation.cc:107] Waiting for socket to connect
0923 22:14:48.382218 (+ 33us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.135.252:36511: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":32}
I20260923 22:14:48.382562 14918 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 22:14:48.382489 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.135.252:36511 (local address 127.0.0.1:56728)
0923 22:14:48.382526 (+ 37us) negotiation.cc:107] Waiting for socket to connect
0923 22:14:48.382547 (+ 21us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.135.252:36511: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":27}
W20260923 22:14:48.382668 14915 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.1.135.252:36511: Network error: Client connection negotiation failed: client connection to 127.1.135.252:36511: connect: Connection refused (error 111)
I20260923 22:14:48.391845 14990 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.1.135.252" port: 36511 } attempt: 2
I20260923 22:14:48.392210 14992 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 22:14:48.392099 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.135.252:36511 (local address 127.0.0.1:56742)
0923 22:14:48.392152 (+ 53us) negotiation.cc:107] Waiting for socket to connect
0923 22:14:48.392182 (+ 30us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.135.252:36511: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":36}
I20260923 22:14:48.392551 14992 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 22:14:48.392466 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.135.252:36511 (local address 127.0.0.1:56752)
0923 22:14:48.392508 (+ 42us) negotiation.cc:107] Waiting for socket to connect
0923 22:14:48.392533 (+ 25us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.135.252:36511: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":29}
W20260923 22:14:48.392673 14990 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.1.135.252:36511: Network error: Client connection negotiation failed: client connection to 127.1.135.252:36511: connect: Connection refused (error 111)
I20260923 22:14:48.402297 14997 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260923 22:14:48.403951 14997 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260923 22:14:48.404166 14997 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260923 22:14:48.407758 14997 rpc_server.cc:307] RPC server started. Bound to: 127.1.135.252:36511
I20260923 22:14:48.407790 15064 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.1.135.252:36511 every 8 connection(s)
I20260923 22:14:48.408150 14997 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskRiQe_a/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790201539513534-1567-0/minicluster-data/master-2/data/info.pb
I20260923 22:14:48.408944 15065 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260923 22:14:48.410365 15065 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.1.135.254" port: 39543 } has no permanent_uuid. Determining permanent_uuid...
I20260923 22:14:48.415094 1567 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskRiQe_a/build/release/bin/kudu as pid 14997
I20260923 22:14:48.415212 1567 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskRiQe_a/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790201539513534-1567-0/minicluster-data/master-2/wal/instance
I20260923 22:14:48.415396 1567 external_mini_cluster.cc:1428] Setting key 9b47f46506496cc000cdaed02c72fe94
Sep 23 22:14:48 dist-test-slave-s2fj krb5kdc[14834](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790201688, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.135.252@KRBTEST.COM for kudu/127.1.135.254@KRBTEST.COM
I20260923 22:14:48.418915 14919 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 22:14:48.410995 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40950 (local address 127.1.135.254:39543)
0923 22:14:48.411048 (+ 53us) server_negotiation.cc:206] Beginning negotiation
0923 22:14:48.411050 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0923 22:14:48.411249 (+ 199us) server_negotiation.cc:407] Connection header received
0923 22:14:48.411689 (+ 440us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 22:14:48.411690 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 22:14:48.411699 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 22:14:48.411746 (+ 47us) server_negotiation.cc:226] Negotiated authn=SASL
0923 22:14:48.412329 (+ 583us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 22:14:48.412778 (+ 449us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 22:14:48.413842 (+ 1064us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 22:14:48.414017 (+ 175us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 22:14:48.414070 (+ 53us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 22:14:48.416213 (+ 2143us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0923 22:14:48.416214 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0923 22:14:48.416215 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0923 22:14:48.416221 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0923 22:14:48.417556 (+ 1335us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0923 22:14:48.417955 (+ 399us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0923 22:14:48.417956 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0923 22:14:48.417957 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0923 22:14:48.418013 (+ 56us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0923 22:14:48.418265 (+ 252us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0923 22:14:48.418265 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0923 22:14:48.418266 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0923 22:14:48.418368 (+ 102us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0923 22:14:48.418438 (+ 70us) server_negotiation.cc:1035] Waiting for connection context
0923 22:14:48.418699 (+ 261us) server_negotiation.cc:299] Negotiation successful
0923 22:14:48.418804 (+ 105us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":33}
I20260923 22:14:48.419270 15067 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 22:14:48.410985 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.135.254:39543 (local address 127.0.0.1:40950)
0923 22:14:48.411158 (+ 173us) negotiation.cc:107] Waiting for socket to connect
0923 22:14:48.411189 (+ 31us) client_negotiation.cc:174] Beginning negotiation
0923 22:14:48.411613 (+ 424us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0923 22:14:48.411793 (+ 180us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0923 22:14:48.411795 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0923 22:14:48.412003 (+ 208us) client_negotiation.cc:189] Negotiated authn=SASL
0923 22:14:48.412256 (+ 253us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 22:14:48.412260 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 22:14:48.412849 (+ 589us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 22:14:48.412849 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 22:14:48.413766 (+ 917us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 22:14:48.413768 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 22:14:48.414105 (+ 337us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 22:14:48.414106 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 22:14:48.414204 (+ 98us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 22:14:48.414568 (+ 364us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0923 22:14:48.414575 (+ 7us) client_negotiation.cc:656] calling sasl_client_start()
0923 22:14:48.416148 (+ 1573us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0923 22:14:48.417655 (+ 1507us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0923 22:14:48.417661 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0923 22:14:48.417663 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0923 22:14:48.417877 (+ 214us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0923 22:14:48.418078 (+ 201us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0923 22:14:48.418078 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0923 22:14:48.418079 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0923 22:14:48.418189 (+ 110us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0923 22:14:48.418432 (+ 243us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0923 22:14:48.418438 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0923 22:14:48.418565 (+ 127us) client_negotiation.cc:769] Sending connection context
0923 22:14:48.418956 (+ 391us) client_negotiation.cc:240] Negotiation successful
0923 22:14:48.419114 (+ 158us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":140,"thread_start_us":93,"threads_started":1}
I20260923 22:14:48.419759 15065 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.1.135.253" port: 37899 } has no permanent_uuid. Determining permanent_uuid...
Sep 23 22:14:48 dist-test-slave-s2fj krb5kdc[14834](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790201687, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.1.135.252@KRBTEST.COM
Sep 23 22:14:48 dist-test-slave-s2fj krb5kdc[14834](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790201688, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.135.252@KRBTEST.COM for kudu/127.1.135.253@KRBTEST.COM
I20260923 22:14:48.423871 15069 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 22:14:48.415758 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56754 (local address 127.1.135.252:36511)
0923 22:14:48.415878 (+ 120us) server_negotiation.cc:206] Beginning negotiation
0923 22:14:48.415882 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0923 22:14:48.416045 (+ 163us) server_negotiation.cc:407] Connection header received
0923 22:14:48.416132 (+ 87us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 22:14:48.416135 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 22:14:48.416154 (+ 19us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 22:14:48.416230 (+ 76us) server_negotiation.cc:226] Negotiated authn=SASL
0923 22:14:48.416776 (+ 546us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 22:14:48.417403 (+ 627us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 22:14:48.418107 (+ 704us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 22:14:48.418397 (+ 290us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 22:14:48.418471 (+ 74us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 22:14:48.420875 (+ 2404us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0923 22:14:48.420882 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0923 22:14:48.420883 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0923 22:14:48.420889 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0923 22:14:48.422545 (+ 1656us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0923 22:14:48.422927 (+ 382us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0923 22:14:48.422929 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0923 22:14:48.422929 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0923 22:14:48.422987 (+ 58us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0923 22:14:48.423190 (+ 203us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0923 22:14:48.423190 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0923 22:14:48.423191 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0923 22:14:48.423301 (+ 110us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0923 22:14:48.423402 (+ 101us) server_negotiation.cc:1035] Waiting for connection context
0923 22:14:48.423713 (+ 311us) server_negotiation.cc:299] Negotiation successful
0923 22:14:48.423787 (+ 74us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":93,"thread_start_us":62,"threads_started":1}
I20260923 22:14:48.426318 14994 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 22:14:48.420095 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55262 (local address 127.1.135.253:37899)
0923 22:14:48.420142 (+ 47us) server_negotiation.cc:206] Beginning negotiation
0923 22:14:48.420144 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0923 22:14:48.420154 (+ 10us) server_negotiation.cc:407] Connection header received
0923 22:14:48.420211 (+ 57us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 22:14:48.420211 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 22:14:48.420220 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 22:14:48.420274 (+ 54us) server_negotiation.cc:226] Negotiated authn=SASL
0923 22:14:48.420525 (+ 251us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 22:14:48.421074 (+ 549us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 22:14:48.421718 (+ 644us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 22:14:48.421931 (+ 213us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 22:14:48.422012 (+ 81us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 22:14:48.423768 (+ 1756us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0923 22:14:48.423769 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0923 22:14:48.423770 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0923 22:14:48.423776 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0923 22:14:48.425357 (+ 1581us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0923 22:14:48.425664 (+ 307us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0923 22:14:48.425665 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0923 22:14:48.425665 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0923 22:14:48.425720 (+ 55us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0923 22:14:48.425881 (+ 161us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0923 22:14:48.425881 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0923 22:14:48.425882 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0923 22:14:48.425994 (+ 112us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0923 22:14:48.426058 (+ 64us) server_negotiation.cc:1035] Waiting for connection context
0923 22:14:48.426199 (+ 141us) server_negotiation.cc:299] Negotiation successful
0923 22:14:48.426267 (+ 68us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":23}
I20260923 22:14:48.426348 15067 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 22:14:48.420007 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.135.253:37899 (local address 127.0.0.1:55262)
0923 22:14:48.420099 (+ 92us) negotiation.cc:107] Waiting for socket to connect
0923 22:14:48.420109 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0923 22:14:48.420148 (+ 39us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0923 22:14:48.420278 (+ 130us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0923 22:14:48.420279 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0923 22:14:48.420409 (+ 130us) client_negotiation.cc:189] Negotiated authn=SASL
0923 22:14:48.420449 (+ 40us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 22:14:48.420450 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 22:14:48.421154 (+ 704us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 22:14:48.421155 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 22:14:48.421633 (+ 478us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 22:14:48.421635 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 22:14:48.422013 (+ 378us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 22:14:48.422014 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 22:14:48.422081 (+ 67us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 22:14:48.422437 (+ 356us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0923 22:14:48.422441 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0923 22:14:48.423687 (+ 1246us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0923 22:14:48.425460 (+ 1773us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0923 22:14:48.425460 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0923 22:14:48.425461 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0923 22:14:48.425601 (+ 140us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0923 22:14:48.425783 (+ 182us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0923 22:14:48.425783 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0923 22:14:48.425784 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0923 22:14:48.425827 (+ 43us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0923 22:14:48.426069 (+ 242us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0923 22:14:48.426070 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0923 22:14:48.426107 (+ 37us) client_negotiation.cc:769] Sending connection context
0923 22:14:48.426182 (+ 75us) client_negotiation.cc:240] Negotiation successful
0923 22:14:48.426269 (+ 87us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":79}
I20260923 22:14:48.426640 15065 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.1.135.252" port: 36511 } has no permanent_uuid. Determining permanent_uuid...
Sep 23 22:14:48 dist-test-slave-s2fj krb5kdc[14834](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790201687, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.1.135.254@KRBTEST.COM
Sep 23 22:14:48 dist-test-slave-s2fj krb5kdc[14834](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790201688, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.135.252@KRBTEST.COM for kudu/127.1.135.252@KRBTEST.COM
I20260923 22:14:48.431696 14919 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 22:14:48.424673 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40964 (local address 127.1.135.254:39543)
0923 22:14:48.424730 (+ 57us) server_negotiation.cc:206] Beginning negotiation
0923 22:14:48.424732 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0923 22:14:48.424932 (+ 200us) server_negotiation.cc:407] Connection header received
0923 22:14:48.424955 (+ 23us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 22:14:48.424956 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 22:14:48.424967 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 22:14:48.425024 (+ 57us) server_negotiation.cc:226] Negotiated authn=SASL
0923 22:14:48.425404 (+ 380us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 22:14:48.426017 (+ 613us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 22:14:48.426980 (+ 963us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 22:14:48.427169 (+ 189us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 22:14:48.427243 (+ 74us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 22:14:48.429206 (+ 1963us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0923 22:14:48.429207 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0923 22:14:48.429207 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0923 22:14:48.429214 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0923 22:14:48.430632 (+ 1418us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0923 22:14:48.430987 (+ 355us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0923 22:14:48.430988 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0923 22:14:48.430988 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0923 22:14:48.431056 (+ 68us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0923 22:14:48.431252 (+ 196us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0923 22:14:48.431253 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0923 22:14:48.431253 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0923 22:14:48.431351 (+ 98us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0923 22:14:48.431510 (+ 159us) server_negotiation.cc:1035] Waiting for connection context
0923 22:14:48.431541 (+ 31us) server_negotiation.cc:299] Negotiation successful
0923 22:14:48.431609 (+ 68us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":36}
I20260923 22:14:48.433455 15067 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 22:14:48.426892 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.135.252:36511 (local address 127.0.0.1:56760)
0923 22:14:48.426929 (+ 37us) negotiation.cc:107] Waiting for socket to connect
0923 22:14:48.426937 (+ 8us) client_negotiation.cc:174] Beginning negotiation
0923 22:14:48.426970 (+ 33us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0923 22:14:48.427090 (+ 120us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0923 22:14:48.427091 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0923 22:14:48.427212 (+ 121us) client_negotiation.cc:189] Negotiated authn=SASL
0923 22:14:48.427232 (+ 20us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 22:14:48.427233 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 22:14:48.427995 (+ 762us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 22:14:48.427996 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 22:14:48.428490 (+ 494us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 22:14:48.428492 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 22:14:48.428850 (+ 358us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 22:14:48.428851 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 22:14:48.428895 (+ 44us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 22:14:48.429261 (+ 366us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0923 22:14:48.429265 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0923 22:14:48.430590 (+ 1325us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0923 22:14:48.432311 (+ 1721us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0923 22:14:48.432311 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0923 22:14:48.432312 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0923 22:14:48.432526 (+ 214us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0923 22:14:48.432850 (+ 324us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0923 22:14:48.432851 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0923 22:14:48.432852 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0923 22:14:48.432915 (+ 63us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0923 22:14:48.433140 (+ 225us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0923 22:14:48.433141 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0923 22:14:48.433189 (+ 48us) client_negotiation.cc:769] Sending connection context
0923 22:14:48.433270 (+ 81us) client_negotiation.cc:240] Negotiation successful
0923 22:14:48.433374 (+ 104us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":25}
I20260923 22:14:48.434041 15069 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 22:14:48.426939 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56760 (local address 127.1.135.252:36511)
0923 22:14:48.426970 (+ 31us) server_negotiation.cc:206] Beginning negotiation
0923 22:14:48.426972 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0923 22:14:48.426979 (+ 7us) server_negotiation.cc:407] Connection header received
0923 22:14:48.427018 (+ 39us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 22:14:48.427019 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 22:14:48.427027 (+ 8us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 22:14:48.427080 (+ 53us) server_negotiation.cc:226] Negotiated authn=SASL
0923 22:14:48.427309 (+ 229us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 22:14:48.427899 (+ 590us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 22:14:48.428584 (+ 685us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 22:14:48.428775 (+ 191us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 22:14:48.428841 (+ 66us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 22:14:48.430670 (+ 1829us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0923 22:14:48.430671 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0923 22:14:48.430671 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0923 22:14:48.430677 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0923 22:14:48.432229 (+ 1552us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0923 22:14:48.432595 (+ 366us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0923 22:14:48.432611 (+ 16us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0923 22:14:48.432611 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0923 22:14:48.432671 (+ 60us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0923 22:14:48.432991 (+ 320us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0923 22:14:48.432991 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0923 22:14:48.432992 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0923 22:14:48.433076 (+ 84us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0923 22:14:48.433132 (+ 56us) server_negotiation.cc:1035] Waiting for connection context
0923 22:14:48.433285 (+ 153us) server_negotiation.cc:299] Negotiation successful
0923 22:14:48.433994 (+ 709us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":647,"server-negotiator.queue_time_us":18}
I20260923 22:14:48.435437 15065 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 972f82227cf14c55b0c7527c0e670a82: Bootstrap starting.
I20260923 22:14:48.436475 15065 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 972f82227cf14c55b0c7527c0e670a82: Neither blocks nor log segments found. Creating new log.
Sep 23 22:14:48 dist-test-slave-s2fj krb5kdc[14834](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790201687, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.1.135.253@KRBTEST.COM
I20260923 22:14:48.436781 15065 log.cc:824] T 00000000000000000000000000000000 P 972f82227cf14c55b0c7527c0e670a82: Log is configured to *not* fsync() on all Append() calls
I20260923 22:14:48.437479 15065 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 972f82227cf14c55b0c7527c0e670a82: No bootstrap required, opened a new log
I20260923 22:14:48.438851 15065 raft_consensus.cc:348] T 00000000000000000000000000000000 P 972f82227cf14c55b0c7527c0e670a82 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d5923ad1b2bd4b5a997a7936eb1ca7f6" member_type: VOTER last_known_addr { host: "127.1.135.254" port: 39543 } } peers { permanent_uuid: "5e991661dbe34bc1b470f3eadf12fb36" member_type: VOTER last_known_addr { host: "127.1.135.253" port: 37899 } } peers { permanent_uuid: "972f82227cf14c55b0c7527c0e670a82" member_type: VOTER last_known_addr { host: "127.1.135.252" port: 36511 } }
I20260923 22:14:48.438999 15065 raft_consensus.cc:374] T 00000000000000000000000000000000 P 972f82227cf14c55b0c7527c0e670a82 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260923 22:14:48.439040 15065 raft_consensus.cc:729] T 00000000000000000000000000000000 P 972f82227cf14c55b0c7527c0e670a82 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 972f82227cf14c55b0c7527c0e670a82, State: Initialized, Role: FOLLOWER
I20260923 22:14:48.439145 15065 consensus_queue.cc:260] T 00000000000000000000000000000000 P 972f82227cf14c55b0c7527c0e670a82 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d5923ad1b2bd4b5a997a7936eb1ca7f6" member_type: VOTER last_known_addr { host: "127.1.135.254" port: 39543 } } peers { permanent_uuid: "5e991661dbe34bc1b470f3eadf12fb36" member_type: VOTER last_known_addr { host: "127.1.135.253" port: 37899 } } peers { permanent_uuid: "972f82227cf14c55b0c7527c0e670a82" member_type: VOTER last_known_addr { host: "127.1.135.252" port: 36511 } }
I20260923 22:14:48.439381 15071 sys_catalog.cc:455] T 00000000000000000000000000000000 P 972f82227cf14c55b0c7527c0e670a82 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d5923ad1b2bd4b5a997a7936eb1ca7f6" member_type: VOTER last_known_addr { host: "127.1.135.254" port: 39543 } } peers { permanent_uuid: "5e991661dbe34bc1b470f3eadf12fb36" member_type: VOTER last_known_addr { host: "127.1.135.253" port: 37899 } } peers { permanent_uuid: "972f82227cf14c55b0c7527c0e670a82" member_type: VOTER last_known_addr { host: "127.1.135.252" port: 36511 } } }
I20260923 22:14:48.439430 15065 sys_catalog.cc:564] T 00000000000000000000000000000000 P 972f82227cf14c55b0c7527c0e670a82 [sys.catalog]: configured and running, proceeding with master startup.
I20260923 22:14:48.439455 15071 sys_catalog.cc:458] T 00000000000000000000000000000000 P 972f82227cf14c55b0c7527c0e670a82 [sys.catalog]: This master's current role is: FOLLOWER
I20260923 22:14:48.439782 14994 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 22:14:48.432697 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55278 (local address 127.1.135.253:37899)
0923 22:14:48.432734 (+ 37us) server_negotiation.cc:206] Beginning negotiation
0923 22:14:48.432736 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0923 22:14:48.432743 (+ 7us) server_negotiation.cc:407] Connection header received
0923 22:14:48.432760 (+ 17us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 22:14:48.432761 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 22:14:48.432768 (+ 7us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 22:14:48.432811 (+ 43us) server_negotiation.cc:226] Negotiated authn=SASL
0923 22:14:48.433390 (+ 579us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 22:14:48.433888 (+ 498us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 22:14:48.434625 (+ 737us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 22:14:48.434786 (+ 161us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 22:14:48.434831 (+ 45us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 22:14:48.437188 (+ 2357us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0923 22:14:48.437190 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0923 22:14:48.437190 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0923 22:14:48.437196 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0923 22:14:48.438488 (+ 1292us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0923 22:14:48.438833 (+ 345us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0923 22:14:48.438834 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0923 22:14:48.438834 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0923 22:14:48.438884 (+ 50us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0923 22:14:48.439097 (+ 213us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0923 22:14:48.439097 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0923 22:14:48.439098 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0923 22:14:48.439205 (+ 107us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0923 22:14:48.439384 (+ 179us) server_negotiation.cc:1035] Waiting for connection context
0923 22:14:48.439583 (+ 199us) server_negotiation.cc:299] Negotiation successful
0923 22:14:48.439658 (+ 75us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":16}
W20260923 22:14:48.441550 15082 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 972f82227cf14c55b0c7527c0e670a82: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260923 22:14:48.441613 15082 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260923 22:14:48.441885 1567 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260923 22:14:48.441787 14997 master_runner.cc:186] Error getting master registration for 127.1.135.254:39543: OK, instance_id { permanent_uuid: "d5923ad1b2bd4b5a997a7936eb1ca7f6" instance_seqno: 1790201688113786 } error { code: CATALOG_MANAGER_NOT_INITIALIZED status { code: SERVICE_UNAVAILABLE message: "Catalog manager is not initialized. State: Starting" } }
I20260923 22:14:48.444844 14997 master_runner.cc:418] Couldn't verify the masters in the cluster. Trying again...
I20260923 22:14:48.451503 14919 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 22:14:48.443393 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40980 (local address 127.1.135.254:39543)
0923 22:14:48.443426 (+ 33us) server_negotiation.cc:206] Beginning negotiation
0923 22:14:48.443428 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0923 22:14:48.443960 (+ 532us) server_negotiation.cc:407] Connection header received
0923 22:14:48.444051 (+ 91us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 22:14:48.444052 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 22:14:48.444115 (+ 63us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 22:14:48.444145 (+ 30us) server_negotiation.cc:226] Negotiated authn=SASL
0923 22:14:48.444968 (+ 823us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 22:14:48.445643 (+ 675us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 22:14:48.446630 (+ 987us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 22:14:48.446911 (+ 281us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 22:14:48.446948 (+ 37us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 22:14:48.448611 (+ 1663us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0923 22:14:48.448612 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0923 22:14:48.448612 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0923 22:14:48.448618 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0923 22:14:48.450228 (+ 1610us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0923 22:14:48.450652 (+ 424us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0923 22:14:48.450653 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0923 22:14:48.450653 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0923 22:14:48.450705 (+ 52us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0923 22:14:48.450912 (+ 207us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0923 22:14:48.450913 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0923 22:14:48.450913 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0923 22:14:48.451035 (+ 122us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0923 22:14:48.451098 (+ 63us) server_negotiation.cc:1035] Waiting for connection context
0923 22:14:48.451336 (+ 238us) server_negotiation.cc:299] Negotiation successful
0923 22:14:48.451419 (+ 83us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":17}
I20260923 22:14:48.451614 15069 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 22:14:48.443587 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56776 (local address 127.1.135.252:36511)
0923 22:14:48.443617 (+ 30us) server_negotiation.cc:206] Beginning negotiation
0923 22:14:48.443619 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0923 22:14:48.443964 (+ 345us) server_negotiation.cc:407] Connection header received
0923 22:14:48.444104 (+ 140us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 22:14:48.444105 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 22:14:48.444114 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 22:14:48.444145 (+ 31us) server_negotiation.cc:226] Negotiated authn=SASL
0923 22:14:48.445037 (+ 892us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 22:14:48.445680 (+ 643us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 22:14:48.446624 (+ 944us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 22:14:48.446910 (+ 286us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 22:14:48.446948 (+ 38us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 22:14:48.448600 (+ 1652us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0923 22:14:48.448601 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0923 22:14:48.448602 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0923 22:14:48.448608 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0923 22:14:48.450402 (+ 1794us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0923 22:14:48.450759 (+ 357us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0923 22:14:48.450759 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0923 22:14:48.450760 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0923 22:14:48.450811 (+ 51us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0923 22:14:48.451003 (+ 192us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0923 22:14:48.451004 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0923 22:14:48.451004 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0923 22:14:48.451172 (+ 168us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0923 22:14:48.451260 (+ 88us) server_negotiation.cc:1035] Waiting for connection context
0923 22:14:48.451458 (+ 198us) server_negotiation.cc:299] Negotiation successful
0923 22:14:48.451525 (+ 67us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":16}
I20260923 22:14:48.453209 14994 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 22:14:48.443769 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55290 (local address 127.1.135.253:37899)
0923 22:14:48.443805 (+ 36us) server_negotiation.cc:206] Beginning negotiation
0923 22:14:48.443808 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0923 22:14:48.444615 (+ 807us) server_negotiation.cc:407] Connection header received
0923 22:14:48.445123 (+ 508us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 22:14:48.445124 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 22:14:48.445133 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 22:14:48.445163 (+ 30us) server_negotiation.cc:226] Negotiated authn=SASL
0923 22:14:48.445817 (+ 654us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 22:14:48.446432 (+ 615us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 22:14:48.447818 (+ 1386us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 22:14:48.448094 (+ 276us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 22:14:48.448131 (+ 37us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 22:14:48.450356 (+ 2225us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0923 22:14:48.450357 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0923 22:14:48.450358 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0923 22:14:48.450364 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0923 22:14:48.451948 (+ 1584us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0923 22:14:48.452493 (+ 545us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0923 22:14:48.452495 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0923 22:14:48.452495 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0923 22:14:48.452546 (+ 51us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0923 22:14:48.452747 (+ 201us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0923 22:14:48.452748 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0923 22:14:48.452748 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0923 22:14:48.452860 (+ 112us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0923 22:14:48.452967 (+ 107us) server_negotiation.cc:1035] Waiting for connection context
0923 22:14:48.453057 (+ 90us) server_negotiation.cc:299] Negotiation successful
0923 22:14:48.453126 (+ 69us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":17}
I20260923 22:14:48.457810 14915 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.1.135.252" port: 36511 } attempt: 2
Sep 23 22:14:48 dist-test-slave-s2fj krb5kdc[14834](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790201688, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.135.254@KRBTEST.COM for kudu/127.1.135.252@KRBTEST.COM
I20260923 22:14:48.463923 15069 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 22:14:48.458132 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56782 (local address 127.1.135.252:36511)
0923 22:14:48.458166 (+ 34us) server_negotiation.cc:206] Beginning negotiation
0923 22:14:48.458167 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0923 22:14:48.458175 (+ 8us) server_negotiation.cc:407] Connection header received
0923 22:14:48.458213 (+ 38us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 22:14:48.458214 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 22:14:48.458222 (+ 8us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 22:14:48.458274 (+ 52us) server_negotiation.cc:226] Negotiated authn=SASL
0923 22:14:48.458505 (+ 231us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 22:14:48.459010 (+ 505us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 22:14:48.459782 (+ 772us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 22:14:48.459979 (+ 197us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 22:14:48.460050 (+ 71us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 22:14:48.461642 (+ 1592us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0923 22:14:48.461643 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0923 22:14:48.461643 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0923 22:14:48.461649 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0923 22:14:48.462825 (+ 1176us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0923 22:14:48.463180 (+ 355us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0923 22:14:48.463181 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0923 22:14:48.463181 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0923 22:14:48.463231 (+ 50us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0923 22:14:48.463430 (+ 199us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0923 22:14:48.463431 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0923 22:14:48.463431 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0923 22:14:48.463548 (+ 117us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0923 22:14:48.463634 (+ 86us) server_negotiation.cc:1035] Waiting for connection context
0923 22:14:48.463779 (+ 145us) server_negotiation.cc:299] Negotiation successful
0923 22:14:48.463842 (+ 63us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":17}
I20260923 22:14:48.463927 14918 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 22:14:48.458052 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.135.252:36511 (local address 127.0.0.1:56782)
0923 22:14:48.458107 (+ 55us) negotiation.cc:107] Waiting for socket to connect
0923 22:14:48.458118 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0923 22:14:48.458161 (+ 43us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0923 22:14:48.458278 (+ 117us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0923 22:14:48.458278 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0923 22:14:48.458408 (+ 130us) client_negotiation.cc:189] Negotiated authn=SASL
0923 22:14:48.458435 (+ 27us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 22:14:48.458436 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 22:14:48.459096 (+ 660us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 22:14:48.459097 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 22:14:48.459689 (+ 592us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 22:14:48.459690 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 22:14:48.460049 (+ 359us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 22:14:48.460050 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 22:14:48.460135 (+ 85us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 22:14:48.460500 (+ 365us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0923 22:14:48.460503 (+ 3us) client_negotiation.cc:656] calling sasl_client_start()
0923 22:14:48.461532 (+ 1029us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0923 22:14:48.462913 (+ 1381us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0923 22:14:48.462914 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0923 22:14:48.462914 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0923 22:14:48.463111 (+ 197us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0923 22:14:48.463306 (+ 195us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0923 22:14:48.463307 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0923 22:14:48.463307 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0923 22:14:48.463362 (+ 55us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0923 22:14:48.463632 (+ 270us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0923 22:14:48.463633 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0923 22:14:48.463673 (+ 40us) client_negotiation.cc:769] Sending connection context
0923 22:14:48.463755 (+ 82us) client_negotiation.cc:240] Negotiation successful
0923 22:14:48.463841 (+ 86us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":36}
I20260923 22:14:48.465404 14915 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P d5923ad1b2bd4b5a997a7936eb1ca7f6: Bootstrap starting.
I20260923 22:14:48.465987 14915 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P d5923ad1b2bd4b5a997a7936eb1ca7f6: Neither blocks nor log segments found. Creating new log.
I20260923 22:14:48.466266 14915 log.cc:824] T 00000000000000000000000000000000 P d5923ad1b2bd4b5a997a7936eb1ca7f6: Log is configured to *not* fsync() on all Append() calls
I20260923 22:14:48.467013 14915 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P d5923ad1b2bd4b5a997a7936eb1ca7f6: No bootstrap required, opened a new log
I20260923 22:14:48.468586 14915 raft_consensus.cc:348] T 00000000000000000000000000000000 P d5923ad1b2bd4b5a997a7936eb1ca7f6 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d5923ad1b2bd4b5a997a7936eb1ca7f6" member_type: VOTER last_known_addr { host: "127.1.135.254" port: 39543 } } peers { permanent_uuid: "5e991661dbe34bc1b470f3eadf12fb36" member_type: VOTER last_known_addr { host: "127.1.135.253" port: 37899 } } peers { permanent_uuid: "972f82227cf14c55b0c7527c0e670a82" member_type: VOTER last_known_addr { host: "127.1.135.252" port: 36511 } }
I20260923 22:14:48.468721 14915 raft_consensus.cc:374] T 00000000000000000000000000000000 P d5923ad1b2bd4b5a997a7936eb1ca7f6 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260923 22:14:48.468797 14915 raft_consensus.cc:729] T 00000000000000000000000000000000 P d5923ad1b2bd4b5a997a7936eb1ca7f6 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d5923ad1b2bd4b5a997a7936eb1ca7f6, State: Initialized, Role: FOLLOWER
I20260923 22:14:48.468935 14915 consensus_queue.cc:260] T 00000000000000000000000000000000 P d5923ad1b2bd4b5a997a7936eb1ca7f6 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d5923ad1b2bd4b5a997a7936eb1ca7f6" member_type: VOTER last_known_addr { host: "127.1.135.254" port: 39543 } } peers { permanent_uuid: "5e991661dbe34bc1b470f3eadf12fb36" member_type: VOTER last_known_addr { host: "127.1.135.253" port: 37899 } } peers { permanent_uuid: "972f82227cf14c55b0c7527c0e670a82" member_type: VOTER last_known_addr { host: "127.1.135.252" port: 36511 } }
I20260923 22:14:48.469240 14915 sys_catalog.cc:564] T 00000000000000000000000000000000 P d5923ad1b2bd4b5a997a7936eb1ca7f6 [sys.catalog]: configured and running, proceeding with master startup.
I20260923 22:14:48.469205 15092 sys_catalog.cc:455] T 00000000000000000000000000000000 P d5923ad1b2bd4b5a997a7936eb1ca7f6 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d5923ad1b2bd4b5a997a7936eb1ca7f6" member_type: VOTER last_known_addr { host: "127.1.135.254" port: 39543 } } peers { permanent_uuid: "5e991661dbe34bc1b470f3eadf12fb36" member_type: VOTER last_known_addr { host: "127.1.135.253" port: 37899 } } peers { permanent_uuid: "972f82227cf14c55b0c7527c0e670a82" member_type: VOTER last_known_addr { host: "127.1.135.252" port: 36511 } } }
I20260923 22:14:48.469332 15092 sys_catalog.cc:458] T 00000000000000000000000000000000 P d5923ad1b2bd4b5a997a7936eb1ca7f6 [sys.catalog]: This master's current role is: FOLLOWER
W20260923 22:14:48.470846 15103 catalog_manager.cc:1558] T 00000000000000000000000000000000 P d5923ad1b2bd4b5a997a7936eb1ca7f6: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260923 22:14:48.470927 15103 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260923 22:14:48.471436 14847 master_runner.cc:186] Error getting master registration for 127.1.135.253:37899: OK, instance_id { permanent_uuid: "5e991661dbe34bc1b470f3eadf12fb36" instance_seqno: 1790201688250210 } error { code: CATALOG_MANAGER_NOT_INITIALIZED status { code: SERVICE_UNAVAILABLE message: "Catalog manager is not initialized. State: Starting" } }
I20260923 22:14:48.474205 14847 master_runner.cc:418] Couldn't verify the masters in the cluster. Trying again...
I20260923 22:14:48.483820 14990 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.1.135.252" port: 36511 } attempt: 3
Sep 23 22:14:48 dist-test-slave-s2fj krb5kdc[14834](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790201688, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.135.253@KRBTEST.COM for kudu/127.1.135.252@KRBTEST.COM
I20260923 22:14:48.491393 14992 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 22:14:48.484051 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.135.252:36511 (local address 127.0.0.1:56796)
0923 22:14:48.484134 (+ 83us) negotiation.cc:107] Waiting for socket to connect
0923 22:14:48.484145 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0923 22:14:48.484193 (+ 48us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0923 22:14:48.484323 (+ 130us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0923 22:14:48.484324 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0923 22:14:48.484490 (+ 166us) client_negotiation.cc:189] Negotiated authn=SASL
0923 22:14:48.484536 (+ 46us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 22:14:48.484537 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 22:14:48.485395 (+ 858us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 22:14:48.485396 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 22:14:48.486174 (+ 778us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 22:14:48.486176 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 22:14:48.486619 (+ 443us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 22:14:48.486620 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 22:14:48.486675 (+ 55us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 22:14:48.487132 (+ 457us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0923 22:14:48.487137 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0923 22:14:48.488566 (+ 1429us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0923 22:14:48.490325 (+ 1759us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0923 22:14:48.490326 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0923 22:14:48.490326 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0923 22:14:48.490538 (+ 212us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0923 22:14:48.490745 (+ 207us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0923 22:14:48.490746 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0923 22:14:48.490746 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0923 22:14:48.490809 (+ 63us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0923 22:14:48.491104 (+ 295us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0923 22:14:48.491105 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0923 22:14:48.491158 (+ 53us) client_negotiation.cc:769] Sending connection context
0923 22:14:48.491223 (+ 65us) client_negotiation.cc:240] Negotiation successful
0923 22:14:48.491302 (+ 79us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":68}
I20260923 22:14:48.491420 15069 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 22:14:48.484120 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56796 (local address 127.1.135.252:36511)
0923 22:14:48.484180 (+ 60us) server_negotiation.cc:206] Beginning negotiation
0923 22:14:48.484182 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0923 22:14:48.484190 (+ 8us) server_negotiation.cc:407] Connection header received
0923 22:14:48.484240 (+ 50us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 22:14:48.484241 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 22:14:48.484250 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 22:14:48.484309 (+ 59us) server_negotiation.cc:226] Negotiated authn=SASL
0923 22:14:48.484610 (+ 301us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 22:14:48.485313 (+ 703us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 22:14:48.486258 (+ 945us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 22:14:48.486544 (+ 286us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 22:14:48.486612 (+ 68us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 22:14:48.488663 (+ 2051us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0923 22:14:48.488664 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0923 22:14:48.488664 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0923 22:14:48.488671 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0923 22:14:48.490243 (+ 1572us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0923 22:14:48.490615 (+ 372us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0923 22:14:48.490616 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0923 22:14:48.490616 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0923 22:14:48.490673 (+ 57us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0923 22:14:48.490875 (+ 202us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0923 22:14:48.490876 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0923 22:14:48.490876 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0923 22:14:48.490996 (+ 120us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0923 22:14:48.491060 (+ 64us) server_negotiation.cc:1035] Waiting for connection context
0923 22:14:48.491257 (+ 197us) server_negotiation.cc:299] Negotiation successful
0923 22:14:48.491340 (+ 83us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":20}
I20260923 22:14:48.493283 14990 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 5e991661dbe34bc1b470f3eadf12fb36: Bootstrap starting.
I20260923 22:14:48.493943 14990 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 5e991661dbe34bc1b470f3eadf12fb36: Neither blocks nor log segments found. Creating new log.
I20260923 22:14:48.494242 14990 log.cc:824] T 00000000000000000000000000000000 P 5e991661dbe34bc1b470f3eadf12fb36: Log is configured to *not* fsync() on all Append() calls
I20260923 22:14:48.495095 14990 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 5e991661dbe34bc1b470f3eadf12fb36: No bootstrap required, opened a new log
I20260923 22:14:48.496882 14990 raft_consensus.cc:348] T 00000000000000000000000000000000 P 5e991661dbe34bc1b470f3eadf12fb36 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d5923ad1b2bd4b5a997a7936eb1ca7f6" member_type: VOTER last_known_addr { host: "127.1.135.254" port: 39543 } } peers { permanent_uuid: "5e991661dbe34bc1b470f3eadf12fb36" member_type: VOTER last_known_addr { host: "127.1.135.253" port: 37899 } } peers { permanent_uuid: "972f82227cf14c55b0c7527c0e670a82" member_type: VOTER last_known_addr { host: "127.1.135.252" port: 36511 } }
I20260923 22:14:48.497008 14990 raft_consensus.cc:374] T 00000000000000000000000000000000 P 5e991661dbe34bc1b470f3eadf12fb36 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260923 22:14:48.497040 14990 raft_consensus.cc:729] T 00000000000000000000000000000000 P 5e991661dbe34bc1b470f3eadf12fb36 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5e991661dbe34bc1b470f3eadf12fb36, State: Initialized, Role: FOLLOWER
I20260923 22:14:48.497210 14990 consensus_queue.cc:260] T 00000000000000000000000000000000 P 5e991661dbe34bc1b470f3eadf12fb36 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d5923ad1b2bd4b5a997a7936eb1ca7f6" member_type: VOTER last_known_addr { host: "127.1.135.254" port: 39543 } } peers { permanent_uuid: "5e991661dbe34bc1b470f3eadf12fb36" member_type: VOTER last_known_addr { host: "127.1.135.253" port: 37899 } } peers { permanent_uuid: "972f82227cf14c55b0c7527c0e670a82" member_type: VOTER last_known_addr { host: "127.1.135.252" port: 36511 } }
I20260923 22:14:48.497511 14990 sys_catalog.cc:564] T 00000000000000000000000000000000 P 5e991661dbe34bc1b470f3eadf12fb36 [sys.catalog]: configured and running, proceeding with master startup.
W20260923 22:14:48.498080 14943 tablet.cc:2369] T 00000000000000000000000000000000 P 5e991661dbe34bc1b470f3eadf12fb36: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260923 22:14:48.497481 15106 sys_catalog.cc:455] T 00000000000000000000000000000000 P 5e991661dbe34bc1b470f3eadf12fb36 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d5923ad1b2bd4b5a997a7936eb1ca7f6" member_type: VOTER last_known_addr { host: "127.1.135.254" port: 39543 } } peers { permanent_uuid: "5e991661dbe34bc1b470f3eadf12fb36" member_type: VOTER last_known_addr { host: "127.1.135.253" port: 37899 } } peers { permanent_uuid: "972f82227cf14c55b0c7527c0e670a82" member_type: VOTER last_known_addr { host: "127.1.135.252" port: 36511 } } }
W20260923 22:14:48.499356 15117 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 5e991661dbe34bc1b470f3eadf12fb36: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260923 22:14:48.499440 15117 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260923 22:14:48.498987 15106 sys_catalog.cc:458] T 00000000000000000000000000000000 P 5e991661dbe34bc1b470f3eadf12fb36 [sys.catalog]: This master's current role is: FOLLOWER
I20260923 22:14:48.501459 14922 master_runner.cc:263] No leader master found from master 5e991661dbe34bc1b470f3eadf12fb36
I20260923 22:14:48.501506 14922 master_runner.cc:418] Couldn't verify the masters in the cluster. Trying again...
I20260923 22:14:48.551995 15106 raft_consensus.cc:482] T 00000000000000000000000000000000 P 5e991661dbe34bc1b470f3eadf12fb36 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260923 22:14:48.552140 15106 raft_consensus.cc:504] T 00000000000000000000000000000000 P 5e991661dbe34bc1b470f3eadf12fb36 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d5923ad1b2bd4b5a997a7936eb1ca7f6" member_type: VOTER last_known_addr { host: "127.1.135.254" port: 39543 } } peers { permanent_uuid: "5e991661dbe34bc1b470f3eadf12fb36" member_type: VOTER last_known_addr { host: "127.1.135.253" port: 37899 } } peers { permanent_uuid: "972f82227cf14c55b0c7527c0e670a82" member_type: VOTER last_known_addr { host: "127.1.135.252" port: 36511 } }
I20260923 22:14:48.552347 15106 leader_election.cc:288] T 00000000000000000000000000000000 P 5e991661dbe34bc1b470f3eadf12fb36 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers d5923ad1b2bd4b5a997a7936eb1ca7f6 (127.1.135.254:39543), 972f82227cf14c55b0c7527c0e670a82 (127.1.135.252:36511)
I20260923 22:14:48.552560 14889 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "5e991661dbe34bc1b470f3eadf12fb36" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d5923ad1b2bd4b5a997a7936eb1ca7f6" is_pre_election: true
I20260923 22:14:48.552695 14889 raft_consensus.cc:2454] T 00000000000000000000000000000000 P d5923ad1b2bd4b5a997a7936eb1ca7f6 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5e991661dbe34bc1b470f3eadf12fb36 in term 0.
I20260923 22:14:48.552685 15039 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "5e991661dbe34bc1b470f3eadf12fb36" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "972f82227cf14c55b0c7527c0e670a82" is_pre_election: true
I20260923 22:14:48.552811 15039 raft_consensus.cc:2454] T 00000000000000000000000000000000 P 972f82227cf14c55b0c7527c0e670a82 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5e991661dbe34bc1b470f3eadf12fb36 in term 0.
I20260923 22:14:48.552870 14939 leader_election.cc:302] T 00000000000000000000000000000000 P 5e991661dbe34bc1b470f3eadf12fb36 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5e991661dbe34bc1b470f3eadf12fb36, d5923ad1b2bd4b5a997a7936eb1ca7f6; no voters:
I20260923 22:14:48.553009 15106 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 5e991661dbe34bc1b470f3eadf12fb36 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260923 22:14:48.553057 15106 raft_consensus.cc:482] T 00000000000000000000000000000000 P 5e991661dbe34bc1b470f3eadf12fb36 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260923 22:14:48.553090 15106 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 5e991661dbe34bc1b470f3eadf12fb36 [term 0 FOLLOWER]: Advancing to term 1
I20260923 22:14:48.554011 15106 raft_consensus.cc:504] T 00000000000000000000000000000000 P 5e991661dbe34bc1b470f3eadf12fb36 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d5923ad1b2bd4b5a997a7936eb1ca7f6" member_type: VOTER last_known_addr { host: "127.1.135.254" port: 39543 } } peers { permanent_uuid: "5e991661dbe34bc1b470f3eadf12fb36" member_type: VOTER last_known_addr { host: "127.1.135.253" port: 37899 } } peers { permanent_uuid: "972f82227cf14c55b0c7527c0e670a82" member_type: VOTER last_known_addr { host: "127.1.135.252" port: 36511 } }
I20260923 22:14:48.554127 15106 leader_election.cc:288] T 00000000000000000000000000000000 P 5e991661dbe34bc1b470f3eadf12fb36 [CANDIDATE]: Term 1 election: Requested vote from peers d5923ad1b2bd4b5a997a7936eb1ca7f6 (127.1.135.254:39543), 972f82227cf14c55b0c7527c0e670a82 (127.1.135.252:36511)
I20260923 22:14:48.554415 14889 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "5e991661dbe34bc1b470f3eadf12fb36" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d5923ad1b2bd4b5a997a7936eb1ca7f6"
I20260923 22:14:48.554486 14889 raft_consensus.cc:3037] T 00000000000000000000000000000000 P d5923ad1b2bd4b5a997a7936eb1ca7f6 [term 0 FOLLOWER]: Advancing to term 1
I20260923 22:14:48.555243 14889 raft_consensus.cc:2454] T 00000000000000000000000000000000 P d5923ad1b2bd4b5a997a7936eb1ca7f6 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 5e991661dbe34bc1b470f3eadf12fb36 in term 1.
I20260923 22:14:48.555442 14939 leader_election.cc:302] T 00000000000000000000000000000000 P 5e991661dbe34bc1b470f3eadf12fb36 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5e991661dbe34bc1b470f3eadf12fb36, d5923ad1b2bd4b5a997a7936eb1ca7f6; no voters:
I20260923 22:14:48.555627 15039 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "5e991661dbe34bc1b470f3eadf12fb36" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "972f82227cf14c55b0c7527c0e670a82"
I20260923 22:14:48.555707 15039 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 972f82227cf14c55b0c7527c0e670a82 [term 0 FOLLOWER]: Advancing to term 1
I20260923 22:14:48.556581 15039 raft_consensus.cc:2454] T 00000000000000000000000000000000 P 972f82227cf14c55b0c7527c0e670a82 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 5e991661dbe34bc1b470f3eadf12fb36 in term 1.
I20260923 22:14:48.556802 15106 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 5e991661dbe34bc1b470f3eadf12fb36 [term 1 FOLLOWER]: Leader election won for term 1
I20260923 22:14:48.556950 15106 raft_consensus.cc:686] T 00000000000000000000000000000000 P 5e991661dbe34bc1b470f3eadf12fb36 [term 1 LEADER]: Becoming Leader. State: Replica: 5e991661dbe34bc1b470f3eadf12fb36, State: Running, Role: LEADER
I20260923 22:14:48.557051 15106 consensus_queue.cc:237] T 00000000000000000000000000000000 P 5e991661dbe34bc1b470f3eadf12fb36 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d5923ad1b2bd4b5a997a7936eb1ca7f6" member_type: VOTER last_known_addr { host: "127.1.135.254" port: 39543 } } peers { permanent_uuid: "5e991661dbe34bc1b470f3eadf12fb36" member_type: VOTER last_known_addr { host: "127.1.135.253" port: 37899 } } peers { permanent_uuid: "972f82227cf14c55b0c7527c0e670a82" member_type: VOTER last_known_addr { host: "127.1.135.252" port: 36511 } }
I20260923 22:14:48.557380 15106 sys_catalog.cc:455] T 00000000000000000000000000000000 P 5e991661dbe34bc1b470f3eadf12fb36 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 5e991661dbe34bc1b470f3eadf12fb36. Latest consensus state: current_term: 1 leader_uuid: "5e991661dbe34bc1b470f3eadf12fb36" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d5923ad1b2bd4b5a997a7936eb1ca7f6" member_type: VOTER last_known_addr { host: "127.1.135.254" port: 39543 } } peers { permanent_uuid: "5e991661dbe34bc1b470f3eadf12fb36" member_type: VOTER last_known_addr { host: "127.1.135.253" port: 37899 } } peers { permanent_uuid: "972f82227cf14c55b0c7527c0e670a82" member_type: VOTER last_known_addr { host: "127.1.135.252" port: 36511 } } }
I20260923 22:14:48.557447 15106 sys_catalog.cc:458] T 00000000000000000000000000000000 P 5e991661dbe34bc1b470f3eadf12fb36 [sys.catalog]: This master's current role is: LEADER
I20260923 22:14:48.557737 15121 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260923 22:14:48.557940 15121 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260923 22:14:48.559860 15039 raft_consensus.cc:1261] T 00000000000000000000000000000000 P 972f82227cf14c55b0c7527c0e670a82 [term 1 FOLLOWER]: Refusing update from remote peer 5e991661dbe34bc1b470f3eadf12fb36: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260923 22:14:48.560098 14889 raft_consensus.cc:1261] T 00000000000000000000000000000000 P d5923ad1b2bd4b5a997a7936eb1ca7f6 [term 1 FOLLOWER]: Refusing update from remote peer 5e991661dbe34bc1b470f3eadf12fb36: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260923 22:14:48.560513 15106 consensus_queue.cc:1035] T 00000000000000000000000000000000 P 5e991661dbe34bc1b470f3eadf12fb36 [LEADER]: Connected to new peer: Peer: permanent_uuid: "972f82227cf14c55b0c7527c0e670a82" member_type: VOTER last_known_addr { host: "127.1.135.252" port: 36511 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260923 22:14:48.560643 15106 consensus_queue.cc:1035] T 00000000000000000000000000000000 P 5e991661dbe34bc1b470f3eadf12fb36 [LEADER]: Connected to new peer: Peer: permanent_uuid: "d5923ad1b2bd4b5a997a7936eb1ca7f6" member_type: VOTER last_known_addr { host: "127.1.135.254" port: 39543 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260923 22:14:48.561815 15092 sys_catalog.cc:455] T 00000000000000000000000000000000 P d5923ad1b2bd4b5a997a7936eb1ca7f6 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 5e991661dbe34bc1b470f3eadf12fb36. Latest consensus state: current_term: 1 leader_uuid: "5e991661dbe34bc1b470f3eadf12fb36" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d5923ad1b2bd4b5a997a7936eb1ca7f6" member_type: VOTER last_known_addr { host: "127.1.135.254" port: 39543 } } peers { permanent_uuid: "5e991661dbe34bc1b470f3eadf12fb36" member_type: VOTER last_known_addr { host: "127.1.135.253" port: 37899 } } peers { permanent_uuid: "972f82227cf14c55b0c7527c0e670a82" member_type: VOTER last_known_addr { host: "127.1.135.252" port: 36511 } } }
I20260923 22:14:48.561890 15092 sys_catalog.cc:458] T 00000000000000000000000000000000 P d5923ad1b2bd4b5a997a7936eb1ca7f6 [sys.catalog]: This master's current role is: FOLLOWER
I20260923 22:14:48.562829 15071 sys_catalog.cc:455] T 00000000000000000000000000000000 P 972f82227cf14c55b0c7527c0e670a82 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 5e991661dbe34bc1b470f3eadf12fb36. Latest consensus state: current_term: 1 leader_uuid: "5e991661dbe34bc1b470f3eadf12fb36" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d5923ad1b2bd4b5a997a7936eb1ca7f6" member_type: VOTER last_known_addr { host: "127.1.135.254" port: 39543 } } peers { permanent_uuid: "5e991661dbe34bc1b470f3eadf12fb36" member_type: VOTER last_known_addr { host: "127.1.135.253" port: 37899 } } peers { permanent_uuid: "972f82227cf14c55b0c7527c0e670a82" member_type: VOTER last_known_addr { host: "127.1.135.252" port: 36511 } } }
I20260923 22:14:48.562907 15071 sys_catalog.cc:458] T 00000000000000000000000000000000 P 972f82227cf14c55b0c7527c0e670a82 [sys.catalog]: This master's current role is: FOLLOWER
I20260923 22:14:48.564733 15106 sys_catalog.cc:455] T 00000000000000000000000000000000 P 5e991661dbe34bc1b470f3eadf12fb36 [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "5e991661dbe34bc1b470f3eadf12fb36" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d5923ad1b2bd4b5a997a7936eb1ca7f6" member_type: VOTER last_known_addr { host: "127.1.135.254" port: 39543 } } peers { permanent_uuid: "5e991661dbe34bc1b470f3eadf12fb36" member_type: VOTER last_known_addr { host: "127.1.135.253" port: 37899 } } peers { permanent_uuid: "972f82227cf14c55b0c7527c0e670a82" member_type: VOTER last_known_addr { host: "127.1.135.252" port: 36511 } } }
I20260923 22:14:48.564805 15106 sys_catalog.cc:458] T 00000000000000000000000000000000 P 5e991661dbe34bc1b470f3eadf12fb36 [sys.catalog]: This master's current role is: LEADER
I20260923 22:14:48.565161 15071 sys_catalog.cc:455] T 00000000000000000000000000000000 P 972f82227cf14c55b0c7527c0e670a82 [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "5e991661dbe34bc1b470f3eadf12fb36" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d5923ad1b2bd4b5a997a7936eb1ca7f6" member_type: VOTER last_known_addr { host: "127.1.135.254" port: 39543 } } peers { permanent_uuid: "5e991661dbe34bc1b470f3eadf12fb36" member_type: VOTER last_known_addr { host: "127.1.135.253" port: 37899 } } peers { permanent_uuid: "972f82227cf14c55b0c7527c0e670a82" member_type: VOTER last_known_addr { host: "127.1.135.252" port: 36511 } } }
I20260923 22:14:48.565232 15071 sys_catalog.cc:458] T 00000000000000000000000000000000 P 972f82227cf14c55b0c7527c0e670a82 [sys.catalog]: This master's current role is: FOLLOWER
I20260923 22:14:48.565454 15122 mvcc.cc:205] Tried to move back new op lower bound from 7332666116338585600 to 7332666116330045440. Current Snapshot: MvccSnapshot[applied={T|T < 7332666116338585600}]
I20260923 22:14:48.565824 15125 mvcc.cc:205] Tried to move back new op lower bound from 7332666116338585600 to 7332666116330045440. Current Snapshot: MvccSnapshot[applied={T|T < 7332666116338585600}]
I20260923 22:14:48.566268 15106 sys_catalog.cc:455] T 00000000000000000000000000000000 P 5e991661dbe34bc1b470f3eadf12fb36 [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "5e991661dbe34bc1b470f3eadf12fb36" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d5923ad1b2bd4b5a997a7936eb1ca7f6" member_type: VOTER last_known_addr { host: "127.1.135.254" port: 39543 } } peers { permanent_uuid: "5e991661dbe34bc1b470f3eadf12fb36" member_type: VOTER last_known_addr { host: "127.1.135.253" port: 37899 } } peers { permanent_uuid: "972f82227cf14c55b0c7527c0e670a82" member_type: VOTER last_known_addr { host: "127.1.135.252" port: 36511 } } }
I20260923 22:14:48.566351 15106 sys_catalog.cc:458] T 00000000000000000000000000000000 P 5e991661dbe34bc1b470f3eadf12fb36 [sys.catalog]: This master's current role is: LEADER
I20260923 22:14:48.566725 15092 sys_catalog.cc:455] T 00000000000000000000000000000000 P d5923ad1b2bd4b5a997a7936eb1ca7f6 [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "5e991661dbe34bc1b470f3eadf12fb36" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d5923ad1b2bd4b5a997a7936eb1ca7f6" member_type: VOTER last_known_addr { host: "127.1.135.254" port: 39543 } } peers { permanent_uuid: "5e991661dbe34bc1b470f3eadf12fb36" member_type: VOTER last_known_addr { host: "127.1.135.253" port: 37899 } } peers { permanent_uuid: "972f82227cf14c55b0c7527c0e670a82" member_type: VOTER last_known_addr { host: "127.1.135.252" port: 36511 } } }
I20260923 22:14:48.566797 15092 sys_catalog.cc:458] T 00000000000000000000000000000000 P d5923ad1b2bd4b5a997a7936eb1ca7f6 [sys.catalog]: This master's current role is: FOLLOWER
I20260923 22:14:48.567461 15123 mvcc.cc:205] Tried to move back new op lower bound from 7332666116338585600 to 7332666116330045440. Current Snapshot: MvccSnapshot[applied={T|T < 7332666116338585600}]
I20260923 22:14:48.567941 15121 catalog_manager.cc:1347] Generated new cluster ID: ecca6e4395734848860e87c6290d85d6
I20260923 22:14:48.567983 15121 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260923 22:14:48.577842 15121 catalog_manager.cc:1370] Generated new certificate authority record
I20260923 22:14:48.578684 15121 catalog_manager.cc:1504] Loading token signing keys...
I20260923 22:14:48.586400 15121 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 5e991661dbe34bc1b470f3eadf12fb36: Generated new TSK 0
I20260923 22:14:48.586604 15121 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260923 22:14:49.442065 15082 catalog_manager.cc:1259] Loaded cluster ID: ecca6e4395734848860e87c6290d85d6
I20260923 22:14:49.442131 15082 catalog_manager.cc:1552] T 00000000000000000000000000000000 P 972f82227cf14c55b0c7527c0e670a82: loading cluster ID for follower catalog manager: success
I20260923 22:14:49.442904 15082 catalog_manager.cc:1574] T 00000000000000000000000000000000 P 972f82227cf14c55b0c7527c0e670a82: acquiring CA information for follower catalog manager: success
I20260923 22:14:49.443240 15082 catalog_manager.cc:1602] T 00000000000000000000000000000000 P 972f82227cf14c55b0c7527c0e670a82: importing token verification keys for follower catalog manager: success; most recent TSK sequence number 0
I20260923 22:14:49.471343 15103 catalog_manager.cc:1259] Loaded cluster ID: ecca6e4395734848860e87c6290d85d6
I20260923 22:14:49.471441 15103 catalog_manager.cc:1552] T 00000000000000000000000000000000 P d5923ad1b2bd4b5a997a7936eb1ca7f6: loading cluster ID for follower catalog manager: success
I20260923 22:14:49.472232 15103 catalog_manager.cc:1574] T 00000000000000000000000000000000 P d5923ad1b2bd4b5a997a7936eb1ca7f6: acquiring CA information for follower catalog manager: success
I20260923 22:14:49.472527 15103 catalog_manager.cc:1602] T 00000000000000000000000000000000 P d5923ad1b2bd4b5a997a7936eb1ca7f6: importing token verification keys for follower catalog manager: success; most recent TSK sequence number 0
I20260923 22:14:49.647199 1567 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskRiQe_a/build/release/bin/kudu with pid 14847
I20260923 22:14:49.652340 1567 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskRiQe_a/build/release/bin/kudu with pid 14922
I20260923 22:14:49.657048 1567 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskRiQe_a/build/release/bin/kudu with pid 14997
2026-09-23T22:14:49Z chronyd exiting
[ OK ] SecurityITest.IPKICACert (2785 ms)
[ RUN ] SecurityITest.IPKICACertWebServerEndPoints
Loading random data
Initializing database '/tmp/dist-test-taskRiQe_a/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1790201539513534-1567-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Sep 23 22:14:49 dist-test-slave-s2fj krb5kdc[15140](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Sep 23 22:14:49 dist-test-slave-s2fj krb5kdc[15140](info): set up 2 sockets
Sep 23 22:14:49 dist-test-slave-s2fj krb5kdc[15140](info): commencing operation
krb5kdc: starting...
W20260923 22:14:50.648226 1567 mini_kdc.cc:121] Time spent starting KDC: real 0.972s user 0.000s sys 0.005s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskRiQe_a/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1790201539513534-1567-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskRiQe_a/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1790201539513534-1567-0/krb5kdc/test-user.keytab.
Sep 23 22:14:50 dist-test-slave-s2fj krb5kdc[15140](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790201690, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-09-23T22:14:50Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-09-23T22:14:50Z Disabled control of system clock
WARNING: no policy specified for kudu/127.1.135.254@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.1.135.254@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.1.135.254 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskRiQe_a/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1790201539513534-1567-0/krb5kdc/kudu_127.1.135.254.keytab.
Entry for principal kudu/127.1.135.254 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskRiQe_a/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1790201539513534-1567-0/krb5kdc/kudu_127.1.135.254.keytab.
I20260923 22:14:50.756224 1567 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskRiQe_a/build/release/bin/kudu
/tmp/dist-test-taskRiQe_a/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskRiQe_a/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1790201539513534-1567-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskRiQe_a/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1790201539513534-1567-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskRiQe_a/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1790201539513534-1567-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskRiQe_a/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1790201539513534-1567-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.1.135.254:33955
--webserver_interface=127.1.135.254
--webserver_port=0
--keytab_file=/tmp/dist-test-taskRiQe_a/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1790201539513534-1567-0/krb5kdc/kudu_127.1.135.254.keytab
--principal=kudu/127.1.135.254
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.1.135.212:36253
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.1.135.254:33955
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskRiQe_a/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1790201539513534-1567-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskRiQe_a/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1790201539513534-1567-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskRiQe_a/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1790201539513534-1567-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260923 22:14:50.831701 15153 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260923 22:14:50.831879 15153 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260923 22:14:50.831900 15153 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260923 22:14:50.833025 15153 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260923 22:14:50.833072 15153 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260923 22:14:50.833087 15153 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260923 22:14:50.833099 15153 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260923 22:14:50.833112 15153 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260923 22:14:50.834451 15153 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.1.135.212:36253
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskRiQe_a/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1790201539513534-1567-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskRiQe_a/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1790201539513534-1567-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.1.135.254:33955
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskRiQe_a/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1790201539513534-1567-0/krb5kdc/kudu_127.1.135.254.keytab
--principal=kudu/127.1.135.254
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.1.135.254:33955
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskRiQe_a/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1790201539513534-1567-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.1.135.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.15153
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskRiQe_a/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1790201539513534-1567-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 23 Sep 2026 20:53:37 UTC on ccycloud.quasar-rkviyx.root.comops.site
build id 4668083
I20260923 22:14:50.834650 15153 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260923 22:14:50.834810 15153 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260923 22:14:50.839147 15161 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260923 22:14:50.839310 15153 server_base.cc:1034] running on GCE node
W20260923 22:14:50.839176 15159 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260923 22:14:50.839159 15158 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260923 22:14:50.839658 15153 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260923 22:14:50.839880 15153 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260923 22:14:50.841012 15153 hybrid_clock.cc:648] HybridClock initialized: now 1790201690840994 us; error 33 us; skew 500 ppm
Sep 23 22:14:50 dist-test-slave-s2fj krb5kdc[15140](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790201690, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.135.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260923 22:14:50.843294 15153 init.cc:377] Logged in from keytab as kudu/127.1.135.254@KRBTEST.COM (short username kudu)
I20260923 22:14:50.843734 15153 webserver.cc:460] Webserver started at http://127.1.135.254:43967/ using document root <none> and password file <none>
I20260923 22:14:50.843946 15153 fs_manager.cc:359] Metadata directory not provided
I20260923 22:14:50.843989 15153 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260923 22:14:50.844182 15153 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260923 22:14:50.845175 15153 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskRiQe_a/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1790201539513534-1567-0/minicluster-data/master-0/data/instance:
uuid: "dd6ef309525f4762ac20897dca48ec54"
format_stamp: "Formatted at 2026-09-23 22:14:50 on dist-test-slave-s2fj"
server_key: "79565256f69568d82dc1adbf76562698"
server_key_iv: "b97eef01170f5d38e577db4d70a2db90"
server_key_version: "encryptionkey@0"
I20260923 22:14:50.845508 15153 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskRiQe_a/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1790201539513534-1567-0/minicluster-data/master-0/wal/instance:
uuid: "dd6ef309525f4762ac20897dca48ec54"
format_stamp: "Formatted at 2026-09-23 22:14:50 on dist-test-slave-s2fj"
server_key: "79565256f69568d82dc1adbf76562698"
server_key_iv: "b97eef01170f5d38e577db4d70a2db90"
server_key_version: "encryptionkey@0"
I20260923 22:14:50.846709 15153 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.000s
I20260923 22:14:50.847373 15168 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260923 22:14:50.847530 15153 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260923 22:14:50.847620 15153 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskRiQe_a/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1790201539513534-1567-0/minicluster-data/master-0/data,/tmp/dist-test-taskRiQe_a/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1790201539513534-1567-0/minicluster-data/master-0/wal
uuid: "dd6ef309525f4762ac20897dca48ec54"
format_stamp: "Formatted at 2026-09-23 22:14:50 on dist-test-slave-s2fj"
server_key: "79565256f69568d82dc1adbf76562698"
server_key_iv: "b97eef01170f5d38e577db4d70a2db90"
server_key_version: "encryptionkey@0"
I20260923 22:14:50.847697 15153 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskRiQe_a/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1790201539513534-1567-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskRiQe_a/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1790201539513534-1567-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskRiQe_a/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1790201539513534-1567-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260923 22:14:50.853840 15153 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260923 22:14:50.855279 15153 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260923 22:14:50.855408 15153 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260923 22:14:50.858857 15153 rpc_server.cc:307] RPC server started. Bound to: 127.1.135.254:33955
I20260923 22:14:50.858883 15220 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.1.135.254:33955 every 8 connection(s)
I20260923 22:14:50.859155 15153 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskRiQe_a/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1790201539513534-1567-0/minicluster-data/master-0/data/info.pb
I20260923 22:14:50.859887 15221 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260923 22:14:50.859977 1567 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskRiQe_a/build/release/bin/kudu as pid 15153
I20260923 22:14:50.860085 1567 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskRiQe_a/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1790201539513534-1567-0/minicluster-data/master-0/wal/instance
I20260923 22:14:50.860332 1567 external_mini_cluster.cc:1428] Setting key 537c787cdcbf42f207eb87955c7c0cb2
I20260923 22:14:50.862864 15221 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P dd6ef309525f4762ac20897dca48ec54: Bootstrap starting.
I20260923 22:14:50.863504 15221 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P dd6ef309525f4762ac20897dca48ec54: Neither blocks nor log segments found. Creating new log.
I20260923 22:14:50.863801 15221 log.cc:824] T 00000000000000000000000000000000 P dd6ef309525f4762ac20897dca48ec54: Log is configured to *not* fsync() on all Append() calls
I20260923 22:14:50.864526 15221 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P dd6ef309525f4762ac20897dca48ec54: No bootstrap required, opened a new log
Sep 23 22:14:50 dist-test-slave-s2fj krb5kdc[15140](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790201690, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.1.135.254@KRBTEST.COM
I20260923 22:14:50.866870 15221 raft_consensus.cc:348] T 00000000000000000000000000000000 P dd6ef309525f4762ac20897dca48ec54 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "dd6ef309525f4762ac20897dca48ec54" member_type: VOTER last_known_addr { host: "127.1.135.254" port: 33955 } }
I20260923 22:14:50.867008 15221 raft_consensus.cc:374] T 00000000000000000000000000000000 P dd6ef309525f4762ac20897dca48ec54 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260923 22:14:50.867053 15221 raft_consensus.cc:729] T 00000000000000000000000000000000 P dd6ef309525f4762ac20897dca48ec54 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: dd6ef309525f4762ac20897dca48ec54, State: Initialized, Role: FOLLOWER
I20260923 22:14:50.867201 15221 consensus_queue.cc:260] T 00000000000000000000000000000000 P dd6ef309525f4762ac20897dca48ec54 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "dd6ef309525f4762ac20897dca48ec54" member_type: VOTER last_known_addr { host: "127.1.135.254" port: 33955 } }
I20260923 22:14:50.867287 15221 raft_consensus.cc:388] T 00000000000000000000000000000000 P dd6ef309525f4762ac20897dca48ec54 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260923 22:14:50.867329 15221 raft_consensus.cc:482] T 00000000000000000000000000000000 P dd6ef309525f4762ac20897dca48ec54 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260923 22:14:50.867370 15221 raft_consensus.cc:3037] T 00000000000000000000000000000000 P dd6ef309525f4762ac20897dca48ec54 [term 0 FOLLOWER]: Advancing to term 1
I20260923 22:14:50.867846 15221 raft_consensus.cc:504] T 00000000000000000000000000000000 P dd6ef309525f4762ac20897dca48ec54 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "dd6ef309525f4762ac20897dca48ec54" member_type: VOTER last_known_addr { host: "127.1.135.254" port: 33955 } }
I20260923 22:14:50.867923 15221 leader_election.cc:302] T 00000000000000000000000000000000 P dd6ef309525f4762ac20897dca48ec54 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: dd6ef309525f4762ac20897dca48ec54; no voters:
I20260923 22:14:50.868104 15221 leader_election.cc:288] T 00000000000000000000000000000000 P dd6ef309525f4762ac20897dca48ec54 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260923 22:14:50.868140 15226 raft_consensus.cc:2781] T 00000000000000000000000000000000 P dd6ef309525f4762ac20897dca48ec54 [term 1 FOLLOWER]: Leader election won for term 1
I20260923 22:14:50.868243 15226 raft_consensus.cc:686] T 00000000000000000000000000000000 P dd6ef309525f4762ac20897dca48ec54 [term 1 LEADER]: Becoming Leader. State: Replica: dd6ef309525f4762ac20897dca48ec54, State: Running, Role: LEADER
I20260923 22:14:50.868345 15221 sys_catalog.cc:564] T 00000000000000000000000000000000 P dd6ef309525f4762ac20897dca48ec54 [sys.catalog]: configured and running, proceeding with master startup.
I20260923 22:14:50.868373 15226 consensus_queue.cc:237] T 00000000000000000000000000000000 P dd6ef309525f4762ac20897dca48ec54 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "dd6ef309525f4762ac20897dca48ec54" member_type: VOTER last_known_addr { host: "127.1.135.254" port: 33955 } }
I20260923 22:14:50.868697 15226 sys_catalog.cc:455] T 00000000000000000000000000000000 P dd6ef309525f4762ac20897dca48ec54 [sys.catalog]: SysCatalogTable state changed. Reason: New leader dd6ef309525f4762ac20897dca48ec54. Latest consensus state: current_term: 1 leader_uuid: "dd6ef309525f4762ac20897dca48ec54" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "dd6ef309525f4762ac20897dca48ec54" member_type: VOTER last_known_addr { host: "127.1.135.254" port: 33955 } } }
I20260923 22:14:50.868710 15227 sys_catalog.cc:455] T 00000000000000000000000000000000 P dd6ef309525f4762ac20897dca48ec54 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "dd6ef309525f4762ac20897dca48ec54" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "dd6ef309525f4762ac20897dca48ec54" member_type: VOTER last_known_addr { host: "127.1.135.254" port: 33955 } } }
I20260923 22:14:50.868909 15227 sys_catalog.cc:458] T 00000000000000000000000000000000 P dd6ef309525f4762ac20897dca48ec54 [sys.catalog]: This master's current role is: LEADER
I20260923 22:14:50.868842 15226 sys_catalog.cc:458] T 00000000000000000000000000000000 P dd6ef309525f4762ac20897dca48ec54 [sys.catalog]: This master's current role is: LEADER
I20260923 22:14:50.869182 15235 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260923 22:14:50.869683 15235 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260923 22:14:50.871003 15224 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 22:14:50.860802 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58400 (local address 127.1.135.254:33955)
0923 22:14:50.860982 (+ 180us) server_negotiation.cc:206] Beginning negotiation
0923 22:14:50.860989 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0923 22:14:50.861013 (+ 24us) server_negotiation.cc:407] Connection header received
0923 22:14:50.861380 (+ 367us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 22:14:50.861386 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 22:14:50.861426 (+ 40us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 22:14:50.861528 (+ 102us) server_negotiation.cc:226] Negotiated authn=SASL
0923 22:14:50.862292 (+ 764us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 22:14:50.863182 (+ 890us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 22:14:50.864269 (+ 1087us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 22:14:50.864675 (+ 406us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 22:14:50.864751 (+ 76us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 22:14:50.867045 (+ 2294us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0923 22:14:50.867049 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0923 22:14:50.867049 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0923 22:14:50.867068 (+ 19us) server_negotiation.cc:922] calling sasl_server_start()
0923 22:14:50.869033 (+ 1965us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0923 22:14:50.869458 (+ 425us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0923 22:14:50.869461 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0923 22:14:50.869463 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0923 22:14:50.869563 (+ 100us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0923 22:14:50.869769 (+ 206us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0923 22:14:50.869769 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0923 22:14:50.869770 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0923 22:14:50.870139 (+ 369us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0923 22:14:50.870636 (+ 497us) server_negotiation.cc:1035] Waiting for connection context
0923 22:14:50.870704 (+ 68us) server_negotiation.cc:299] Negotiation successful
0923 22:14:50.870827 (+ 123us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":134,"thread_start_us":87,"threads_started":1}
I20260923 22:14:50.871546 15235 catalog_manager.cc:1347] Generated new cluster ID: 5a10d5bf9e6c4e039578297b20e10b1a
I20260923 22:14:50.871632 15235 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260923 22:14:50.871903 1567 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260923 22:14:50.876674 15235 catalog_manager.cc:1370] Generated new certificate authority record
I20260923 22:14:50.877151 15235 catalog_manager.cc:1504] Loading token signing keys...
I20260923 22:14:50.880729 15235 catalog_manager.cc:5868] T 00000000000000000000000000000000 P dd6ef309525f4762ac20897dca48ec54: Generated new TSK 0
I20260923 22:14:50.880883 15235 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260923 22:14:50.925457 1567 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskRiQe_a/build/release/bin/kudu with pid 15153
2026-09-23T22:14:50Z chronyd exiting
[ OK ] SecurityITest.IPKICACertWebServerEndPoints (1267 ms)
[ RUN ] SecurityITest.TestEnableAndDisableEncryptedIPKICert
Loading random data
Initializing database '/tmp/dist-test-taskRiQe_a/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790201539513534-1567-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Sep 23 22:14:50 dist-test-slave-s2fj krb5kdc[15248](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Sep 23 22:14:50 dist-test-slave-s2fj krb5kdc[15248](info): set up 2 sockets
Sep 23 22:14:50 dist-test-slave-s2fj krb5kdc[15248](info): commencing operation
krb5kdc: starting...
W20260923 22:14:51.912294 1567 mini_kdc.cc:121] Time spent starting KDC: real 0.969s user 0.000s sys 0.005s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskRiQe_a/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790201539513534-1567-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskRiQe_a/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790201539513534-1567-0/krb5kdc/test-user.keytab.
Sep 23 22:14:51 dist-test-slave-s2fj krb5kdc[15248](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790201691, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-09-23T22:14:51Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-09-23T22:14:51Z Disabled control of system clock
WARNING: no policy specified for kudu/127.1.135.254@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.1.135.254@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.1.135.254 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskRiQe_a/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790201539513534-1567-0/krb5kdc/kudu_127.1.135.254.keytab.
Entry for principal kudu/127.1.135.254 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskRiQe_a/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790201539513534-1567-0/krb5kdc/kudu_127.1.135.254.keytab.
I20260923 22:14:52.020781 1567 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskRiQe_a/build/release/bin/kudu
/tmp/dist-test-taskRiQe_a/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskRiQe_a/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790201539513534-1567-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskRiQe_a/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790201539513534-1567-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskRiQe_a/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790201539513534-1567-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskRiQe_a/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790201539513534-1567-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.1.135.254:37515
--webserver_interface=127.1.135.254
--webserver_port=0
--keytab_file=/tmp/dist-test-taskRiQe_a/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790201539513534-1567-0/krb5kdc/kudu_127.1.135.254.keytab
--principal=kudu/127.1.135.254
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.1.135.212:38425
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.1.135.254:37515
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskRiQe_a/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790201539513534-1567-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskRiQe_a/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790201539513534-1567-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskRiQe_a/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790201539513534-1567-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260923 22:14:52.095938 15261 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260923 22:14:52.096192 15261 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260923 22:14:52.096227 15261 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260923 22:14:52.097280 15261 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260923 22:14:52.097348 15261 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260923 22:14:52.097373 15261 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260923 22:14:52.097394 15261 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260923 22:14:52.097421 15261 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260923 22:14:52.098810 15261 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.1.135.212:38425
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskRiQe_a/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790201539513534-1567-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskRiQe_a/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790201539513534-1567-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.1.135.254:37515
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskRiQe_a/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790201539513534-1567-0/krb5kdc/kudu_127.1.135.254.keytab
--principal=kudu/127.1.135.254
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.1.135.254:37515
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskRiQe_a/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790201539513534-1567-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.1.135.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.15261
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskRiQe_a/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790201539513534-1567-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 23 Sep 2026 20:53:37 UTC on ccycloud.quasar-rkviyx.root.comops.site
build id 4668083
I20260923 22:14:52.099056 15261 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260923 22:14:52.099267 15261 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260923 22:14:52.103746 15266 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260923 22:14:52.103773 15267 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260923 22:14:52.103936 15269 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260923 22:14:52.103968 15261 server_base.cc:1034] running on GCE node
I20260923 22:14:52.104223 15261 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260923 22:14:52.104502 15261 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260923 22:14:52.105696 15261 hybrid_clock.cc:648] HybridClock initialized: now 1790201692105683 us; error 25 us; skew 500 ppm
Sep 23 22:14:52 dist-test-slave-s2fj krb5kdc[15248](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790201692, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.135.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260923 22:14:52.107867 15261 init.cc:377] Logged in from keytab as kudu/127.1.135.254@KRBTEST.COM (short username kudu)
I20260923 22:14:52.108297 15261 webserver.cc:460] Webserver started at http://127.1.135.254:45941/ using document root <none> and password file <none>
I20260923 22:14:52.108469 15261 fs_manager.cc:359] Metadata directory not provided
I20260923 22:14:52.108506 15261 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260923 22:14:52.108673 15261 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260923 22:14:52.109683 15261 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskRiQe_a/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790201539513534-1567-0/minicluster-data/master-0/data/instance:
uuid: "d32493cb30cb4eda8ee9e9f9a9c35ce2"
format_stamp: "Formatted at 2026-09-23 22:14:52 on dist-test-slave-s2fj"
server_key: "8358f9839aaaf6002d5bb63dbf35b23e"
server_key_iv: "887f4f7c2f04783e830210e387213916"
server_key_version: "encryptionkey@0"
I20260923 22:14:52.109962 15261 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskRiQe_a/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790201539513534-1567-0/minicluster-data/master-0/wal/instance:
uuid: "d32493cb30cb4eda8ee9e9f9a9c35ce2"
format_stamp: "Formatted at 2026-09-23 22:14:52 on dist-test-slave-s2fj"
server_key: "8358f9839aaaf6002d5bb63dbf35b23e"
server_key_iv: "887f4f7c2f04783e830210e387213916"
server_key_version: "encryptionkey@0"
I20260923 22:14:52.111114 15261 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.001s
I20260923 22:14:52.111756 15276 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260923 22:14:52.111907 15261 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260923 22:14:52.111986 15261 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskRiQe_a/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790201539513534-1567-0/minicluster-data/master-0/data,/tmp/dist-test-taskRiQe_a/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790201539513534-1567-0/minicluster-data/master-0/wal
uuid: "d32493cb30cb4eda8ee9e9f9a9c35ce2"
format_stamp: "Formatted at 2026-09-23 22:14:52 on dist-test-slave-s2fj"
server_key: "8358f9839aaaf6002d5bb63dbf35b23e"
server_key_iv: "887f4f7c2f04783e830210e387213916"
server_key_version: "encryptionkey@0"
I20260923 22:14:52.112038 15261 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskRiQe_a/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790201539513534-1567-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskRiQe_a/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790201539513534-1567-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskRiQe_a/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790201539513534-1567-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260923 22:14:52.128332 15261 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260923 22:14:52.130431 15261 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260923 22:14:52.130532 15261 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260923 22:14:52.134160 15261 rpc_server.cc:307] RPC server started. Bound to: 127.1.135.254:37515
I20260923 22:14:52.134229 15328 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.1.135.254:37515 every 8 connection(s)
I20260923 22:14:52.134533 15261 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskRiQe_a/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790201539513534-1567-0/minicluster-data/master-0/data/info.pb
I20260923 22:14:52.135561 15329 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260923 22:14:52.137707 15329 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P d32493cb30cb4eda8ee9e9f9a9c35ce2: Bootstrap starting.
I20260923 22:14:52.138258 15329 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P d32493cb30cb4eda8ee9e9f9a9c35ce2: Neither blocks nor log segments found. Creating new log.
I20260923 22:14:52.138548 15329 log.cc:824] T 00000000000000000000000000000000 P d32493cb30cb4eda8ee9e9f9a9c35ce2: Log is configured to *not* fsync() on all Append() calls
I20260923 22:14:52.139205 15329 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P d32493cb30cb4eda8ee9e9f9a9c35ce2: No bootstrap required, opened a new log
I20260923 22:14:52.140761 15329 raft_consensus.cc:348] T 00000000000000000000000000000000 P d32493cb30cb4eda8ee9e9f9a9c35ce2 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d32493cb30cb4eda8ee9e9f9a9c35ce2" member_type: VOTER last_known_addr { host: "127.1.135.254" port: 37515 } }
I20260923 22:14:52.140913 15329 raft_consensus.cc:374] T 00000000000000000000000000000000 P d32493cb30cb4eda8ee9e9f9a9c35ce2 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260923 22:14:52.140955 15329 raft_consensus.cc:729] T 00000000000000000000000000000000 P d32493cb30cb4eda8ee9e9f9a9c35ce2 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d32493cb30cb4eda8ee9e9f9a9c35ce2, State: Initialized, Role: FOLLOWER
I20260923 22:14:52.141141 15329 consensus_queue.cc:260] T 00000000000000000000000000000000 P d32493cb30cb4eda8ee9e9f9a9c35ce2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d32493cb30cb4eda8ee9e9f9a9c35ce2" member_type: VOTER last_known_addr { host: "127.1.135.254" port: 37515 } }
I20260923 22:14:52.141239 15329 raft_consensus.cc:388] T 00000000000000000000000000000000 P d32493cb30cb4eda8ee9e9f9a9c35ce2 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260923 22:14:52.141274 15329 raft_consensus.cc:482] T 00000000000000000000000000000000 P d32493cb30cb4eda8ee9e9f9a9c35ce2 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260923 22:14:52.141324 15329 raft_consensus.cc:3037] T 00000000000000000000000000000000 P d32493cb30cb4eda8ee9e9f9a9c35ce2 [term 0 FOLLOWER]: Advancing to term 1
I20260923 22:14:52.142073 15329 raft_consensus.cc:504] T 00000000000000000000000000000000 P d32493cb30cb4eda8ee9e9f9a9c35ce2 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d32493cb30cb4eda8ee9e9f9a9c35ce2" member_type: VOTER last_known_addr { host: "127.1.135.254" port: 37515 } }
I20260923 22:14:52.142190 15329 leader_election.cc:302] T 00000000000000000000000000000000 P d32493cb30cb4eda8ee9e9f9a9c35ce2 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: d32493cb30cb4eda8ee9e9f9a9c35ce2; no voters:
I20260923 22:14:52.142367 15329 leader_election.cc:288] T 00000000000000000000000000000000 P d32493cb30cb4eda8ee9e9f9a9c35ce2 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260923 22:14:52.142387 15332 raft_consensus.cc:2781] T 00000000000000000000000000000000 P d32493cb30cb4eda8ee9e9f9a9c35ce2 [term 1 FOLLOWER]: Leader election won for term 1
I20260923 22:14:52.142522 15332 raft_consensus.cc:686] T 00000000000000000000000000000000 P d32493cb30cb4eda8ee9e9f9a9c35ce2 [term 1 LEADER]: Becoming Leader. State: Replica: d32493cb30cb4eda8ee9e9f9a9c35ce2, State: Running, Role: LEADER
I20260923 22:14:52.142622 15329 sys_catalog.cc:564] T 00000000000000000000000000000000 P d32493cb30cb4eda8ee9e9f9a9c35ce2 [sys.catalog]: configured and running, proceeding with master startup.
I20260923 22:14:52.142654 15332 consensus_queue.cc:237] T 00000000000000000000000000000000 P d32493cb30cb4eda8ee9e9f9a9c35ce2 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d32493cb30cb4eda8ee9e9f9a9c35ce2" member_type: VOTER last_known_addr { host: "127.1.135.254" port: 37515 } }
I20260923 22:14:52.143081 15334 sys_catalog.cc:455] T 00000000000000000000000000000000 P d32493cb30cb4eda8ee9e9f9a9c35ce2 [sys.catalog]: SysCatalogTable state changed. Reason: New leader d32493cb30cb4eda8ee9e9f9a9c35ce2. Latest consensus state: current_term: 1 leader_uuid: "d32493cb30cb4eda8ee9e9f9a9c35ce2" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d32493cb30cb4eda8ee9e9f9a9c35ce2" member_type: VOTER last_known_addr { host: "127.1.135.254" port: 37515 } } }
I20260923 22:14:52.143144 15334 sys_catalog.cc:458] T 00000000000000000000000000000000 P d32493cb30cb4eda8ee9e9f9a9c35ce2 [sys.catalog]: This master's current role is: LEADER
I20260923 22:14:52.143054 15333 sys_catalog.cc:455] T 00000000000000000000000000000000 P d32493cb30cb4eda8ee9e9f9a9c35ce2 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "d32493cb30cb4eda8ee9e9f9a9c35ce2" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d32493cb30cb4eda8ee9e9f9a9c35ce2" member_type: VOTER last_known_addr { host: "127.1.135.254" port: 37515 } } }
I20260923 22:14:52.143245 15333 sys_catalog.cc:458] T 00000000000000000000000000000000 P d32493cb30cb4eda8ee9e9f9a9c35ce2 [sys.catalog]: This master's current role is: LEADER
I20260923 22:14:52.143441 15341 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260923 22:14:52.144196 15341 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260923 22:14:52.144660 1567 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskRiQe_a/build/release/bin/kudu as pid 15261
I20260923 22:14:52.144786 1567 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskRiQe_a/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790201539513534-1567-0/minicluster-data/master-0/wal/instance
I20260923 22:14:52.144984 1567 external_mini_cluster.cc:1428] Setting key a972d3a9b080dc2a07719c17951f9814
I20260923 22:14:52.145642 15341 catalog_manager.cc:1347] Generated new cluster ID: 6fbcafe1f67a46a2a305ad3d03ea6e60
I20260923 22:14:52.145699 15341 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260923 22:14:52.151261 15341 catalog_manager.cc:1370] Generated new certificate authority record
Sep 23 22:14:52 dist-test-slave-s2fj krb5kdc[15248](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790201691, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.1.135.254@KRBTEST.COM
I20260923 22:14:52.152427 15341 catalog_manager.cc:1504] Loading token signing keys...
I20260923 22:14:52.156589 15352 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 22:14:52.145379 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49286 (local address 127.1.135.254:37515)
0923 22:14:52.145758 (+ 379us) server_negotiation.cc:206] Beginning negotiation
0923 22:14:52.145767 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0923 22:14:52.145797 (+ 30us) server_negotiation.cc:407] Connection header received
0923 22:14:52.145903 (+ 106us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 22:14:52.145910 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 22:14:52.145971 (+ 61us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 22:14:52.146093 (+ 122us) server_negotiation.cc:226] Negotiated authn=SASL
0923 22:14:52.146841 (+ 748us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 22:14:52.148213 (+ 1372us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 22:14:52.149268 (+ 1055us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 22:14:52.149844 (+ 576us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 22:14:52.149935 (+ 91us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 22:14:52.152777 (+ 2842us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0923 22:14:52.152780 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0923 22:14:52.152781 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0923 22:14:52.152792 (+ 11us) server_negotiation.cc:922] calling sasl_server_start()
0923 22:14:52.154778 (+ 1986us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0923 22:14:52.155165 (+ 387us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0923 22:14:52.155168 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0923 22:14:52.155168 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0923 22:14:52.155273 (+ 105us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0923 22:14:52.155512 (+ 239us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0923 22:14:52.155513 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0923 22:14:52.155513 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0923 22:14:52.155780 (+ 267us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0923 22:14:52.155920 (+ 140us) server_negotiation.cc:1035] Waiting for connection context
0923 22:14:52.156225 (+ 305us) server_negotiation.cc:299] Negotiation successful
0923 22:14:52.156375 (+ 150us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":338,"thread_start_us":95,"threads_started":1}
I20260923 22:14:52.156950 15341 catalog_manager.cc:5868] T 00000000000000000000000000000000 P d32493cb30cb4eda8ee9e9f9a9c35ce2: Generated new TSK 0
I20260923 22:14:52.157100 15341 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.1.135.193@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.1.135.193@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.1.135.193 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskRiQe_a/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790201539513534-1567-0/krb5kdc/kudu_127.1.135.193.keytab.
Entry for principal kudu/127.1.135.193 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskRiQe_a/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790201539513534-1567-0/krb5kdc/kudu_127.1.135.193.keytab.
I20260923 22:14:52.186157 1567 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskRiQe_a/build/release/bin/kudu
/tmp/dist-test-taskRiQe_a/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskRiQe_a/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790201539513534-1567-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskRiQe_a/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790201539513534-1567-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskRiQe_a/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790201539513534-1567-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskRiQe_a/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790201539513534-1567-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.1.135.193:0
--local_ip_for_outbound_sockets=127.1.135.193
--webserver_interface=127.1.135.193
--webserver_port=0
--tserver_master_addrs=127.1.135.254:37515
--keytab_file=/tmp/dist-test-taskRiQe_a/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790201539513534-1567-0/krb5kdc/kudu_127.1.135.193.keytab
--principal=kudu/127.1.135.193
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.1.135.212:38425
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskRiQe_a/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790201539513534-1567-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskRiQe_a/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790201539513534-1567-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskRiQe_a/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790201539513534-1567-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260923 22:14:52.263756 15355 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260923 22:14:52.263933 15355 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260923 22:14:52.263955 15355 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260923 22:14:52.265051 15355 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260923 22:14:52.265097 15355 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260923 22:14:52.265122 15355 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.1.135.193
I20260923 22:14:52.266495 15355 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.1.135.212:38425
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskRiQe_a/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790201539513534-1567-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskRiQe_a/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790201539513534-1567-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskRiQe_a/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790201539513534-1567-0/krb5kdc/kudu_127.1.135.193.keytab
--principal=kudu/127.1.135.193
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.1.135.193:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskRiQe_a/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790201539513534-1567-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.1.135.193
--webserver_port=0
--tserver_master_addrs=127.1.135.254:37515
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.15355
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.1.135.193
--log_dir=/tmp/dist-test-taskRiQe_a/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790201539513534-1567-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 23 Sep 2026 20:53:37 UTC on ccycloud.quasar-rkviyx.root.comops.site
build id 4668083
I20260923 22:14:52.266726 15355 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260923 22:14:52.266945 15355 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260923 22:14:52.271642 15360 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260923 22:14:52.271673 15363 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260923 22:14:52.271672 15361 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260923 22:14:52.271891 15355 server_base.cc:1034] running on GCE node
I20260923 22:14:52.272145 15355 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260923 22:14:52.272389 15355 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260923 22:14:52.273547 15355 hybrid_clock.cc:648] HybridClock initialized: now 1790201692273529 us; error 30 us; skew 500 ppm
Sep 23 22:14:52 dist-test-slave-s2fj krb5kdc[15248](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790201692, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.135.193@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260923 22:14:52.275949 15355 init.cc:377] Logged in from keytab as kudu/127.1.135.193@KRBTEST.COM (short username kudu)
I20260923 22:14:52.276391 15355 webserver.cc:460] Webserver started at http://127.1.135.193:34527/ using document root <none> and password file <none>
I20260923 22:14:52.276566 15355 fs_manager.cc:359] Metadata directory not provided
I20260923 22:14:52.276607 15355 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260923 22:14:52.276687 15355 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260923 22:14:52.277616 15355 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskRiQe_a/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790201539513534-1567-0/minicluster-data/ts-0/data/instance:
uuid: "2ac26bd2c8eb4b4fb12e08af1af156e2"
format_stamp: "Formatted at 2026-09-23 22:14:52 on dist-test-slave-s2fj"
server_key: "7ce9468d20e1b5fc5cba598c1a59b91d"
server_key_iv: "af2c59483b907adc33e1ce686719613d"
server_key_version: "encryptionkey@0"
I20260923 22:14:52.277887 15355 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskRiQe_a/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790201539513534-1567-0/minicluster-data/ts-0/wal/instance:
uuid: "2ac26bd2c8eb4b4fb12e08af1af156e2"
format_stamp: "Formatted at 2026-09-23 22:14:52 on dist-test-slave-s2fj"
server_key: "7ce9468d20e1b5fc5cba598c1a59b91d"
server_key_iv: "af2c59483b907adc33e1ce686719613d"
server_key_version: "encryptionkey@0"
I20260923 22:14:52.279049 15355 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.002s
I20260923 22:14:52.279706 15370 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260923 22:14:52.279870 15355 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.001s
I20260923 22:14:52.279947 15355 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskRiQe_a/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790201539513534-1567-0/minicluster-data/ts-0/data,/tmp/dist-test-taskRiQe_a/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790201539513534-1567-0/minicluster-data/ts-0/wal
uuid: "2ac26bd2c8eb4b4fb12e08af1af156e2"
format_stamp: "Formatted at 2026-09-23 22:14:52 on dist-test-slave-s2fj"
server_key: "7ce9468d20e1b5fc5cba598c1a59b91d"
server_key_iv: "af2c59483b907adc33e1ce686719613d"
server_key_version: "encryptionkey@0"
I20260923 22:14:52.280000 15355 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskRiQe_a/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790201539513534-1567-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskRiQe_a/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790201539513534-1567-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskRiQe_a/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790201539513534-1567-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260923 22:14:52.307528 15355 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260923 22:14:52.309268 15355 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260923 22:14:52.309369 15355 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260923 22:14:52.309540 15355 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260923 22:14:52.309839 15355 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260923 22:14:52.309868 15355 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260923 22:14:52.309919 15355 ts_tablet_manager.cc:610] Registered 0 tablets
I20260923 22:14:52.309943 15355 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260923 22:14:52.315356 15355 rpc_server.cc:307] RPC server started. Bound to: 127.1.135.193:37137
I20260923 22:14:52.315428 15483 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.1.135.193:37137 every 8 connection(s)
I20260923 22:14:52.315768 15355 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskRiQe_a/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790201539513534-1567-0/minicluster-data/ts-0/data/info.pb
I20260923 22:14:52.320585 1567 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskRiQe_a/build/release/bin/kudu as pid 15355
I20260923 22:14:52.320699 1567 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskRiQe_a/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790201539513534-1567-0/minicluster-data/ts-0/wal/instance
I20260923 22:14:52.320900 1567 external_mini_cluster.cc:1428] Setting key 56c36ca70acb9fd6769073a630739337
Sep 23 22:14:52 dist-test-slave-s2fj krb5kdc[15248](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790201692, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.135.193@KRBTEST.COM for kudu/127.1.135.254@KRBTEST.COM
I20260923 22:14:52.325150 15352 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 22:14:52.316499 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.135.193:47171 (local address 127.1.135.254:37515)
0923 22:14:52.316607 (+ 108us) server_negotiation.cc:206] Beginning negotiation
0923 22:14:52.316609 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0923 22:14:52.316795 (+ 186us) server_negotiation.cc:407] Connection header received
0923 22:14:52.317239 (+ 444us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 22:14:52.317240 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 22:14:52.317250 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 22:14:52.317300 (+ 50us) server_negotiation.cc:226] Negotiated authn=SASL
0923 22:14:52.318138 (+ 838us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 22:14:52.318621 (+ 483us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 22:14:52.319518 (+ 897us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 22:14:52.319703 (+ 185us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 22:14:52.319752 (+ 49us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 22:14:52.322307 (+ 2555us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0923 22:14:52.322309 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0923 22:14:52.322309 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0923 22:14:52.322316 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0923 22:14:52.323727 (+ 1411us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0923 22:14:52.324271 (+ 544us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0923 22:14:52.324272 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0923 22:14:52.324272 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0923 22:14:52.324327 (+ 55us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0923 22:14:52.324589 (+ 262us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0923 22:14:52.324590 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0923 22:14:52.324590 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0923 22:14:52.324686 (+ 96us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0923 22:14:52.324738 (+ 52us) server_negotiation.cc:1035] Waiting for connection context
0923 22:14:52.325025 (+ 287us) server_negotiation.cc:299] Negotiation successful
0923 22:14:52.325090 (+ 65us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":60}
I20260923 22:14:52.325464 15486 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 22:14:52.316463 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.135.254:37515 (local address 127.1.135.193:47171)
0923 22:14:52.316665 (+ 202us) negotiation.cc:107] Waiting for socket to connect
0923 22:14:52.316697 (+ 32us) client_negotiation.cc:174] Beginning negotiation
0923 22:14:52.317135 (+ 438us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0923 22:14:52.317374 (+ 239us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0923 22:14:52.317377 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0923 22:14:52.317886 (+ 509us) client_negotiation.cc:189] Negotiated authn=SASL
0923 22:14:52.318058 (+ 172us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 22:14:52.318062 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 22:14:52.318695 (+ 633us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 22:14:52.318696 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 22:14:52.319421 (+ 725us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 22:14:52.319422 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 22:14:52.319771 (+ 349us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 22:14:52.319772 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 22:14:52.319864 (+ 92us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 22:14:52.320420 (+ 556us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0923 22:14:52.320432 (+ 12us) client_negotiation.cc:656] calling sasl_client_start()
0923 22:14:52.322199 (+ 1767us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0923 22:14:52.323821 (+ 1622us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0923 22:14:52.323828 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0923 22:14:52.323830 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0923 22:14:52.324180 (+ 350us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0923 22:14:52.324394 (+ 214us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0923 22:14:52.324395 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0923 22:14:52.324395 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0923 22:14:52.324526 (+ 131us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0923 22:14:52.324748 (+ 222us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0923 22:14:52.324754 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0923 22:14:52.324898 (+ 144us) client_negotiation.cc:769] Sending connection context
0923 22:14:52.325003 (+ 105us) client_negotiation.cc:240] Negotiation successful
0923 22:14:52.325174 (+ 171us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":169,"thread_start_us":133,"threads_started":1}
I20260923 22:14:52.326068 15484 heartbeater.cc:347] Connected to a master server at 127.1.135.254:37515
I20260923 22:14:52.326172 15484 heartbeater.cc:464] Registering TS with master...
I20260923 22:14:52.326356 15484 heartbeater.cc:511] Master 127.1.135.254:37515 requested a full tablet report, sending...
I20260923 22:14:52.326846 15293 ts_manager.cc:194] Registered new tserver with Master: 2ac26bd2c8eb4b4fb12e08af1af156e2 (127.1.135.193:37137)
I20260923 22:14:52.327430 15293 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.1.135.193@KRBTEST.COM'} at 127.1.135.193:47171
WARNING: no policy specified for kudu/127.1.135.194@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.1.135.194@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.1.135.194 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskRiQe_a/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790201539513534-1567-0/krb5kdc/kudu_127.1.135.194.keytab.
Entry for principal kudu/127.1.135.194 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskRiQe_a/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790201539513534-1567-0/krb5kdc/kudu_127.1.135.194.keytab.
I20260923 22:14:52.346823 1567 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskRiQe_a/build/release/bin/kudu
/tmp/dist-test-taskRiQe_a/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskRiQe_a/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790201539513534-1567-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskRiQe_a/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790201539513534-1567-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskRiQe_a/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790201539513534-1567-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskRiQe_a/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790201539513534-1567-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.1.135.194:0
--local_ip_for_outbound_sockets=127.1.135.194
--webserver_interface=127.1.135.194
--webserver_port=0
--tserver_master_addrs=127.1.135.254:37515
--keytab_file=/tmp/dist-test-taskRiQe_a/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790201539513534-1567-0/krb5kdc/kudu_127.1.135.194.keytab
--principal=kudu/127.1.135.194
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.1.135.212:38425
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskRiQe_a/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790201539513534-1567-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskRiQe_a/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790201539513534-1567-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskRiQe_a/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790201539513534-1567-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260923 22:14:52.423979 15489 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260923 22:14:52.424209 15489 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260923 22:14:52.424232 15489 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260923 22:14:52.425379 15489 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260923 22:14:52.425419 15489 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260923 22:14:52.425469 15489 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.1.135.194
I20260923 22:14:52.426822 15489 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.1.135.212:38425
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskRiQe_a/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790201539513534-1567-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskRiQe_a/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790201539513534-1567-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskRiQe_a/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790201539513534-1567-0/krb5kdc/kudu_127.1.135.194.keytab
--principal=kudu/127.1.135.194
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.1.135.194:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskRiQe_a/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790201539513534-1567-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.1.135.194
--webserver_port=0
--tserver_master_addrs=127.1.135.254:37515
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.15489
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.1.135.194
--log_dir=/tmp/dist-test-taskRiQe_a/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790201539513534-1567-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 23 Sep 2026 20:53:37 UTC on ccycloud.quasar-rkviyx.root.comops.site
build id 4668083
I20260923 22:14:52.427014 15489 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260923 22:14:52.427209 15489 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260923 22:14:52.431794 15495 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260923 22:14:52.431932 15489 server_base.cc:1034] running on GCE node
W20260923 22:14:52.431803 15494 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260923 22:14:52.431803 15497 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260923 22:14:52.432247 15489 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260923 22:14:52.432471 15489 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260923 22:14:52.433611 15489 hybrid_clock.cc:648] HybridClock initialized: now 1790201692433600 us; error 27 us; skew 500 ppm
Sep 23 22:14:52 dist-test-slave-s2fj krb5kdc[15248](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790201692, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.135.194@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260923 22:14:52.436033 15489 init.cc:377] Logged in from keytab as kudu/127.1.135.194@KRBTEST.COM (short username kudu)
I20260923 22:14:52.436509 15489 webserver.cc:460] Webserver started at http://127.1.135.194:36489/ using document root <none> and password file <none>
I20260923 22:14:52.436726 15489 fs_manager.cc:359] Metadata directory not provided
I20260923 22:14:52.436789 15489 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260923 22:14:52.436929 15489 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260923 22:14:52.437879 15489 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskRiQe_a/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790201539513534-1567-0/minicluster-data/ts-1/data/instance:
uuid: "7d6af16a92f948c98fa0a8c4d04da648"
format_stamp: "Formatted at 2026-09-23 22:14:52 on dist-test-slave-s2fj"
server_key: "5ed27962f37132d31849db12e425c61f"
server_key_iv: "ce92088ae7b1e9838e4e018406906389"
server_key_version: "encryptionkey@0"
I20260923 22:14:52.438213 15489 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskRiQe_a/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790201539513534-1567-0/minicluster-data/ts-1/wal/instance:
uuid: "7d6af16a92f948c98fa0a8c4d04da648"
format_stamp: "Formatted at 2026-09-23 22:14:52 on dist-test-slave-s2fj"
server_key: "5ed27962f37132d31849db12e425c61f"
server_key_iv: "ce92088ae7b1e9838e4e018406906389"
server_key_version: "encryptionkey@0"
I20260923 22:14:52.439438 15489 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260923 22:14:52.440135 15504 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260923 22:14:52.440310 15489 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260923 22:14:52.440397 15489 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskRiQe_a/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790201539513534-1567-0/minicluster-data/ts-1/data,/tmp/dist-test-taskRiQe_a/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790201539513534-1567-0/minicluster-data/ts-1/wal
uuid: "7d6af16a92f948c98fa0a8c4d04da648"
format_stamp: "Formatted at 2026-09-23 22:14:52 on dist-test-slave-s2fj"
server_key: "5ed27962f37132d31849db12e425c61f"
server_key_iv: "ce92088ae7b1e9838e4e018406906389"
server_key_version: "encryptionkey@0"
I20260923 22:14:52.440481 15489 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskRiQe_a/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790201539513534-1567-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskRiQe_a/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790201539513534-1567-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskRiQe_a/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790201539513534-1567-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260923 22:14:52.455106 15489 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260923 22:14:52.457187 15489 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260923 22:14:52.457286 15489 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260923 22:14:52.457434 15489 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260923 22:14:52.457731 15489 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260923 22:14:52.457764 15489 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260923 22:14:52.457785 15489 ts_tablet_manager.cc:610] Registered 0 tablets
I20260923 22:14:52.457842 15489 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260923 22:14:52.463407 15489 rpc_server.cc:307] RPC server started. Bound to: 127.1.135.194:46495
I20260923 22:14:52.463474 15617 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.1.135.194:46495 every 8 connection(s)
I20260923 22:14:52.463769 15489 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskRiQe_a/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790201539513534-1567-0/minicluster-data/ts-1/data/info.pb
Sep 23 22:14:52 dist-test-slave-s2fj krb5kdc[15248](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790201692, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.135.194@KRBTEST.COM for kudu/127.1.135.254@KRBTEST.COM
I20260923 22:14:52.471520 1567 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskRiQe_a/build/release/bin/kudu as pid 15489
I20260923 22:14:52.471637 1567 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskRiQe_a/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790201539513534-1567-0/minicluster-data/ts-1/wal/instance
I20260923 22:14:52.471843 1567 external_mini_cluster.cc:1428] Setting key 74f85348d95b18f93263f138ce0fec35
I20260923 22:14:52.472360 15352 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 22:14:52.464469 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.135.194:44547 (local address 127.1.135.254:37515)
0923 22:14:52.464558 (+ 89us) server_negotiation.cc:206] Beginning negotiation
0923 22:14:52.464561 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0923 22:14:52.464701 (+ 140us) server_negotiation.cc:407] Connection header received
0923 22:14:52.465174 (+ 473us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 22:14:52.465175 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 22:14:52.465186 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 22:14:52.465248 (+ 62us) server_negotiation.cc:226] Negotiated authn=SASL
0923 22:14:52.466093 (+ 845us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 22:14:52.466580 (+ 487us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 22:14:52.467521 (+ 941us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 22:14:52.467728 (+ 207us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 22:14:52.467808 (+ 80us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 22:14:52.469715 (+ 1907us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0923 22:14:52.469717 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0923 22:14:52.469718 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0923 22:14:52.469725 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0923 22:14:52.470882 (+ 1157us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0923 22:14:52.471279 (+ 397us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0923 22:14:52.471280 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0923 22:14:52.471281 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0923 22:14:52.471341 (+ 60us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0923 22:14:52.471637 (+ 296us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0923 22:14:52.471638 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0923 22:14:52.471638 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0923 22:14:52.471769 (+ 131us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0923 22:14:52.471854 (+ 85us) server_negotiation.cc:1035] Waiting for connection context
0923 22:14:52.472203 (+ 349us) server_negotiation.cc:299] Negotiation successful
0923 22:14:52.472301 (+ 98us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":38}
I20260923 22:14:52.472558 15620 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 22:14:52.464418 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.135.254:37515 (local address 127.1.135.194:44547)
0923 22:14:52.464585 (+ 167us) negotiation.cc:107] Waiting for socket to connect
0923 22:14:52.464630 (+ 45us) client_negotiation.cc:174] Beginning negotiation
0923 22:14:52.465076 (+ 446us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0923 22:14:52.465310 (+ 234us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0923 22:14:52.465313 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0923 22:14:52.465817 (+ 504us) client_negotiation.cc:189] Negotiated authn=SASL
0923 22:14:52.465982 (+ 165us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 22:14:52.465986 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 22:14:52.466672 (+ 686us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 22:14:52.466673 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 22:14:52.467419 (+ 746us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 22:14:52.467421 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 22:14:52.467818 (+ 397us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 22:14:52.467819 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 22:14:52.467932 (+ 113us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 22:14:52.468386 (+ 454us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0923 22:14:52.468393 (+ 7us) client_negotiation.cc:656] calling sasl_client_start()
0923 22:14:52.469622 (+ 1229us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0923 22:14:52.470980 (+ 1358us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0923 22:14:52.470987 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0923 22:14:52.470989 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0923 22:14:52.471202 (+ 213us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0923 22:14:52.471431 (+ 229us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0923 22:14:52.471432 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0923 22:14:52.471432 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0923 22:14:52.471536 (+ 104us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0923 22:14:52.471845 (+ 309us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0923 22:14:52.471851 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0923 22:14:52.472018 (+ 167us) client_negotiation.cc:769] Sending connection context
0923 22:14:52.472164 (+ 146us) client_negotiation.cc:240] Negotiation successful
0923 22:14:52.472349 (+ 185us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":128,"thread_start_us":84,"threads_started":1}
I20260923 22:14:52.472977 15618 heartbeater.cc:347] Connected to a master server at 127.1.135.254:37515
I20260923 22:14:52.473074 15618 heartbeater.cc:464] Registering TS with master...
I20260923 22:14:52.473237 15618 heartbeater.cc:511] Master 127.1.135.254:37515 requested a full tablet report, sending...
I20260923 22:14:52.473578 15293 ts_manager.cc:194] Registered new tserver with Master: 7d6af16a92f948c98fa0a8c4d04da648 (127.1.135.194:46495)
I20260923 22:14:52.473968 15293 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.1.135.194@KRBTEST.COM'} at 127.1.135.194:44547
WARNING: no policy specified for kudu/127.1.135.195@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.1.135.195@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.1.135.195 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskRiQe_a/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790201539513534-1567-0/krb5kdc/kudu_127.1.135.195.keytab.
Entry for principal kudu/127.1.135.195 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskRiQe_a/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790201539513534-1567-0/krb5kdc/kudu_127.1.135.195.keytab.
I20260923 22:14:52.495205 1567 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskRiQe_a/build/release/bin/kudu
/tmp/dist-test-taskRiQe_a/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskRiQe_a/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790201539513534-1567-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskRiQe_a/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790201539513534-1567-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskRiQe_a/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790201539513534-1567-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskRiQe_a/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790201539513534-1567-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.1.135.195:0
--local_ip_for_outbound_sockets=127.1.135.195
--webserver_interface=127.1.135.195
--webserver_port=0
--tserver_master_addrs=127.1.135.254:37515
--keytab_file=/tmp/dist-test-taskRiQe_a/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790201539513534-1567-0/krb5kdc/kudu_127.1.135.195.keytab
--principal=kudu/127.1.135.195
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.1.135.212:38425
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskRiQe_a/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790201539513534-1567-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskRiQe_a/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790201539513534-1567-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskRiQe_a/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790201539513534-1567-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260923 22:14:52.571916 15623 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260923 22:14:52.572099 15623 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260923 22:14:52.572121 15623 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260923 22:14:52.573215 15623 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260923 22:14:52.573248 15623 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260923 22:14:52.573273 15623 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.1.135.195
I20260923 22:14:52.574517 15623 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.1.135.212:38425
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskRiQe_a/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790201539513534-1567-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskRiQe_a/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790201539513534-1567-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskRiQe_a/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790201539513534-1567-0/krb5kdc/kudu_127.1.135.195.keytab
--principal=kudu/127.1.135.195
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.1.135.195:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskRiQe_a/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790201539513534-1567-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.1.135.195
--webserver_port=0
--tserver_master_addrs=127.1.135.254:37515
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.15623
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.1.135.195
--log_dir=/tmp/dist-test-taskRiQe_a/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790201539513534-1567-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 23 Sep 2026 20:53:37 UTC on ccycloud.quasar-rkviyx.root.comops.site
build id 4668083
I20260923 22:14:52.574693 15623 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260923 22:14:52.574841 15623 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260923 22:14:52.579473 15631 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260923 22:14:52.579469 15629 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260923 22:14:52.579453 15628 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260923 22:14:52.579727 15623 server_base.cc:1034] running on GCE node
I20260923 22:14:52.579957 15623 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260923 22:14:52.580221 15623 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260923 22:14:52.581379 15623 hybrid_clock.cc:648] HybridClock initialized: now 1790201692581369 us; error 27 us; skew 500 ppm
Sep 23 22:14:52 dist-test-slave-s2fj krb5kdc[15248](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790201692, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.135.195@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260923 22:14:52.583750 15623 init.cc:377] Logged in from keytab as kudu/127.1.135.195@KRBTEST.COM (short username kudu)
I20260923 22:14:52.584236 15623 webserver.cc:460] Webserver started at http://127.1.135.195:37309/ using document root <none> and password file <none>
I20260923 22:14:52.584414 15623 fs_manager.cc:359] Metadata directory not provided
I20260923 22:14:52.584451 15623 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260923 22:14:52.584532 15623 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260923 22:14:52.585412 15623 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskRiQe_a/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790201539513534-1567-0/minicluster-data/ts-2/data/instance:
uuid: "7f8b6e024a774ca996fd7858a93c42b2"
format_stamp: "Formatted at 2026-09-23 22:14:52 on dist-test-slave-s2fj"
server_key: "93f7a16cb6e8af959504c1302d41db8e"
server_key_iv: "8b41b00d37b8db1e35715c0c63d26e2d"
server_key_version: "encryptionkey@0"
I20260923 22:14:52.585692 15623 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskRiQe_a/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790201539513534-1567-0/minicluster-data/ts-2/wal/instance:
uuid: "7f8b6e024a774ca996fd7858a93c42b2"
format_stamp: "Formatted at 2026-09-23 22:14:52 on dist-test-slave-s2fj"
server_key: "93f7a16cb6e8af959504c1302d41db8e"
server_key_iv: "8b41b00d37b8db1e35715c0c63d26e2d"
server_key_version: "encryptionkey@0"
I20260923 22:14:52.586796 15623 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.002s
I20260923 22:14:52.587399 15638 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260923 22:14:52.587571 15623 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.001s
I20260923 22:14:52.587656 15623 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskRiQe_a/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790201539513534-1567-0/minicluster-data/ts-2/data,/tmp/dist-test-taskRiQe_a/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790201539513534-1567-0/minicluster-data/ts-2/wal
uuid: "7f8b6e024a774ca996fd7858a93c42b2"
format_stamp: "Formatted at 2026-09-23 22:14:52 on dist-test-slave-s2fj"
server_key: "93f7a16cb6e8af959504c1302d41db8e"
server_key_iv: "8b41b00d37b8db1e35715c0c63d26e2d"
server_key_version: "encryptionkey@0"
I20260923 22:14:52.587734 15623 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskRiQe_a/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790201539513534-1567-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskRiQe_a/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790201539513534-1567-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskRiQe_a/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790201539513534-1567-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260923 22:14:52.615324 15623 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260923 22:14:52.617028 15623 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260923 22:14:52.617179 15623 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260923 22:14:52.617380 15623 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260923 22:14:52.617666 15623 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260923 22:14:52.617717 15623 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260923 22:14:52.617754 15623 ts_tablet_manager.cc:610] Registered 0 tablets
I20260923 22:14:52.617781 15623 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260923 22:14:52.623466 15623 rpc_server.cc:307] RPC server started. Bound to: 127.1.135.195:34569
I20260923 22:14:52.623517 15751 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.1.135.195:34569 every 8 connection(s)
I20260923 22:14:52.623829 15623 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskRiQe_a/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790201539513534-1567-0/minicluster-data/ts-2/data/info.pb
Sep 23 22:14:52 dist-test-slave-s2fj krb5kdc[15248](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790201692, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.135.195@KRBTEST.COM for kudu/127.1.135.254@KRBTEST.COM
I20260923 22:14:52.629664 1567 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskRiQe_a/build/release/bin/kudu as pid 15623
I20260923 22:14:52.629756 1567 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskRiQe_a/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790201539513534-1567-0/minicluster-data/ts-2/wal/instance
I20260923 22:14:52.629930 1567 external_mini_cluster.cc:1428] Setting key b9dd8b469cc285bfbf2eeb1a076bf1a4
I20260923 22:14:52.632333 15352 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 22:14:52.624542 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.135.195:60145 (local address 127.1.135.254:37515)
0923 22:14:52.624585 (+ 43us) server_negotiation.cc:206] Beginning negotiation
0923 22:14:52.624587 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0923 22:14:52.624735 (+ 148us) server_negotiation.cc:407] Connection header received
0923 22:14:52.625155 (+ 420us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 22:14:52.625156 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 22:14:52.625164 (+ 8us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 22:14:52.625232 (+ 68us) server_negotiation.cc:226] Negotiated authn=SASL
0923 22:14:52.626020 (+ 788us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 22:14:52.626457 (+ 437us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 22:14:52.627306 (+ 849us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 22:14:52.627479 (+ 173us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 22:14:52.627523 (+ 44us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 22:14:52.629585 (+ 2062us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0923 22:14:52.629587 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0923 22:14:52.629588 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0923 22:14:52.629596 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0923 22:14:52.631219 (+ 1623us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0923 22:14:52.631625 (+ 406us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0923 22:14:52.631625 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0923 22:14:52.631625 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0923 22:14:52.631669 (+ 44us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0923 22:14:52.631851 (+ 182us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0923 22:14:52.631852 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0923 22:14:52.631852 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0923 22:14:52.631934 (+ 82us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0923 22:14:52.631978 (+ 44us) server_negotiation.cc:1035] Waiting for connection context
0923 22:14:52.632228 (+ 250us) server_negotiation.cc:299] Negotiation successful
0923 22:14:52.632279 (+ 51us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":20}
I20260923 22:14:52.632556 15754 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 22:14:52.624503 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.135.254:37515 (local address 127.1.135.195:60145)
0923 22:14:52.624638 (+ 135us) negotiation.cc:107] Waiting for socket to connect
0923 22:14:52.624673 (+ 35us) client_negotiation.cc:174] Beginning negotiation
0923 22:14:52.625078 (+ 405us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0923 22:14:52.625277 (+ 199us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0923 22:14:52.625279 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0923 22:14:52.625794 (+ 515us) client_negotiation.cc:189] Negotiated authn=SASL
0923 22:14:52.625951 (+ 157us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 22:14:52.625956 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 22:14:52.626526 (+ 570us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 22:14:52.626527 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 22:14:52.627233 (+ 706us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 22:14:52.627235 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 22:14:52.627549 (+ 314us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 22:14:52.627550 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 22:14:52.627635 (+ 85us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 22:14:52.627980 (+ 345us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0923 22:14:52.627987 (+ 7us) client_negotiation.cc:656] calling sasl_client_start()
0923 22:14:52.629480 (+ 1493us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0923 22:14:52.631294 (+ 1814us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0923 22:14:52.631300 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0923 22:14:52.631303 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0923 22:14:52.631565 (+ 262us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0923 22:14:52.631719 (+ 154us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0923 22:14:52.631720 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0923 22:14:52.631720 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0923 22:14:52.631803 (+ 83us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0923 22:14:52.631982 (+ 179us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0923 22:14:52.631985 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0923 22:14:52.632128 (+ 143us) client_negotiation.cc:769] Sending connection context
0923 22:14:52.632204 (+ 76us) client_negotiation.cc:240] Negotiation successful
0923 22:14:52.632301 (+ 97us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":101,"thread_start_us":78,"threads_started":1}
I20260923 22:14:52.632990 15752 heartbeater.cc:347] Connected to a master server at 127.1.135.254:37515
I20260923 22:14:52.633080 15752 heartbeater.cc:464] Registering TS with master...
I20260923 22:14:52.633255 15752 heartbeater.cc:511] Master 127.1.135.254:37515 requested a full tablet report, sending...
I20260923 22:14:52.633612 15293 ts_manager.cc:194] Registered new tserver with Master: 7f8b6e024a774ca996fd7858a93c42b2 (127.1.135.195:34569)
I20260923 22:14:52.633926 15293 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.1.135.195@KRBTEST.COM'} at 127.1.135.195:60145
I20260923 22:14:52.641742 1567 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260923 22:14:52.649214 15352 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 22:14:52.643136 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49298 (local address 127.1.135.254:37515)
0923 22:14:52.643196 (+ 60us) server_negotiation.cc:206] Beginning negotiation
0923 22:14:52.643199 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0923 22:14:52.643313 (+ 114us) server_negotiation.cc:407] Connection header received
0923 22:14:52.643390 (+ 77us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 22:14:52.643391 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 22:14:52.643400 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 22:14:52.643469 (+ 69us) server_negotiation.cc:226] Negotiated authn=SASL
0923 22:14:52.644153 (+ 684us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 22:14:52.644613 (+ 460us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 22:14:52.645346 (+ 733us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 22:14:52.645512 (+ 166us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 22:14:52.645610 (+ 98us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 22:14:52.646888 (+ 1278us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0923 22:14:52.646889 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0923 22:14:52.646890 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0923 22:14:52.646897 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0923 22:14:52.648009 (+ 1112us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0923 22:14:52.648393 (+ 384us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0923 22:14:52.648395 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0923 22:14:52.648395 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0923 22:14:52.648450 (+ 55us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0923 22:14:52.648648 (+ 198us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0923 22:14:52.648649 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0923 22:14:52.648649 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0923 22:14:52.648786 (+ 137us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0923 22:14:52.648885 (+ 99us) server_negotiation.cc:1035] Waiting for connection context
0923 22:14:52.649034 (+ 149us) server_negotiation.cc:299] Negotiation successful
0923 22:14:52.649115 (+ 81us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":35}
I20260923 22:14:52.650574 15293 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:49298:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260923 22:14:52.652046 15293 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260923 22:14:52.658500 15765 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 22:14:52.654486 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42540 (local address 127.1.135.195:34569)
0923 22:14:52.654941 (+ 455us) server_negotiation.cc:206] Beginning negotiation
0923 22:14:52.655155 (+ 214us) server_negotiation.cc:399] Waiting for connection header
0923 22:14:52.655165 (+ 10us) server_negotiation.cc:407] Connection header received
0923 22:14:52.655205 (+ 40us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 22:14:52.655207 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 22:14:52.655222 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 22:14:52.655295 (+ 73us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0923 22:14:52.655415 (+ 120us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 22:14:52.656490 (+ 1075us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 22:14:52.657689 (+ 1199us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 22:14:52.658198 (+ 509us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 22:14:52.658268 (+ 70us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 22:14:52.658321 (+ 53us) server_negotiation.cc:1035] Waiting for connection context
0923 22:14:52.658386 (+ 65us) server_negotiation.cc:299] Negotiation successful
0923 22:14:52.658424 (+ 38us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":414,"thread_start_us":98,"threads_started":1}
I20260923 22:14:52.658485 15760 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 22:14:52.654171 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.135.193:37137 (local address 127.0.0.1:55652)
0923 22:14:52.654277 (+ 106us) negotiation.cc:107] Waiting for socket to connect
0923 22:14:52.654308 (+ 31us) client_negotiation.cc:174] Beginning negotiation
0923 22:14:52.654371 (+ 63us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0923 22:14:52.654857 (+ 486us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0923 22:14:52.654858 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0923 22:14:52.654861 (+ 3us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0923 22:14:52.654970 (+ 109us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 22:14:52.654973 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 22:14:52.655988 (+ 1015us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 22:14:52.655989 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 22:14:52.657322 (+ 1333us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 22:14:52.657325 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 22:14:52.658174 (+ 849us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 22:14:52.658175 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 22:14:52.658260 (+ 85us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 22:14:52.658275 (+ 15us) client_negotiation.cc:769] Sending connection context
0923 22:14:52.658341 (+ 66us) client_negotiation.cc:240] Negotiation successful
0923 22:14:52.658402 (+ 61us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":98,"thread_start_us":50,"threads_started":1}
I20260923 22:14:52.658526 15764 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 22:14:52.654483 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48736 (local address 127.1.135.194:46495)
0923 22:14:52.654659 (+ 176us) server_negotiation.cc:206] Beginning negotiation
0923 22:14:52.654662 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0923 22:14:52.654670 (+ 8us) server_negotiation.cc:407] Connection header received
0923 22:14:52.654700 (+ 30us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 22:14:52.654702 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 22:14:52.654713 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 22:14:52.654799 (+ 86us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0923 22:14:52.654954 (+ 155us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 22:14:52.656109 (+ 1155us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 22:14:52.657357 (+ 1248us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 22:14:52.658078 (+ 721us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 22:14:52.658129 (+ 51us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 22:14:52.658186 (+ 57us) server_negotiation.cc:1035] Waiting for connection context
0923 22:14:52.658404 (+ 218us) server_negotiation.cc:299] Negotiation successful
0923 22:14:52.658445 (+ 41us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":147,"thread_start_us":98,"threads_started":1}
I20260923 22:14:52.658488 15762 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 22:14:52.654332 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.135.195:34569 (local address 127.0.0.1:42540)
0923 22:14:52.654999 (+ 667us) negotiation.cc:107] Waiting for socket to connect
0923 22:14:52.655009 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0923 22:14:52.655061 (+ 52us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0923 22:14:52.655313 (+ 252us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0923 22:14:52.655314 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0923 22:14:52.655316 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0923 22:14:52.655357 (+ 41us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 22:14:52.655358 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 22:14:52.656590 (+ 1232us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 22:14:52.656591 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 22:14:52.657605 (+ 1014us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 22:14:52.657607 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 22:14:52.658267 (+ 660us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 22:14:52.658267 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 22:14:52.658314 (+ 47us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 22:14:52.658317 (+ 3us) client_negotiation.cc:769] Sending connection context
0923 22:14:52.658358 (+ 41us) client_negotiation.cc:240] Negotiation successful
0923 22:14:52.658403 (+ 45us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":652,"mutex_wait_us":10,"thread_start_us":35,"threads_started":1}
I20260923 22:14:52.658712 15761 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 22:14:52.654126 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.135.194:46495 (local address 127.0.0.1:48736)
0923 22:14:52.654269 (+ 143us) negotiation.cc:107] Waiting for socket to connect
0923 22:14:52.654308 (+ 39us) client_negotiation.cc:174] Beginning negotiation
0923 22:14:52.654371 (+ 63us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0923 22:14:52.654799 (+ 428us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0923 22:14:52.654802 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0923 22:14:52.654804 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0923 22:14:52.654863 (+ 59us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 22:14:52.654865 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 22:14:52.656242 (+ 1377us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 22:14:52.656242 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 22:14:52.657244 (+ 1002us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 22:14:52.657246 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 22:14:52.658247 (+ 1001us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 22:14:52.658248 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 22:14:52.658310 (+ 62us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 22:14:52.658314 (+ 4us) client_negotiation.cc:769] Sending connection context
0923 22:14:52.658607 (+ 293us) client_negotiation.cc:240] Negotiation successful
0923 22:14:52.658651 (+ 44us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":117,"thread_start_us":74,"threads_started":1}
I20260923 22:14:52.658859 15763 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 22:14:52.654297 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55652 (local address 127.1.135.193:37137)
0923 22:14:52.654702 (+ 405us) server_negotiation.cc:206] Beginning negotiation
0923 22:14:52.654706 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0923 22:14:52.654714 (+ 8us) server_negotiation.cc:407] Connection header received
0923 22:14:52.654745 (+ 31us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 22:14:52.654746 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 22:14:52.654757 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 22:14:52.654846 (+ 89us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0923 22:14:52.655053 (+ 207us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 22:14:52.655888 (+ 835us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 22:14:52.657400 (+ 1512us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 22:14:52.658078 (+ 678us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 22:14:52.658154 (+ 76us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 22:14:52.658202 (+ 48us) server_negotiation.cc:1035] Waiting for connection context
0923 22:14:52.658715 (+ 513us) server_negotiation.cc:299] Negotiation successful
0923 22:14:52.658772 (+ 57us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":376,"thread_start_us":78,"threads_started":1}
I20260923 22:14:52.659590 15552 tablet_service.cc:1467] Processing CreateTablet for tablet 3fc83020634044638e2efad4a6bef386 (DEFAULT_TABLE table=test-table [id=0098df898fff4bf69cf3519e4d5d48ab]), partition=RANGE (key) PARTITION UNBOUNDED
I20260923 22:14:52.659545 15418 tablet_service.cc:1467] Processing CreateTablet for tablet 3fc83020634044638e2efad4a6bef386 (DEFAULT_TABLE table=test-table [id=0098df898fff4bf69cf3519e4d5d48ab]), partition=RANGE (key) PARTITION UNBOUNDED
I20260923 22:14:52.659906 15418 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 3fc83020634044638e2efad4a6bef386. 1 dirs total, 0 dirs full, 0 dirs failed
I20260923 22:14:52.659906 15552 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 3fc83020634044638e2efad4a6bef386. 1 dirs total, 0 dirs full, 0 dirs failed
I20260923 22:14:52.660421 15686 tablet_service.cc:1467] Processing CreateTablet for tablet 3fc83020634044638e2efad4a6bef386 (DEFAULT_TABLE table=test-table [id=0098df898fff4bf69cf3519e4d5d48ab]), partition=RANGE (key) PARTITION UNBOUNDED
I20260923 22:14:52.660725 15686 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 3fc83020634044638e2efad4a6bef386. 1 dirs total, 0 dirs full, 0 dirs failed
I20260923 22:14:52.663059 15766 tablet_bootstrap.cc:492] T 3fc83020634044638e2efad4a6bef386 P 7f8b6e024a774ca996fd7858a93c42b2: Bootstrap starting.
I20260923 22:14:52.663456 15768 tablet_bootstrap.cc:492] T 3fc83020634044638e2efad4a6bef386 P 2ac26bd2c8eb4b4fb12e08af1af156e2: Bootstrap starting.
I20260923 22:14:52.663795 15766 tablet_bootstrap.cc:654] T 3fc83020634044638e2efad4a6bef386 P 7f8b6e024a774ca996fd7858a93c42b2: Neither blocks nor log segments found. Creating new log.
I20260923 22:14:52.664125 15768 tablet_bootstrap.cc:654] T 3fc83020634044638e2efad4a6bef386 P 2ac26bd2c8eb4b4fb12e08af1af156e2: Neither blocks nor log segments found. Creating new log.
I20260923 22:14:52.664238 15767 tablet_bootstrap.cc:492] T 3fc83020634044638e2efad4a6bef386 P 7d6af16a92f948c98fa0a8c4d04da648: Bootstrap starting.
I20260923 22:14:52.664458 15766 log.cc:824] T 3fc83020634044638e2efad4a6bef386 P 7f8b6e024a774ca996fd7858a93c42b2: Log is configured to *not* fsync() on all Append() calls
I20260923 22:14:52.664700 15768 log.cc:824] T 3fc83020634044638e2efad4a6bef386 P 2ac26bd2c8eb4b4fb12e08af1af156e2: Log is configured to *not* fsync() on all Append() calls
I20260923 22:14:52.664817 15767 tablet_bootstrap.cc:654] T 3fc83020634044638e2efad4a6bef386 P 7d6af16a92f948c98fa0a8c4d04da648: Neither blocks nor log segments found. Creating new log.
I20260923 22:14:52.665256 15766 tablet_bootstrap.cc:492] T 3fc83020634044638e2efad4a6bef386 P 7f8b6e024a774ca996fd7858a93c42b2: No bootstrap required, opened a new log
I20260923 22:14:52.665369 15766 ts_tablet_manager.cc:1397] T 3fc83020634044638e2efad4a6bef386 P 7f8b6e024a774ca996fd7858a93c42b2: Time spent bootstrapping tablet: real 0.002s user 0.000s sys 0.002s
I20260923 22:14:52.665385 15767 log.cc:824] T 3fc83020634044638e2efad4a6bef386 P 7d6af16a92f948c98fa0a8c4d04da648: Log is configured to *not* fsync() on all Append() calls
I20260923 22:14:52.665472 15768 tablet_bootstrap.cc:492] T 3fc83020634044638e2efad4a6bef386 P 2ac26bd2c8eb4b4fb12e08af1af156e2: No bootstrap required, opened a new log
I20260923 22:14:52.665532 15768 ts_tablet_manager.cc:1397] T 3fc83020634044638e2efad4a6bef386 P 2ac26bd2c8eb4b4fb12e08af1af156e2: Time spent bootstrapping tablet: real 0.002s user 0.000s sys 0.002s
I20260923 22:14:52.666023 15767 tablet_bootstrap.cc:492] T 3fc83020634044638e2efad4a6bef386 P 7d6af16a92f948c98fa0a8c4d04da648: No bootstrap required, opened a new log
I20260923 22:14:52.666106 15767 ts_tablet_manager.cc:1397] T 3fc83020634044638e2efad4a6bef386 P 7d6af16a92f948c98fa0a8c4d04da648: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20260923 22:14:52.667344 15767 raft_consensus.cc:348] T 3fc83020634044638e2efad4a6bef386 P 7d6af16a92f948c98fa0a8c4d04da648 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7d6af16a92f948c98fa0a8c4d04da648" member_type: VOTER last_known_addr { host: "127.1.135.194" port: 46495 } } peers { permanent_uuid: "2ac26bd2c8eb4b4fb12e08af1af156e2" member_type: VOTER last_known_addr { host: "127.1.135.193" port: 37137 } } peers { permanent_uuid: "7f8b6e024a774ca996fd7858a93c42b2" member_type: VOTER last_known_addr { host: "127.1.135.195" port: 34569 } }
I20260923 22:14:52.667500 15767 raft_consensus.cc:374] T 3fc83020634044638e2efad4a6bef386 P 7d6af16a92f948c98fa0a8c4d04da648 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260923 22:14:52.667407 15766 raft_consensus.cc:348] T 3fc83020634044638e2efad4a6bef386 P 7f8b6e024a774ca996fd7858a93c42b2 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7d6af16a92f948c98fa0a8c4d04da648" member_type: VOTER last_known_addr { host: "127.1.135.194" port: 46495 } } peers { permanent_uuid: "2ac26bd2c8eb4b4fb12e08af1af156e2" member_type: VOTER last_known_addr { host: "127.1.135.193" port: 37137 } } peers { permanent_uuid: "7f8b6e024a774ca996fd7858a93c42b2" member_type: VOTER last_known_addr { host: "127.1.135.195" port: 34569 } }
I20260923 22:14:52.667546 15767 raft_consensus.cc:729] T 3fc83020634044638e2efad4a6bef386 P 7d6af16a92f948c98fa0a8c4d04da648 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 7d6af16a92f948c98fa0a8c4d04da648, State: Initialized, Role: FOLLOWER
I20260923 22:14:52.667407 15768 raft_consensus.cc:348] T 3fc83020634044638e2efad4a6bef386 P 2ac26bd2c8eb4b4fb12e08af1af156e2 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7d6af16a92f948c98fa0a8c4d04da648" member_type: VOTER last_known_addr { host: "127.1.135.194" port: 46495 } } peers { permanent_uuid: "2ac26bd2c8eb4b4fb12e08af1af156e2" member_type: VOTER last_known_addr { host: "127.1.135.193" port: 37137 } } peers { permanent_uuid: "7f8b6e024a774ca996fd7858a93c42b2" member_type: VOTER last_known_addr { host: "127.1.135.195" port: 34569 } }
I20260923 22:14:52.667562 15766 raft_consensus.cc:374] T 3fc83020634044638e2efad4a6bef386 P 7f8b6e024a774ca996fd7858a93c42b2 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260923 22:14:52.667582 15768 raft_consensus.cc:374] T 3fc83020634044638e2efad4a6bef386 P 2ac26bd2c8eb4b4fb12e08af1af156e2 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260923 22:14:52.667599 15766 raft_consensus.cc:729] T 3fc83020634044638e2efad4a6bef386 P 7f8b6e024a774ca996fd7858a93c42b2 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 7f8b6e024a774ca996fd7858a93c42b2, State: Initialized, Role: FOLLOWER
I20260923 22:14:52.667615 15768 raft_consensus.cc:729] T 3fc83020634044638e2efad4a6bef386 P 2ac26bd2c8eb4b4fb12e08af1af156e2 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 2ac26bd2c8eb4b4fb12e08af1af156e2, State: Initialized, Role: FOLLOWER
I20260923 22:14:52.667650 15767 consensus_queue.cc:260] T 3fc83020634044638e2efad4a6bef386 P 7d6af16a92f948c98fa0a8c4d04da648 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7d6af16a92f948c98fa0a8c4d04da648" member_type: VOTER last_known_addr { host: "127.1.135.194" port: 46495 } } peers { permanent_uuid: "2ac26bd2c8eb4b4fb12e08af1af156e2" member_type: VOTER last_known_addr { host: "127.1.135.193" port: 37137 } } peers { permanent_uuid: "7f8b6e024a774ca996fd7858a93c42b2" member_type: VOTER last_known_addr { host: "127.1.135.195" port: 34569 } }
I20260923 22:14:52.667721 15766 consensus_queue.cc:260] T 3fc83020634044638e2efad4a6bef386 P 7f8b6e024a774ca996fd7858a93c42b2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7d6af16a92f948c98fa0a8c4d04da648" member_type: VOTER last_known_addr { host: "127.1.135.194" port: 46495 } } peers { permanent_uuid: "2ac26bd2c8eb4b4fb12e08af1af156e2" member_type: VOTER last_known_addr { host: "127.1.135.193" port: 37137 } } peers { permanent_uuid: "7f8b6e024a774ca996fd7858a93c42b2" member_type: VOTER last_known_addr { host: "127.1.135.195" port: 34569 } }
I20260923 22:14:52.667711 15768 consensus_queue.cc:260] T 3fc83020634044638e2efad4a6bef386 P 2ac26bd2c8eb4b4fb12e08af1af156e2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7d6af16a92f948c98fa0a8c4d04da648" member_type: VOTER last_known_addr { host: "127.1.135.194" port: 46495 } } peers { permanent_uuid: "2ac26bd2c8eb4b4fb12e08af1af156e2" member_type: VOTER last_known_addr { host: "127.1.135.193" port: 37137 } } peers { permanent_uuid: "7f8b6e024a774ca996fd7858a93c42b2" member_type: VOTER last_known_addr { host: "127.1.135.195" port: 34569 } }
I20260923 22:14:52.667917 15767 ts_tablet_manager.cc:1428] T 3fc83020634044638e2efad4a6bef386 P 7d6af16a92f948c98fa0a8c4d04da648: Time spent starting tablet: real 0.002s user 0.002s sys 0.000s
I20260923 22:14:52.667919 15618 heartbeater.cc:503] Master 127.1.135.254:37515 was elected leader, sending a full tablet report...
I20260923 22:14:52.667960 15768 ts_tablet_manager.cc:1428] T 3fc83020634044638e2efad4a6bef386 P 2ac26bd2c8eb4b4fb12e08af1af156e2: Time spent starting tablet: real 0.002s user 0.000s sys 0.002s
I20260923 22:14:52.667960 15766 ts_tablet_manager.cc:1428] T 3fc83020634044638e2efad4a6bef386 P 7f8b6e024a774ca996fd7858a93c42b2: Time spent starting tablet: real 0.002s user 0.000s sys 0.002s
I20260923 22:14:52.668104 15752 heartbeater.cc:503] Master 127.1.135.254:37515 was elected leader, sending a full tablet report...
I20260923 22:14:52.668248 15484 heartbeater.cc:503] Master 127.1.135.254:37515 was elected leader, sending a full tablet report...
I20260923 22:14:52.707146 15772 raft_consensus.cc:482] T 3fc83020634044638e2efad4a6bef386 P 7d6af16a92f948c98fa0a8c4d04da648 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260923 22:14:52.707243 15772 raft_consensus.cc:504] T 3fc83020634044638e2efad4a6bef386 P 7d6af16a92f948c98fa0a8c4d04da648 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7d6af16a92f948c98fa0a8c4d04da648" member_type: VOTER last_known_addr { host: "127.1.135.194" port: 46495 } } peers { permanent_uuid: "2ac26bd2c8eb4b4fb12e08af1af156e2" member_type: VOTER last_known_addr { host: "127.1.135.193" port: 37137 } } peers { permanent_uuid: "7f8b6e024a774ca996fd7858a93c42b2" member_type: VOTER last_known_addr { host: "127.1.135.195" port: 34569 } }
I20260923 22:14:52.707630 15772 leader_election.cc:288] T 3fc83020634044638e2efad4a6bef386 P 7d6af16a92f948c98fa0a8c4d04da648 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 2ac26bd2c8eb4b4fb12e08af1af156e2 (127.1.135.193:37137), 7f8b6e024a774ca996fd7858a93c42b2 (127.1.135.195:34569)
I20260923 22:14:52.711148 15763 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 22:14:52.707834 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.135.194:43789 (local address 127.1.135.193:37137)
0923 22:14:52.707872 (+ 38us) server_negotiation.cc:206] Beginning negotiation
0923 22:14:52.707874 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0923 22:14:52.707883 (+ 9us) server_negotiation.cc:407] Connection header received
0923 22:14:52.707932 (+ 49us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 22:14:52.707933 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 22:14:52.707942 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 22:14:52.708001 (+ 59us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0923 22:14:52.708177 (+ 176us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 22:14:52.708923 (+ 746us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 22:14:52.710222 (+ 1299us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 22:14:52.710807 (+ 585us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 22:14:52.710885 (+ 78us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 22:14:52.710892 (+ 7us) server_negotiation.cc:1035] Waiting for connection context
0923 22:14:52.711048 (+ 156us) server_negotiation.cc:299] Negotiation successful
0923 22:14:52.711085 (+ 37us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":19}
I20260923 22:14:52.711148 15765 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 22:14:52.707847 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.135.194:47827 (local address 127.1.135.195:34569)
0923 22:14:52.707882 (+ 35us) server_negotiation.cc:206] Beginning negotiation
0923 22:14:52.707884 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0923 22:14:52.708108 (+ 224us) server_negotiation.cc:407] Connection header received
0923 22:14:52.708165 (+ 57us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 22:14:52.708166 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 22:14:52.708175 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 22:14:52.708240 (+ 65us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0923 22:14:52.708345 (+ 105us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 22:14:52.708983 (+ 638us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 22:14:52.710222 (+ 1239us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 22:14:52.710807 (+ 585us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 22:14:52.710884 (+ 77us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 22:14:52.710892 (+ 8us) server_negotiation.cc:1035] Waiting for connection context
0923 22:14:52.711048 (+ 156us) server_negotiation.cc:299] Negotiation successful
0923 22:14:52.711090 (+ 42us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":18}
I20260923 22:14:52.711171 15620 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 22:14:52.707782 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.135.193:37137 (local address 127.1.135.194:43789)
0923 22:14:52.707828 (+ 46us) negotiation.cc:107] Waiting for socket to connect
0923 22:14:52.707840 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0923 22:14:52.707879 (+ 39us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0923 22:14:52.708004 (+ 125us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0923 22:14:52.708005 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0923 22:14:52.708006 (+ 1us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0923 22:14:52.708100 (+ 94us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 22:14:52.708102 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 22:14:52.708993 (+ 891us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 22:14:52.708993 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 22:14:52.710137 (+ 1144us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 22:14:52.710139 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 22:14:52.710891 (+ 752us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 22:14:52.710892 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 22:14:52.710974 (+ 82us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 22:14:52.710976 (+ 2us) client_negotiation.cc:769] Sending connection context
0923 22:14:52.711031 (+ 55us) client_negotiation.cc:240] Negotiation successful
0923 22:14:52.711104 (+ 73us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":27,"mutex_wait_us":46}
I20260923 22:14:52.711180 15775 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 22:14:52.707789 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.135.195:34569 (local address 127.1.135.194:47827)
0923 22:14:52.708022 (+ 233us) negotiation.cc:107] Waiting for socket to connect
0923 22:14:52.708032 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0923 22:14:52.708110 (+ 78us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0923 22:14:52.708244 (+ 134us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0923 22:14:52.708245 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0923 22:14:52.708246 (+ 1us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0923 22:14:52.708288 (+ 42us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 22:14:52.708289 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 22:14:52.709054 (+ 765us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 22:14:52.709055 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 22:14:52.710136 (+ 1081us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 22:14:52.710139 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 22:14:52.710892 (+ 753us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 22:14:52.710892 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 22:14:52.710971 (+ 79us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 22:14:52.710975 (+ 4us) client_negotiation.cc:769] Sending connection context
0923 22:14:52.711031 (+ 56us) client_negotiation.cc:240] Negotiation successful
0923 22:14:52.711095 (+ 64us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":215,"mutex_wait_us":66,"thread_start_us":95,"threads_started":1}
I20260923 22:14:52.711511 15438 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "3fc83020634044638e2efad4a6bef386" candidate_uuid: "7d6af16a92f948c98fa0a8c4d04da648" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "2ac26bd2c8eb4b4fb12e08af1af156e2" is_pre_election: true
I20260923 22:14:52.711551 15706 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "3fc83020634044638e2efad4a6bef386" candidate_uuid: "7d6af16a92f948c98fa0a8c4d04da648" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7f8b6e024a774ca996fd7858a93c42b2" is_pre_election: true
I20260923 22:14:52.711655 15438 raft_consensus.cc:2454] T 3fc83020634044638e2efad4a6bef386 P 2ac26bd2c8eb4b4fb12e08af1af156e2 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7d6af16a92f948c98fa0a8c4d04da648 in term 0.
I20260923 22:14:52.711665 15706 raft_consensus.cc:2454] T 3fc83020634044638e2efad4a6bef386 P 7f8b6e024a774ca996fd7858a93c42b2 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7d6af16a92f948c98fa0a8c4d04da648 in term 0.
I20260923 22:14:52.711853 15506 leader_election.cc:302] T 3fc83020634044638e2efad4a6bef386 P 7d6af16a92f948c98fa0a8c4d04da648 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 7d6af16a92f948c98fa0a8c4d04da648, 7f8b6e024a774ca996fd7858a93c42b2; no voters:
I20260923 22:14:52.711973 15772 raft_consensus.cc:2781] T 3fc83020634044638e2efad4a6bef386 P 7d6af16a92f948c98fa0a8c4d04da648 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260923 22:14:52.712021 15772 raft_consensus.cc:482] T 3fc83020634044638e2efad4a6bef386 P 7d6af16a92f948c98fa0a8c4d04da648 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260923 22:14:52.712044 15772 raft_consensus.cc:3037] T 3fc83020634044638e2efad4a6bef386 P 7d6af16a92f948c98fa0a8c4d04da648 [term 0 FOLLOWER]: Advancing to term 1
I20260923 22:14:52.712664 15772 raft_consensus.cc:504] T 3fc83020634044638e2efad4a6bef386 P 7d6af16a92f948c98fa0a8c4d04da648 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7d6af16a92f948c98fa0a8c4d04da648" member_type: VOTER last_known_addr { host: "127.1.135.194" port: 46495 } } peers { permanent_uuid: "2ac26bd2c8eb4b4fb12e08af1af156e2" member_type: VOTER last_known_addr { host: "127.1.135.193" port: 37137 } } peers { permanent_uuid: "7f8b6e024a774ca996fd7858a93c42b2" member_type: VOTER last_known_addr { host: "127.1.135.195" port: 34569 } }
I20260923 22:14:52.712785 15772 leader_election.cc:288] T 3fc83020634044638e2efad4a6bef386 P 7d6af16a92f948c98fa0a8c4d04da648 [CANDIDATE]: Term 1 election: Requested vote from peers 2ac26bd2c8eb4b4fb12e08af1af156e2 (127.1.135.193:37137), 7f8b6e024a774ca996fd7858a93c42b2 (127.1.135.195:34569)
I20260923 22:14:52.712914 15438 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "3fc83020634044638e2efad4a6bef386" candidate_uuid: "7d6af16a92f948c98fa0a8c4d04da648" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "2ac26bd2c8eb4b4fb12e08af1af156e2"
I20260923 22:14:52.712929 15706 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "3fc83020634044638e2efad4a6bef386" candidate_uuid: "7d6af16a92f948c98fa0a8c4d04da648" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7f8b6e024a774ca996fd7858a93c42b2"
I20260923 22:14:52.712994 15438 raft_consensus.cc:3037] T 3fc83020634044638e2efad4a6bef386 P 2ac26bd2c8eb4b4fb12e08af1af156e2 [term 0 FOLLOWER]: Advancing to term 1
I20260923 22:14:52.713027 15706 raft_consensus.cc:3037] T 3fc83020634044638e2efad4a6bef386 P 7f8b6e024a774ca996fd7858a93c42b2 [term 0 FOLLOWER]: Advancing to term 1
I20260923 22:14:52.713663 15706 raft_consensus.cc:2454] T 3fc83020634044638e2efad4a6bef386 P 7f8b6e024a774ca996fd7858a93c42b2 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7d6af16a92f948c98fa0a8c4d04da648 in term 1.
I20260923 22:14:52.713685 15438 raft_consensus.cc:2454] T 3fc83020634044638e2efad4a6bef386 P 2ac26bd2c8eb4b4fb12e08af1af156e2 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7d6af16a92f948c98fa0a8c4d04da648 in term 1.
I20260923 22:14:52.713843 15506 leader_election.cc:302] T 3fc83020634044638e2efad4a6bef386 P 7d6af16a92f948c98fa0a8c4d04da648 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 7d6af16a92f948c98fa0a8c4d04da648, 7f8b6e024a774ca996fd7858a93c42b2; no voters:
I20260923 22:14:52.713932 15772 raft_consensus.cc:2781] T 3fc83020634044638e2efad4a6bef386 P 7d6af16a92f948c98fa0a8c4d04da648 [term 1 FOLLOWER]: Leader election won for term 1
I20260923 22:14:52.714042 15772 raft_consensus.cc:686] T 3fc83020634044638e2efad4a6bef386 P 7d6af16a92f948c98fa0a8c4d04da648 [term 1 LEADER]: Becoming Leader. State: Replica: 7d6af16a92f948c98fa0a8c4d04da648, State: Running, Role: LEADER
I20260923 22:14:52.714143 15772 consensus_queue.cc:237] T 3fc83020634044638e2efad4a6bef386 P 7d6af16a92f948c98fa0a8c4d04da648 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7d6af16a92f948c98fa0a8c4d04da648" member_type: VOTER last_known_addr { host: "127.1.135.194" port: 46495 } } peers { permanent_uuid: "2ac26bd2c8eb4b4fb12e08af1af156e2" member_type: VOTER last_known_addr { host: "127.1.135.193" port: 37137 } } peers { permanent_uuid: "7f8b6e024a774ca996fd7858a93c42b2" member_type: VOTER last_known_addr { host: "127.1.135.195" port: 34569 } }
W20260923 22:14:52.714323 15619 tablet.cc:2369] T 3fc83020634044638e2efad4a6bef386 P 7d6af16a92f948c98fa0a8c4d04da648: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260923 22:14:52.714900 15292 catalog_manager.cc:5495] T 3fc83020634044638e2efad4a6bef386 P 7d6af16a92f948c98fa0a8c4d04da648 reported cstate change: term changed from 0 to 1, leader changed from <none> to 7d6af16a92f948c98fa0a8c4d04da648 (127.1.135.194). New cstate: current_term: 1 leader_uuid: "7d6af16a92f948c98fa0a8c4d04da648" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7d6af16a92f948c98fa0a8c4d04da648" member_type: VOTER last_known_addr { host: "127.1.135.194" port: 46495 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "2ac26bd2c8eb4b4fb12e08af1af156e2" member_type: VOTER last_known_addr { host: "127.1.135.193" port: 37137 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "7f8b6e024a774ca996fd7858a93c42b2" member_type: VOTER last_known_addr { host: "127.1.135.195" port: 34569 } health_report { overall_health: UNKNOWN } } }
I20260923 22:14:52.733613 15764 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 22:14:52.731550 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48748 (local address 127.1.135.194:46495)
0923 22:14:52.731609 (+ 59us) server_negotiation.cc:206] Beginning negotiation
0923 22:14:52.731611 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0923 22:14:52.731618 (+ 7us) server_negotiation.cc:407] Connection header received
0923 22:14:52.731661 (+ 43us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 22:14:52.731662 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 22:14:52.731675 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 22:14:52.731728 (+ 53us) server_negotiation.cc:226] Negotiated authn=TOKEN
0923 22:14:52.731823 (+ 95us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 22:14:52.732315 (+ 492us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 22:14:52.732998 (+ 683us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 22:14:52.733161 (+ 163us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 22:14:52.733206 (+ 45us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 22:14:52.733319 (+ 113us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0923 22:14:52.733399 (+ 80us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0923 22:14:52.733451 (+ 52us) server_negotiation.cc:1035] Waiting for connection context
0923 22:14:52.733478 (+ 27us) server_negotiation.cc:299] Negotiation successful
0923 22:14:52.733492 (+ 14us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":42}
I20260923 22:14:52.735116 15438 raft_consensus.cc:1261] T 3fc83020634044638e2efad4a6bef386 P 2ac26bd2c8eb4b4fb12e08af1af156e2 [term 1 FOLLOWER]: Refusing update from remote peer 7d6af16a92f948c98fa0a8c4d04da648: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260923 22:14:52.735118 15706 raft_consensus.cc:1261] T 3fc83020634044638e2efad4a6bef386 P 7f8b6e024a774ca996fd7858a93c42b2 [term 1 FOLLOWER]: Refusing update from remote peer 7d6af16a92f948c98fa0a8c4d04da648: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260923 22:14:52.735445 15776 consensus_queue.cc:1035] T 3fc83020634044638e2efad4a6bef386 P 7d6af16a92f948c98fa0a8c4d04da648 [LEADER]: Connected to new peer: Peer: permanent_uuid: "7f8b6e024a774ca996fd7858a93c42b2" member_type: VOTER last_known_addr { host: "127.1.135.195" port: 34569 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260923 22:14:52.735558 15772 consensus_queue.cc:1035] T 3fc83020634044638e2efad4a6bef386 P 7d6af16a92f948c98fa0a8c4d04da648 [LEADER]: Connected to new peer: Peer: permanent_uuid: "2ac26bd2c8eb4b4fb12e08af1af156e2" member_type: VOTER last_known_addr { host: "127.1.135.193" port: 37137 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260923 22:14:52.737838 15779 mvcc.cc:205] Tried to move back new op lower bound from 7332666133440757760 to 7332666133357813760. Current Snapshot: MvccSnapshot[applied={T|T < 7332666133440757760}]
I20260923 22:14:52.739773 15292 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:49298:
table { table_name: "test-table" } modify_external_catalogs: true
I20260923 22:14:52.739862 15292 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:49298:
table { table_name: "test-table" } modify_external_catalogs: true
I20260923 22:14:52.740628 15292 catalog_manager.cc:5782] T 00000000000000000000000000000000 P d32493cb30cb4eda8ee9e9f9a9c35ce2: Sending DeleteTablet for 3 replicas of tablet 3fc83020634044638e2efad4a6bef386
I20260923 22:14:52.740939 15552 tablet_service.cc:1514] Processing DeleteTablet for tablet 3fc83020634044638e2efad4a6bef386 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-23 22:14:52 UTC) from {username='kudu', principal='kudu/127.1.135.254@KRBTEST.COM'} at 127.0.0.1:48736
I20260923 22:14:52.740924 15418 tablet_service.cc:1514] Processing DeleteTablet for tablet 3fc83020634044638e2efad4a6bef386 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-23 22:14:52 UTC) from {username='kudu', principal='kudu/127.1.135.254@KRBTEST.COM'} at 127.0.0.1:55652
I20260923 22:14:52.740990 15686 tablet_service.cc:1514] Processing DeleteTablet for tablet 3fc83020634044638e2efad4a6bef386 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-23 22:14:52 UTC) from {username='kudu', principal='kudu/127.1.135.254@KRBTEST.COM'} at 127.0.0.1:42540
I20260923 22:14:52.741138 1567 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskRiQe_a/build/release/bin/kudu with pid 15355
I20260923 22:14:52.741149 15789 tablet_replica.cc:330] T 3fc83020634044638e2efad4a6bef386 P 2ac26bd2c8eb4b4fb12e08af1af156e2: stopping tablet replica
I20260923 22:14:52.741745 15791 tablet_replica.cc:330] T 3fc83020634044638e2efad4a6bef386 P 7f8b6e024a774ca996fd7858a93c42b2: stopping tablet replica
I20260923 22:14:52.741760 15790 tablet_replica.cc:330] T 3fc83020634044638e2efad4a6bef386 P 7d6af16a92f948c98fa0a8c4d04da648: stopping tablet replica
I20260923 22:14:52.741840 15791 raft_consensus.cc:2229] T 3fc83020634044638e2efad4a6bef386 P 7f8b6e024a774ca996fd7858a93c42b2 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260923 22:14:52.741840 15790 raft_consensus.cc:2229] T 3fc83020634044638e2efad4a6bef386 P 7d6af16a92f948c98fa0a8c4d04da648 [term 1 LEADER]: Raft consensus shutting down.
I20260923 22:14:52.741945 15791 raft_consensus.cc:2258] T 3fc83020634044638e2efad4a6bef386 P 7f8b6e024a774ca996fd7858a93c42b2 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260923 22:14:52.741990 15790 raft_consensus.cc:2258] T 3fc83020634044638e2efad4a6bef386 P 7d6af16a92f948c98fa0a8c4d04da648 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260923 22:14:52.742405 15790 ts_tablet_manager.cc:1905] T 3fc83020634044638e2efad4a6bef386 P 7d6af16a92f948c98fa0a8c4d04da648: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260923 22:14:52.742452 15791 ts_tablet_manager.cc:1905] T 3fc83020634044638e2efad4a6bef386 P 7f8b6e024a774ca996fd7858a93c42b2: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260923 22:14:52.744308 15790 ts_tablet_manager.cc:1918] T 3fc83020634044638e2efad4a6bef386 P 7d6af16a92f948c98fa0a8c4d04da648: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260923 22:14:52.744328 15791 ts_tablet_manager.cc:1918] T 3fc83020634044638e2efad4a6bef386 P 7f8b6e024a774ca996fd7858a93c42b2: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260923 22:14:52.744388 15791 log.cc:1196] T 3fc83020634044638e2efad4a6bef386 P 7f8b6e024a774ca996fd7858a93c42b2: Deleting WAL directory at /tmp/dist-test-taskRiQe_a/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790201539513534-1567-0/minicluster-data/ts-2/wal/wals/3fc83020634044638e2efad4a6bef386
I20260923 22:14:52.744470 15790 log.cc:1196] T 3fc83020634044638e2efad4a6bef386 P 7d6af16a92f948c98fa0a8c4d04da648: Deleting WAL directory at /tmp/dist-test-taskRiQe_a/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790201539513534-1567-0/minicluster-data/ts-1/wal/wals/3fc83020634044638e2efad4a6bef386
I20260923 22:14:52.744685 15791 ts_tablet_manager.cc:1939] T 3fc83020634044638e2efad4a6bef386 P 7f8b6e024a774ca996fd7858a93c42b2: Deleting consensus metadata
I20260923 22:14:52.744793 15790 ts_tablet_manager.cc:1939] T 3fc83020634044638e2efad4a6bef386 P 7d6af16a92f948c98fa0a8c4d04da648: Deleting consensus metadata
I20260923 22:14:52.745065 15278 catalog_manager.cc:4841] TS 7f8b6e024a774ca996fd7858a93c42b2 (127.1.135.195:34569): tablet 3fc83020634044638e2efad4a6bef386 (table test-table [id=0098df898fff4bf69cf3519e4d5d48ab]) successfully deleted
I20260923 22:14:52.745193 15278 catalog_manager.cc:4841] TS 7d6af16a92f948c98fa0a8c4d04da648 (127.1.135.194:46495): tablet 3fc83020634044638e2efad4a6bef386 (table test-table [id=0098df898fff4bf69cf3519e4d5d48ab]) successfully deleted
W20260923 22:14:52.747258 15279 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.1.135.193:37137 (error 108)
I20260923 22:14:52.747687 15761 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 22:14:52.747551 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.135.193:37137 (local address 127.0.0.1:55666)
0923 22:14:52.747620 (+ 69us) negotiation.cc:107] Waiting for socket to connect
0923 22:14:52.747653 (+ 33us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.135.193:37137: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":53}
I20260923 22:14:52.747768 1567 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskRiQe_a/build/release/bin/kudu with pid 15489
W20260923 22:14:52.747816 15279 catalog_manager.cc:4584] TS 2ac26bd2c8eb4b4fb12e08af1af156e2 (127.1.135.193:37137): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet 3fc83020634044638e2efad4a6bef386: Network error: Client connection negotiation failed: client connection to 127.1.135.193:37137: connect: Connection refused (error 111)
I20260923 22:14:52.753456 1567 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskRiQe_a/build/release/bin/kudu with pid 15623
I20260923 22:14:52.758419 1567 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskRiQe_a/build/release/bin/kudu with pid 15261
I20260923 22:14:52.763250 1567 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskRiQe_a/build/release/bin/kudu
/tmp/dist-test-taskRiQe_a/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskRiQe_a/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790201539513534-1567-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskRiQe_a/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790201539513534-1567-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskRiQe_a/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790201539513534-1567-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskRiQe_a/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790201539513534-1567-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.1.135.254:37515
--webserver_interface=127.1.135.254
--webserver_port=45941
--keytab_file=/tmp/dist-test-taskRiQe_a/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790201539513534-1567-0/krb5kdc/kudu_127.1.135.254.keytab
--principal=kudu/127.1.135.254
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.1.135.212:38425
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.1.135.254:37515
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--ipki_private_key_password_cmd=echo foo with env {KRB5CCNAME=/tmp/dist-test-taskRiQe_a/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790201539513534-1567-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskRiQe_a/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790201539513534-1567-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskRiQe_a/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790201539513534-1567-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260923 22:14:52.840767 15793 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260923 22:14:52.840940 15793 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260923 22:14:52.840969 15793 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260923 22:14:52.842052 15793 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260923 22:14:52.842087 15793 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260923 22:14:52.842099 15793 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260923 22:14:52.842137 15793 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260923 22:14:52.842160 15793 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260923 22:14:52.843621 15793 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.1.135.212:38425
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskRiQe_a/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790201539513534-1567-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskRiQe_a/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790201539513534-1567-0/minicluster-data/master-0/wal
--ipki_private_key_password_cmd=echo foo
--ipki_ca_key_size=768
--master_addresses=127.1.135.254:37515
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskRiQe_a/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790201539513534-1567-0/krb5kdc/kudu_127.1.135.254.keytab
--principal=kudu/127.1.135.254
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.1.135.254:37515
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskRiQe_a/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790201539513534-1567-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.1.135.254
--webserver_port=45941
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.15793
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskRiQe_a/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790201539513534-1567-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 23 Sep 2026 20:53:37 UTC on ccycloud.quasar-rkviyx.root.comops.site
build id 4668083
I20260923 22:14:52.843855 15793 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260923 22:14:52.844087 15793 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260923 22:14:52.851940 15799 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260923 22:14:52.851997 15793 server_base.cc:1034] running on GCE node
W20260923 22:14:52.851944 15802 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260923 22:14:52.851958 15800 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260923 22:14:52.852376 15793 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260923 22:14:52.852674 15793 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260923 22:14:52.853830 15793 hybrid_clock.cc:648] HybridClock initialized: now 1790201692853827 us; error 33 us; skew 500 ppm
Sep 23 22:14:52 dist-test-slave-s2fj krb5kdc[15248](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790201692, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.135.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260923 22:14:52.856238 15793 init.cc:377] Logged in from keytab as kudu/127.1.135.254@KRBTEST.COM (short username kudu)
I20260923 22:14:52.856637 15793 webserver.cc:460] Webserver started at http://127.1.135.254:45941/ using document root <none> and password file <none>
I20260923 22:14:52.856863 15793 fs_manager.cc:359] Metadata directory not provided
I20260923 22:14:52.856932 15793 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260923 22:14:52.858170 15793 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.001s sys 0.000s
I20260923 22:14:52.858822 15809 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260923 22:14:52.858986 15793 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260923 22:14:52.859094 15793 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskRiQe_a/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790201539513534-1567-0/minicluster-data/master-0/data,/tmp/dist-test-taskRiQe_a/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790201539513534-1567-0/minicluster-data/master-0/wal
uuid: "d32493cb30cb4eda8ee9e9f9a9c35ce2"
format_stamp: "Formatted at 2026-09-23 22:14:52 on dist-test-slave-s2fj"
server_key: "8358f9839aaaf6002d5bb63dbf35b23e"
server_key_iv: "887f4f7c2f04783e830210e387213916"
server_key_version: "encryptionkey@0"
I20260923 22:14:52.859439 15793 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskRiQe_a/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790201539513534-1567-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskRiQe_a/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790201539513534-1567-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskRiQe_a/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790201539513534-1567-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260923 22:14:52.883284 15793 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260923 22:14:52.885001 15793 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260923 22:14:52.885166 15793 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260923 22:14:52.888851 15793 rpc_server.cc:307] RPC server started. Bound to: 127.1.135.254:37515
I20260923 22:14:52.888883 15861 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.1.135.254:37515 every 8 connection(s)
I20260923 22:14:52.889266 15793 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskRiQe_a/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790201539513534-1567-0/minicluster-data/master-0/data/info.pb
I20260923 22:14:52.890726 15862 sys_catalog.cc:263] Verifying existing consensus state
I20260923 22:14:52.891333 15862 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P d32493cb30cb4eda8ee9e9f9a9c35ce2: Bootstrap starting.
I20260923 22:14:52.897840 1567 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskRiQe_a/build/release/bin/kudu as pid 15793
I20260923 22:14:52.898168 1567 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskRiQe_a/build/release/bin/kudu
/tmp/dist-test-taskRiQe_a/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskRiQe_a/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790201539513534-1567-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskRiQe_a/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790201539513534-1567-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskRiQe_a/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790201539513534-1567-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskRiQe_a/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790201539513534-1567-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.1.135.193:37137
--local_ip_for_outbound_sockets=127.1.135.193
--tserver_master_addrs=127.1.135.254:37515
--webserver_port=34527
--webserver_interface=127.1.135.193
--keytab_file=/tmp/dist-test-taskRiQe_a/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790201539513534-1567-0/krb5kdc/kudu_127.1.135.193.keytab
--principal=kudu/127.1.135.193
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.1.135.212:38425
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskRiQe_a/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790201539513534-1567-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskRiQe_a/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790201539513534-1567-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskRiQe_a/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790201539513534-1567-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260923 22:14:52.900151 15862 log.cc:824] T 00000000000000000000000000000000 P d32493cb30cb4eda8ee9e9f9a9c35ce2: Log is configured to *not* fsync() on all Append() calls
I20260923 22:14:52.901927 15862 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P d32493cb30cb4eda8ee9e9f9a9c35ce2: Bootstrap replayed 1/1 log segments. Stats: ops{read=8 overwritten=0 applied=8 ignored=0} inserts{seen=5 ignored=0} mutations{seen=4 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260923 22:14:52.902204 15862 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P d32493cb30cb4eda8ee9e9f9a9c35ce2: Bootstrap complete.
I20260923 22:14:52.904469 15862 raft_consensus.cc:348] T 00000000000000000000000000000000 P d32493cb30cb4eda8ee9e9f9a9c35ce2 [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d32493cb30cb4eda8ee9e9f9a9c35ce2" member_type: VOTER last_known_addr { host: "127.1.135.254" port: 37515 } }
I20260923 22:14:52.904690 15862 raft_consensus.cc:729] T 00000000000000000000000000000000 P d32493cb30cb4eda8ee9e9f9a9c35ce2 [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: d32493cb30cb4eda8ee9e9f9a9c35ce2, State: Initialized, Role: FOLLOWER
I20260923 22:14:52.904835 15862 consensus_queue.cc:260] T 00000000000000000000000000000000 P d32493cb30cb4eda8ee9e9f9a9c35ce2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 8, Last appended: 1.8, Last appended by leader: 8, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d32493cb30cb4eda8ee9e9f9a9c35ce2" member_type: VOTER last_known_addr { host: "127.1.135.254" port: 37515 } }
I20260923 22:14:52.904887 15862 raft_consensus.cc:388] T 00000000000000000000000000000000 P d32493cb30cb4eda8ee9e9f9a9c35ce2 [term 1 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260923 22:14:52.904942 15862 raft_consensus.cc:482] T 00000000000000000000000000000000 P d32493cb30cb4eda8ee9e9f9a9c35ce2 [term 1 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260923 22:14:52.904983 15862 raft_consensus.cc:3037] T 00000000000000000000000000000000 P d32493cb30cb4eda8ee9e9f9a9c35ce2 [term 1 FOLLOWER]: Advancing to term 2
I20260923 22:14:52.905906 15862 raft_consensus.cc:504] T 00000000000000000000000000000000 P d32493cb30cb4eda8ee9e9f9a9c35ce2 [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d32493cb30cb4eda8ee9e9f9a9c35ce2" member_type: VOTER last_known_addr { host: "127.1.135.254" port: 37515 } }
I20260923 22:14:52.905983 15862 leader_election.cc:302] T 00000000000000000000000000000000 P d32493cb30cb4eda8ee9e9f9a9c35ce2 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: d32493cb30cb4eda8ee9e9f9a9c35ce2; no voters:
I20260923 22:14:52.906162 15862 leader_election.cc:288] T 00000000000000000000000000000000 P d32493cb30cb4eda8ee9e9f9a9c35ce2 [CANDIDATE]: Term 2 election: Requested vote from peers
I20260923 22:14:52.906195 15866 raft_consensus.cc:2781] T 00000000000000000000000000000000 P d32493cb30cb4eda8ee9e9f9a9c35ce2 [term 2 FOLLOWER]: Leader election won for term 2
I20260923 22:14:52.906306 15866 raft_consensus.cc:686] T 00000000000000000000000000000000 P d32493cb30cb4eda8ee9e9f9a9c35ce2 [term 2 LEADER]: Becoming Leader. State: Replica: d32493cb30cb4eda8ee9e9f9a9c35ce2, State: Running, Role: LEADER
I20260923 22:14:52.906404 15862 sys_catalog.cc:564] T 00000000000000000000000000000000 P d32493cb30cb4eda8ee9e9f9a9c35ce2 [sys.catalog]: configured and running, proceeding with master startup.
I20260923 22:14:52.906384 15866 consensus_queue.cc:237] T 00000000000000000000000000000000 P d32493cb30cb4eda8ee9e9f9a9c35ce2 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 1.8, Last appended by leader: 8, Current term: 2, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d32493cb30cb4eda8ee9e9f9a9c35ce2" member_type: VOTER last_known_addr { host: "127.1.135.254" port: 37515 } }
I20260923 22:14:52.906723 15867 sys_catalog.cc:455] T 00000000000000000000000000000000 P d32493cb30cb4eda8ee9e9f9a9c35ce2 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 2 leader_uuid: "d32493cb30cb4eda8ee9e9f9a9c35ce2" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d32493cb30cb4eda8ee9e9f9a9c35ce2" member_type: VOTER last_known_addr { host: "127.1.135.254" port: 37515 } } }
I20260923 22:14:52.906802 15867 sys_catalog.cc:458] T 00000000000000000000000000000000 P d32493cb30cb4eda8ee9e9f9a9c35ce2 [sys.catalog]: This master's current role is: LEADER
I20260923 22:14:52.907035 15868 sys_catalog.cc:455] T 00000000000000000000000000000000 P d32493cb30cb4eda8ee9e9f9a9c35ce2 [sys.catalog]: SysCatalogTable state changed. Reason: New leader d32493cb30cb4eda8ee9e9f9a9c35ce2. Latest consensus state: current_term: 2 leader_uuid: "d32493cb30cb4eda8ee9e9f9a9c35ce2" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d32493cb30cb4eda8ee9e9f9a9c35ce2" member_type: VOTER last_known_addr { host: "127.1.135.254" port: 37515 } } }
I20260923 22:14:52.907124 15868 sys_catalog.cc:458] T 00000000000000000000000000000000 P d32493cb30cb4eda8ee9e9f9a9c35ce2 [sys.catalog]: This master's current role is: LEADER
I20260923 22:14:52.907308 15874 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260923 22:14:52.908069 15874 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260923 22:14:52.908195 15874 catalog_manager.cc:1259] Loaded cluster ID: 6fbcafe1f67a46a2a305ad3d03ea6e60
I20260923 22:14:52.908267 15874 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260923 22:14:52.908667 15874 catalog_manager.cc:1314] IPKI private key was stored in cleartext, attempting to encrypt it.
I20260923 22:14:52.911414 15874 catalog_manager.cc:1324] IPKI private key successfully encrypted.
I20260923 22:14:52.916963 15874 catalog_manager.cc:1504] Loading token signing keys...
I20260923 22:14:52.917166 15874 catalog_manager.cc:5879] T 00000000000000000000000000000000 P d32493cb30cb4eda8ee9e9f9a9c35ce2: Loaded TSK: 0
I20260923 22:14:52.917400 15874 catalog_manager.cc:1514] Initializing in-progress tserver states...
W20260923 22:14:52.989204 15864 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260923 22:14:52.989373 15864 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260923 22:14:52.989394 15864 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260923 22:14:52.990506 15864 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260923 22:14:52.990548 15864 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260923 22:14:52.990574 15864 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.1.135.193
I20260923 22:14:52.991956 15864 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.1.135.212:38425
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskRiQe_a/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790201539513534-1567-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskRiQe_a/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790201539513534-1567-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskRiQe_a/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790201539513534-1567-0/krb5kdc/kudu_127.1.135.193.keytab
--principal=kudu/127.1.135.193
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.1.135.193:37137
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskRiQe_a/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790201539513534-1567-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.1.135.193
--webserver_port=34527
--tserver_master_addrs=127.1.135.254:37515
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.15864
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.1.135.193
--log_dir=/tmp/dist-test-taskRiQe_a/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790201539513534-1567-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 23 Sep 2026 20:53:37 UTC on ccycloud.quasar-rkviyx.root.comops.site
build id 4668083
I20260923 22:14:52.992197 15864 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260923 22:14:52.992443 15864 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260923 22:14:52.997233 15890 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260923 22:14:52.997242 15889 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260923 22:14:52.997299 15892 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260923 22:14:52.997671 15864 server_base.cc:1034] running on GCE node
I20260923 22:14:52.997876 15864 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260923 22:14:52.998109 15864 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260923 22:14:52.999253 15864 hybrid_clock.cc:648] HybridClock initialized: now 1790201692999242 us; error 31 us; skew 500 ppm
Sep 23 22:14:52 dist-test-slave-s2fj krb5kdc[15248](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790201692, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.135.193@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260923 22:14:53.001864 15864 init.cc:377] Logged in from keytab as kudu/127.1.135.193@KRBTEST.COM (short username kudu)
I20260923 22:14:53.002329 15864 webserver.cc:460] Webserver started at http://127.1.135.193:34527/ using document root <none> and password file <none>
I20260923 22:14:53.002549 15864 fs_manager.cc:359] Metadata directory not provided
I20260923 22:14:53.002622 15864 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260923 22:14:53.004357 15864 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.001s sys 0.000s
I20260923 22:14:53.005074 15899 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260923 22:14:53.005255 15864 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260923 22:14:53.005336 15864 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskRiQe_a/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790201539513534-1567-0/minicluster-data/ts-0/data,/tmp/dist-test-taskRiQe_a/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790201539513534-1567-0/minicluster-data/ts-0/wal
uuid: "2ac26bd2c8eb4b4fb12e08af1af156e2"
format_stamp: "Formatted at 2026-09-23 22:14:52 on dist-test-slave-s2fj"
server_key: "7ce9468d20e1b5fc5cba598c1a59b91d"
server_key_iv: "af2c59483b907adc33e1ce686719613d"
server_key_version: "encryptionkey@0"
I20260923 22:14:53.005712 15864 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskRiQe_a/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790201539513534-1567-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskRiQe_a/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790201539513534-1567-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskRiQe_a/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790201539513534-1567-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260923 22:14:53.023028 15864 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260923 22:14:53.025345 15864 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260923 22:14:53.025473 15864 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260923 22:14:53.025681 15864 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260923 22:14:53.026144 15906 ts_tablet_manager.cc:536] Loading tablet metadata (0/1 complete)
I20260923 22:14:53.027463 15864 ts_tablet_manager.cc:579] Loaded tablet metadata (1 total tablets, 1 live tablets)
I20260923 22:14:53.027518 15864 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.001s user 0.000s sys 0.000s
I20260923 22:14:53.027547 15864 ts_tablet_manager.cc:594] Registering tablets (0/1 complete)
I20260923 22:14:53.028369 15864 ts_tablet_manager.cc:610] Registered 1 tablets
I20260923 22:14:53.028415 15864 ts_tablet_manager.cc:589] Time spent register tablets: real 0.001s user 0.000s sys 0.001s
I20260923 22:14:53.028479 15906 tablet_bootstrap.cc:492] T 3fc83020634044638e2efad4a6bef386 P 2ac26bd2c8eb4b4fb12e08af1af156e2: Bootstrap starting.
I20260923 22:14:53.035266 15864 rpc_server.cc:307] RPC server started. Bound to: 127.1.135.193:37137
I20260923 22:14:53.035359 16013 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.1.135.193:37137 every 8 connection(s)
I20260923 22:14:53.035708 15864 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskRiQe_a/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790201539513534-1567-0/minicluster-data/ts-0/data/info.pb
I20260923 22:14:53.039062 15906 log.cc:824] T 3fc83020634044638e2efad4a6bef386 P 2ac26bd2c8eb4b4fb12e08af1af156e2: Log is configured to *not* fsync() on all Append() calls
I20260923 22:14:53.041320 15906 tablet_bootstrap.cc:492] T 3fc83020634044638e2efad4a6bef386 P 2ac26bd2c8eb4b4fb12e08af1af156e2: Bootstrap replayed 1/1 log segments. Stats: ops{read=2 overwritten=0 applied=2 ignored=0} inserts{seen=1 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260923 22:14:53.041620 15906 tablet_bootstrap.cc:492] T 3fc83020634044638e2efad4a6bef386 P 2ac26bd2c8eb4b4fb12e08af1af156e2: Bootstrap complete.
I20260923 22:14:53.042609 15906 ts_tablet_manager.cc:1397] T 3fc83020634044638e2efad4a6bef386 P 2ac26bd2c8eb4b4fb12e08af1af156e2: Time spent bootstrapping tablet: real 0.014s user 0.005s sys 0.006s
I20260923 22:14:53.043421 1567 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskRiQe_a/build/release/bin/kudu as pid 15864
I20260923 22:14:53.043715 1567 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskRiQe_a/build/release/bin/kudu
/tmp/dist-test-taskRiQe_a/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskRiQe_a/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790201539513534-1567-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskRiQe_a/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790201539513534-1567-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskRiQe_a/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790201539513534-1567-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskRiQe_a/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790201539513534-1567-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.1.135.194:46495
--local_ip_for_outbound_sockets=127.1.135.194
--tserver_master_addrs=127.1.135.254:37515
--webserver_port=36489
--webserver_interface=127.1.135.194
--keytab_file=/tmp/dist-test-taskRiQe_a/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790201539513534-1567-0/krb5kdc/kudu_127.1.135.194.keytab
--principal=kudu/127.1.135.194
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.1.135.212:38425
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskRiQe_a/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790201539513534-1567-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskRiQe_a/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790201539513534-1567-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskRiQe_a/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790201539513534-1567-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260923 22:14:53.044010 15906 raft_consensus.cc:348] T 3fc83020634044638e2efad4a6bef386 P 2ac26bd2c8eb4b4fb12e08af1af156e2 [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7d6af16a92f948c98fa0a8c4d04da648" member_type: VOTER last_known_addr { host: "127.1.135.194" port: 46495 } } peers { permanent_uuid: "2ac26bd2c8eb4b4fb12e08af1af156e2" member_type: VOTER last_known_addr { host: "127.1.135.193" port: 37137 } } peers { permanent_uuid: "7f8b6e024a774ca996fd7858a93c42b2" member_type: VOTER last_known_addr { host: "127.1.135.195" port: 34569 } }
I20260923 22:14:53.044314 15906 raft_consensus.cc:729] T 3fc83020634044638e2efad4a6bef386 P 2ac26bd2c8eb4b4fb12e08af1af156e2 [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: 2ac26bd2c8eb4b4fb12e08af1af156e2, State: Initialized, Role: FOLLOWER
I20260923 22:14:53.044436 15906 consensus_queue.cc:260] T 3fc83020634044638e2efad4a6bef386 P 2ac26bd2c8eb4b4fb12e08af1af156e2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 2, Last appended: 1.2, Last appended by leader: 2, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7d6af16a92f948c98fa0a8c4d04da648" member_type: VOTER last_known_addr { host: "127.1.135.194" port: 46495 } } peers { permanent_uuid: "2ac26bd2c8eb4b4fb12e08af1af156e2" member_type: VOTER last_known_addr { host: "127.1.135.193" port: 37137 } } peers { permanent_uuid: "7f8b6e024a774ca996fd7858a93c42b2" member_type: VOTER last_known_addr { host: "127.1.135.195" port: 34569 } }
I20260923 22:14:53.044680 15906 ts_tablet_manager.cc:1428] T 3fc83020634044638e2efad4a6bef386 P 2ac26bd2c8eb4b4fb12e08af1af156e2: Time spent starting tablet: real 0.002s user 0.002s sys 0.000s
Sep 23 22:14:53 dist-test-slave-s2fj krb5kdc[15248](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790201692, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.135.193@KRBTEST.COM for kudu/127.1.135.254@KRBTEST.COM
I20260923 22:14:53.050041 16017 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 22:14:53.036550 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.135.193:50299 (local address 127.1.135.254:37515)
0923 22:14:53.036730 (+ 180us) server_negotiation.cc:206] Beginning negotiation
0923 22:14:53.036738 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0923 22:14:53.036933 (+ 195us) server_negotiation.cc:407] Connection header received
0923 22:14:53.037756 (+ 823us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 22:14:53.037762 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 22:14:53.037812 (+ 50us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 22:14:53.037943 (+ 131us) server_negotiation.cc:226] Negotiated authn=SASL
0923 22:14:53.038901 (+ 958us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 22:14:53.040301 (+ 1400us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 22:14:53.041740 (+ 1439us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 22:14:53.042346 (+ 606us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 22:14:53.042454 (+ 108us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 22:14:53.046002 (+ 3548us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0923 22:14:53.046008 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0923 22:14:53.046008 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0923 22:14:53.046024 (+ 16us) server_negotiation.cc:922] calling sasl_server_start()
0923 22:14:53.048197 (+ 2173us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0923 22:14:53.048647 (+ 450us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0923 22:14:53.048650 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0923 22:14:53.048651 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0923 22:14:53.048752 (+ 101us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0923 22:14:53.049063 (+ 311us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0923 22:14:53.049064 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0923 22:14:53.049065 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0923 22:14:53.049322 (+ 257us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0923 22:14:53.049399 (+ 77us) server_negotiation.cc:1035] Waiting for connection context
0923 22:14:53.049742 (+ 343us) server_negotiation.cc:299] Negotiation successful
0923 22:14:53.049875 (+ 133us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":126,"thread_start_us":89,"threads_started":1}
I20260923 22:14:53.050101 16016 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 22:14:53.036429 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.135.254:37515 (local address 127.1.135.193:50299)
0923 22:14:53.036758 (+ 329us) negotiation.cc:107] Waiting for socket to connect
0923 22:14:53.036808 (+ 50us) client_negotiation.cc:174] Beginning negotiation
0923 22:14:53.037502 (+ 694us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0923 22:14:53.038018 (+ 516us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0923 22:14:53.038024 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0923 22:14:53.038498 (+ 474us) client_negotiation.cc:189] Negotiated authn=SASL
0923 22:14:53.038777 (+ 279us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 22:14:53.038784 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 22:14:53.040425 (+ 1641us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 22:14:53.040427 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 22:14:53.041670 (+ 1243us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 22:14:53.041673 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 22:14:53.042447 (+ 774us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 22:14:53.042448 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 22:14:53.042595 (+ 147us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 22:14:53.043201 (+ 606us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0923 22:14:53.043211 (+ 10us) client_negotiation.cc:656] calling sasl_client_start()
0923 22:14:53.045887 (+ 2676us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0923 22:14:53.048275 (+ 2388us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0923 22:14:53.048281 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0923 22:14:53.048282 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0923 22:14:53.048538 (+ 256us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0923 22:14:53.048830 (+ 292us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0923 22:14:53.048831 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0923 22:14:53.048832 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0923 22:14:53.048974 (+ 142us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0923 22:14:53.049397 (+ 423us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0923 22:14:53.049404 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0923 22:14:53.049567 (+ 163us) client_negotiation.cc:769] Sending connection context
0923 22:14:53.049687 (+ 120us) client_negotiation.cc:240] Negotiation successful
0923 22:14:53.049869 (+ 182us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":277,"thread_start_us":86,"threads_started":1}
I20260923 22:14:53.051066 16014 heartbeater.cc:347] Connected to a master server at 127.1.135.254:37515
I20260923 22:14:53.051165 16014 heartbeater.cc:464] Registering TS with master...
I20260923 22:14:53.051358 16014 heartbeater.cc:511] Master 127.1.135.254:37515 requested a full tablet report, sending...
I20260923 22:14:53.052129 15826 ts_manager.cc:194] Registered new tserver with Master: 2ac26bd2c8eb4b4fb12e08af1af156e2 (127.1.135.193:37137)
I20260923 22:14:53.053118 15826 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.1.135.193@KRBTEST.COM'} at 127.1.135.193:50299
I20260923 22:14:53.054059 16014 heartbeater.cc:503] Master 127.1.135.254:37515 was elected leader, sending a full tablet report...
Sep 23 22:14:53 dist-test-slave-s2fj krb5kdc[15248](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790201692, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.135.254@KRBTEST.COM for kudu/127.1.135.193@KRBTEST.COM
I20260923 22:14:53.061744 16021 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 22:14:53.052654 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.135.193:37137 (local address 127.0.0.1:55680)
0923 22:14:53.053011 (+ 357us) negotiation.cc:107] Waiting for socket to connect
0923 22:14:53.053059 (+ 48us) client_negotiation.cc:174] Beginning negotiation
0923 22:14:53.053120 (+ 61us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0923 22:14:53.053440 (+ 320us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0923 22:14:53.053445 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0923 22:14:53.054564 (+ 1119us) client_negotiation.cc:189] Negotiated authn=SASL
0923 22:14:53.054623 (+ 59us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 22:14:53.054625 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 22:14:53.055846 (+ 1221us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 22:14:53.055847 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 22:14:53.056801 (+ 954us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 22:14:53.056804 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 22:14:53.057243 (+ 439us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 22:14:53.057244 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 22:14:53.057315 (+ 71us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 22:14:53.057744 (+ 429us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0923 22:14:53.057748 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0923 22:14:53.059024 (+ 1276us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0923 22:14:53.060612 (+ 1588us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0923 22:14:53.060619 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0923 22:14:53.060620 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0923 22:14:53.060807 (+ 187us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0923 22:14:53.061026 (+ 219us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0923 22:14:53.061027 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0923 22:14:53.061027 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0923 22:14:53.061089 (+ 62us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0923 22:14:53.061354 (+ 265us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0923 22:14:53.061360 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0923 22:14:53.061407 (+ 47us) client_negotiation.cc:769] Sending connection context
0923 22:14:53.061511 (+ 104us) client_negotiation.cc:240] Negotiation successful
0923 22:14:53.061645 (+ 134us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":330,"thread_start_us":65,"threads_started":1}
I20260923 22:14:53.061744 16022 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 22:14:53.052868 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55680 (local address 127.1.135.193:37137)
0923 22:14:53.053246 (+ 378us) server_negotiation.cc:206] Beginning negotiation
0923 22:14:53.053250 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0923 22:14:53.053275 (+ 25us) server_negotiation.cc:407] Connection header received
0923 22:14:53.053302 (+ 27us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 22:14:53.053304 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 22:14:53.053320 (+ 16us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 22:14:53.053381 (+ 61us) server_negotiation.cc:226] Negotiated authn=SASL
0923 22:14:53.054720 (+ 1339us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 22:14:53.055762 (+ 1042us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 22:14:53.056907 (+ 1145us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 22:14:53.057170 (+ 263us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 22:14:53.057227 (+ 57us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 22:14:53.059112 (+ 1885us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0923 22:14:53.059118 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0923 22:14:53.059119 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0923 22:14:53.059127 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0923 22:14:53.060527 (+ 1400us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0923 22:14:53.060889 (+ 362us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0923 22:14:53.060892 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0923 22:14:53.060892 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0923 22:14:53.060949 (+ 57us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0923 22:14:53.061160 (+ 211us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0923 22:14:53.061161 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0923 22:14:53.061161 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0923 22:14:53.061284 (+ 123us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0923 22:14:53.061373 (+ 89us) server_negotiation.cc:1035] Waiting for connection context
0923 22:14:53.061566 (+ 193us) server_negotiation.cc:299] Negotiation successful
0923 22:14:53.061647 (+ 81us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":351,"thread_start_us":64,"threads_started":1}
I20260923 22:14:53.062222 15947 tablet_service.cc:1514] Processing DeleteTablet for tablet 3fc83020634044638e2efad4a6bef386 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-23 22:14:52 UTC) from {username='kudu', principal='kudu/127.1.135.254@KRBTEST.COM'} at 127.0.0.1:55680
I20260923 22:14:53.062726 16023 tablet_replica.cc:330] stopping tablet replica
I20260923 22:14:53.062860 16023 raft_consensus.cc:2229] T 3fc83020634044638e2efad4a6bef386 P 2ac26bd2c8eb4b4fb12e08af1af156e2 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260923 22:14:53.062950 16023 raft_consensus.cc:2258] T 3fc83020634044638e2efad4a6bef386 P 2ac26bd2c8eb4b4fb12e08af1af156e2 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260923 22:14:53.063235 16023 ts_tablet_manager.cc:1905] T 3fc83020634044638e2efad4a6bef386 P 2ac26bd2c8eb4b4fb12e08af1af156e2: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260923 22:14:53.065183 16023 ts_tablet_manager.cc:1918] T 3fc83020634044638e2efad4a6bef386 P 2ac26bd2c8eb4b4fb12e08af1af156e2: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260923 22:14:53.065239 16023 log.cc:1196] T 3fc83020634044638e2efad4a6bef386 P 2ac26bd2c8eb4b4fb12e08af1af156e2: Deleting WAL directory at /tmp/dist-test-taskRiQe_a/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790201539513534-1567-0/minicluster-data/ts-0/wal/wals/3fc83020634044638e2efad4a6bef386
I20260923 22:14:53.065475 16023 ts_tablet_manager.cc:1939] T 3fc83020634044638e2efad4a6bef386 P 2ac26bd2c8eb4b4fb12e08af1af156e2: Deleting consensus metadata
I20260923 22:14:53.065910 15812 catalog_manager.cc:4841] TS 2ac26bd2c8eb4b4fb12e08af1af156e2 (127.1.135.193:37137): tablet 3fc83020634044638e2efad4a6bef386 (table test-table [id=0098df898fff4bf69cf3519e4d5d48ab]) successfully deleted
W20260923 22:14:53.125952 16020 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260923 22:14:53.126154 16020 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260923 22:14:53.126189 16020 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260923 22:14:53.127257 16020 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260923 22:14:53.127318 16020 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260923 22:14:53.127362 16020 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.1.135.194
I20260923 22:14:53.128835 16020 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.1.135.212:38425
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskRiQe_a/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790201539513534-1567-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskRiQe_a/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790201539513534-1567-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskRiQe_a/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790201539513534-1567-0/krb5kdc/kudu_127.1.135.194.keytab
--principal=kudu/127.1.135.194
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.1.135.194:46495
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskRiQe_a/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790201539513534-1567-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.1.135.194
--webserver_port=36489
--tserver_master_addrs=127.1.135.254:37515
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.16020
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.1.135.194
--log_dir=/tmp/dist-test-taskRiQe_a/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790201539513534-1567-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 23 Sep 2026 20:53:37 UTC on ccycloud.quasar-rkviyx.root.comops.site
build id 4668083
I20260923 22:14:53.129061 16020 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260923 22:14:53.129256 16020 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260923 22:14:53.133924 16028 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260923 22:14:53.133955 16031 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260923 22:14:53.134001 16029 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260923 22:14:53.134220 16020 server_base.cc:1034] running on GCE node
I20260923 22:14:53.134414 16020 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260923 22:14:53.134675 16020 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260923 22:14:53.135835 16020 hybrid_clock.cc:648] HybridClock initialized: now 1790201693135797 us; error 36 us; skew 500 ppm
Sep 23 22:14:53 dist-test-slave-s2fj krb5kdc[15248](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790201693, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.135.194@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260923 22:14:53.141641 16020 init.cc:377] Logged in from keytab as kudu/127.1.135.194@KRBTEST.COM (short username kudu)
I20260923 22:14:53.142057 16020 webserver.cc:460] Webserver started at http://127.1.135.194:36489/ using document root <none> and password file <none>
I20260923 22:14:53.142233 16020 fs_manager.cc:359] Metadata directory not provided
I20260923 22:14:53.142282 16020 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260923 22:14:53.143602 16020 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.001s sys 0.000s
I20260923 22:14:53.144204 16038 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260923 22:14:53.144347 16020 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.000s
I20260923 22:14:53.144426 16020 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskRiQe_a/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790201539513534-1567-0/minicluster-data/ts-1/data,/tmp/dist-test-taskRiQe_a/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790201539513534-1567-0/minicluster-data/ts-1/wal
uuid: "7d6af16a92f948c98fa0a8c4d04da648"
format_stamp: "Formatted at 2026-09-23 22:14:52 on dist-test-slave-s2fj"
server_key: "5ed27962f37132d31849db12e425c61f"
server_key_iv: "ce92088ae7b1e9838e4e018406906389"
server_key_version: "encryptionkey@0"
I20260923 22:14:53.144688 16020 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskRiQe_a/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790201539513534-1567-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskRiQe_a/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790201539513534-1567-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskRiQe_a/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790201539513534-1567-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260923 22:14:53.155853 16020 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260923 22:14:53.157676 16020 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260923 22:14:53.157778 16020 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260923 22:14:53.157927 16020 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260923 22:14:53.158157 16020 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260923 22:14:53.158186 16020 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260923 22:14:53.158216 16020 ts_tablet_manager.cc:610] Registered 0 tablets
I20260923 22:14:53.158228 16020 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260923 22:14:53.163503 16020 rpc_server.cc:307] RPC server started. Bound to: 127.1.135.194:46495
I20260923 22:14:53.163582 16151 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.1.135.194:46495 every 8 connection(s)
I20260923 22:14:53.163877 16020 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskRiQe_a/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790201539513534-1567-0/minicluster-data/ts-1/data/info.pb
I20260923 22:14:53.169109 1567 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskRiQe_a/build/release/bin/kudu as pid 16020
I20260923 22:14:53.169374 1567 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskRiQe_a/build/release/bin/kudu
/tmp/dist-test-taskRiQe_a/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskRiQe_a/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790201539513534-1567-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskRiQe_a/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790201539513534-1567-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskRiQe_a/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790201539513534-1567-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskRiQe_a/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790201539513534-1567-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.1.135.195:34569
--local_ip_for_outbound_sockets=127.1.135.195
--tserver_master_addrs=127.1.135.254:37515
--webserver_port=37309
--webserver_interface=127.1.135.195
--keytab_file=/tmp/dist-test-taskRiQe_a/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790201539513534-1567-0/krb5kdc/kudu_127.1.135.195.keytab
--principal=kudu/127.1.135.195
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.1.135.212:38425
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskRiQe_a/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790201539513534-1567-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskRiQe_a/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790201539513534-1567-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskRiQe_a/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790201539513534-1567-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
Sep 23 22:14:53 dist-test-slave-s2fj krb5kdc[15248](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790201693, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.135.194@KRBTEST.COM for kudu/127.1.135.254@KRBTEST.COM
I20260923 22:14:53.173089 16017 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 22:14:53.164537 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.135.194:51423 (local address 127.1.135.254:37515)
0923 22:14:53.164600 (+ 63us) server_negotiation.cc:206] Beginning negotiation
0923 22:14:53.164602 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0923 22:14:53.164796 (+ 194us) server_negotiation.cc:407] Connection header received
0923 22:14:53.165192 (+ 396us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 22:14:53.165193 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 22:14:53.165202 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 22:14:53.165247 (+ 45us) server_negotiation.cc:226] Negotiated authn=SASL
0923 22:14:53.166076 (+ 829us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 22:14:53.166557 (+ 481us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 22:14:53.167706 (+ 1149us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 22:14:53.167891 (+ 185us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 22:14:53.167940 (+ 49us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 22:14:53.170097 (+ 2157us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0923 22:14:53.170099 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0923 22:14:53.170099 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0923 22:14:53.170107 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0923 22:14:53.171747 (+ 1640us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0923 22:14:53.172154 (+ 407us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0923 22:14:53.172155 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0923 22:14:53.172156 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0923 22:14:53.172219 (+ 63us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0923 22:14:53.172475 (+ 256us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0923 22:14:53.172476 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0923 22:14:53.172477 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0923 22:14:53.172606 (+ 129us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0923 22:14:53.172672 (+ 66us) server_negotiation.cc:1035] Waiting for connection context
0923 22:14:53.172921 (+ 249us) server_negotiation.cc:299] Negotiation successful
0923 22:14:53.173004 (+ 83us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":38}
I20260923 22:14:53.173245 16154 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 22:14:53.164530 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.135.254:37515 (local address 127.1.135.194:51423)
0923 22:14:53.164683 (+ 153us) negotiation.cc:107] Waiting for socket to connect
0923 22:14:53.164718 (+ 35us) client_negotiation.cc:174] Beginning negotiation
0923 22:14:53.165118 (+ 400us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0923 22:14:53.165299 (+ 181us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0923 22:14:53.165302 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0923 22:14:53.165837 (+ 535us) client_negotiation.cc:189] Negotiated authn=SASL
0923 22:14:53.166003 (+ 166us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 22:14:53.166007 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 22:14:53.166630 (+ 623us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 22:14:53.166631 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 22:14:53.167627 (+ 996us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 22:14:53.167630 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 22:14:53.167958 (+ 328us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 22:14:53.167959 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 22:14:53.168078 (+ 119us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 22:14:53.168549 (+ 471us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0923 22:14:53.168556 (+ 7us) client_negotiation.cc:656] calling sasl_client_start()
0923 22:14:53.169995 (+ 1439us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0923 22:14:53.171815 (+ 1820us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0923 22:14:53.171821 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0923 22:14:53.171823 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0923 22:14:53.172028 (+ 205us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0923 22:14:53.172297 (+ 269us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0923 22:14:53.172298 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0923 22:14:53.172298 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0923 22:14:53.172391 (+ 93us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0923 22:14:53.172675 (+ 284us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0923 22:14:53.172681 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0923 22:14:53.172797 (+ 116us) client_negotiation.cc:769] Sending connection context
0923 22:14:53.172888 (+ 91us) client_negotiation.cc:240] Negotiation successful
0923 22:14:53.173020 (+ 132us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":113,"thread_start_us":85,"threads_started":1}
I20260923 22:14:53.174038 16152 heartbeater.cc:347] Connected to a master server at 127.1.135.254:37515
I20260923 22:14:53.174136 16152 heartbeater.cc:464] Registering TS with master...
I20260923 22:14:53.174295 16152 heartbeater.cc:511] Master 127.1.135.254:37515 requested a full tablet report, sending...
I20260923 22:14:53.174723 15826 ts_manager.cc:194] Registered new tserver with Master: 7d6af16a92f948c98fa0a8c4d04da648 (127.1.135.194:46495)
I20260923 22:14:53.175063 15826 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.1.135.194@KRBTEST.COM'} at 127.1.135.194:51423
W20260923 22:14:53.249636 16155 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260923 22:14:53.249809 16155 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260923 22:14:53.249831 16155 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260923 22:14:53.250942 16155 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260923 22:14:53.250983 16155 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260923 22:14:53.251036 16155 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.1.135.195
I20260923 22:14:53.252521 16155 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.1.135.212:38425
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskRiQe_a/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790201539513534-1567-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskRiQe_a/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790201539513534-1567-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskRiQe_a/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790201539513534-1567-0/krb5kdc/kudu_127.1.135.195.keytab
--principal=kudu/127.1.135.195
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.1.135.195:34569
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskRiQe_a/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790201539513534-1567-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.1.135.195
--webserver_port=37309
--tserver_master_addrs=127.1.135.254:37515
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.16155
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.1.135.195
--log_dir=/tmp/dist-test-taskRiQe_a/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790201539513534-1567-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 23 Sep 2026 20:53:37 UTC on ccycloud.quasar-rkviyx.root.comops.site
build id 4668083
I20260923 22:14:53.252728 16155 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260923 22:14:53.252952 16155 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260923 22:14:53.257598 16161 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260923 22:14:53.257580 16160 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260923 22:14:53.257592 16163 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260923 22:14:53.257795 16155 server_base.cc:1034] running on GCE node
I20260923 22:14:53.257984 16155 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260923 22:14:53.258221 16155 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260923 22:14:53.259395 16155 hybrid_clock.cc:648] HybridClock initialized: now 1790201693259377 us; error 29 us; skew 500 ppm
Sep 23 22:14:53 dist-test-slave-s2fj krb5kdc[15248](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790201693, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.135.195@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260923 22:14:53.261786 16155 init.cc:377] Logged in from keytab as kudu/127.1.135.195@KRBTEST.COM (short username kudu)
I20260923 22:14:53.262151 16155 webserver.cc:460] Webserver started at http://127.1.135.195:37309/ using document root <none> and password file <none>
I20260923 22:14:53.262336 16155 fs_manager.cc:359] Metadata directory not provided
I20260923 22:14:53.262382 16155 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260923 22:14:53.263592 16155 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.001s sys 0.000s
I20260923 22:14:53.264158 16170 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260923 22:14:53.264302 16155 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.000s
I20260923 22:14:53.264381 16155 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskRiQe_a/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790201539513534-1567-0/minicluster-data/ts-2/data,/tmp/dist-test-taskRiQe_a/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790201539513534-1567-0/minicluster-data/ts-2/wal
uuid: "7f8b6e024a774ca996fd7858a93c42b2"
format_stamp: "Formatted at 2026-09-23 22:14:52 on dist-test-slave-s2fj"
server_key: "93f7a16cb6e8af959504c1302d41db8e"
server_key_iv: "8b41b00d37b8db1e35715c0c63d26e2d"
server_key_version: "encryptionkey@0"
I20260923 22:14:53.264670 16155 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskRiQe_a/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790201539513534-1567-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskRiQe_a/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790201539513534-1567-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskRiQe_a/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790201539513534-1567-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260923 22:14:53.276854 16155 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260923 22:14:53.278884 16155 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260923 22:14:53.278982 16155 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260923 22:14:53.279130 16155 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260923 22:14:53.279335 16155 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260923 22:14:53.279362 16155 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260923 22:14:53.279381 16155 ts_tablet_manager.cc:610] Registered 0 tablets
I20260923 22:14:53.279393 16155 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260923 22:14:53.284827 16155 rpc_server.cc:307] RPC server started. Bound to: 127.1.135.195:34569
I20260923 22:14:53.284879 16283 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.1.135.195:34569 every 8 connection(s)
I20260923 22:14:53.285152 16155 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskRiQe_a/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790201539513534-1567-0/minicluster-data/ts-2/data/info.pb
Sep 23 22:14:53 dist-test-slave-s2fj krb5kdc[15248](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790201693, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.135.195@KRBTEST.COM for kudu/127.1.135.254@KRBTEST.COM
I20260923 22:14:53.293200 16017 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 22:14:53.285784 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.135.195:49717 (local address 127.1.135.254:37515)
0923 22:14:53.285850 (+ 66us) server_negotiation.cc:206] Beginning negotiation
0923 22:14:53.285853 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0923 22:14:53.286048 (+ 195us) server_negotiation.cc:407] Connection header received
0923 22:14:53.286502 (+ 454us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 22:14:53.286503 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 22:14:53.286514 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 22:14:53.286560 (+ 46us) server_negotiation.cc:226] Negotiated authn=SASL
0923 22:14:53.287399 (+ 839us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 22:14:53.287841 (+ 442us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 22:14:53.288791 (+ 950us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 22:14:53.288970 (+ 179us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 22:14:53.289034 (+ 64us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 22:14:53.290830 (+ 1796us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0923 22:14:53.290831 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0923 22:14:53.290832 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0923 22:14:53.290838 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0923 22:14:53.292229 (+ 1391us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0923 22:14:53.292534 (+ 305us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0923 22:14:53.292535 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0923 22:14:53.292535 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0923 22:14:53.292580 (+ 45us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0923 22:14:53.292763 (+ 183us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0923 22:14:53.292763 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0923 22:14:53.292763 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0923 22:14:53.292852 (+ 89us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0923 22:14:53.292899 (+ 47us) server_negotiation.cc:1035] Waiting for connection context
0923 22:14:53.293087 (+ 188us) server_negotiation.cc:299] Negotiation successful
0923 22:14:53.293140 (+ 53us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":36}
I20260923 22:14:53.293435 16286 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 22:14:53.285777 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.135.254:37515 (local address 127.1.135.195:49717)
0923 22:14:53.285939 (+ 162us) negotiation.cc:107] Waiting for socket to connect
0923 22:14:53.285967 (+ 28us) client_negotiation.cc:174] Beginning negotiation
0923 22:14:53.286420 (+ 453us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0923 22:14:53.286624 (+ 204us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0923 22:14:53.286627 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0923 22:14:53.287141 (+ 514us) client_negotiation.cc:189] Negotiated authn=SASL
0923 22:14:53.287313 (+ 172us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 22:14:53.287316 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 22:14:53.287927 (+ 611us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 22:14:53.287928 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 22:14:53.288719 (+ 791us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 22:14:53.288721 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 22:14:53.289042 (+ 321us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 22:14:53.289043 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 22:14:53.289131 (+ 88us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 22:14:53.289500 (+ 369us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0923 22:14:53.289506 (+ 6us) client_negotiation.cc:656] calling sasl_client_start()
0923 22:14:53.290753 (+ 1247us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0923 22:14:53.292289 (+ 1536us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0923 22:14:53.292294 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0923 22:14:53.292295 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0923 22:14:53.292476 (+ 181us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0923 22:14:53.292632 (+ 156us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0923 22:14:53.292632 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0923 22:14:53.292633 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0923 22:14:53.292714 (+ 81us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0923 22:14:53.292903 (+ 189us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0923 22:14:53.292906 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0923 22:14:53.292999 (+ 93us) client_negotiation.cc:769] Sending connection context
0923 22:14:53.293069 (+ 70us) client_negotiation.cc:240] Negotiation successful
0923 22:14:53.293169 (+ 100us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":125,"thread_start_us":91,"threads_started":1}
I20260923 22:14:53.293893 16284 heartbeater.cc:347] Connected to a master server at 127.1.135.254:37515
I20260923 22:14:53.293990 16284 heartbeater.cc:464] Registering TS with master...
I20260923 22:14:53.294148 16284 heartbeater.cc:511] Master 127.1.135.254:37515 requested a full tablet report, sending...
I20260923 22:14:53.294494 1567 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskRiQe_a/build/release/bin/kudu as pid 16155
I20260923 22:14:53.294565 15826 ts_manager.cc:194] Registered new tserver with Master: 7f8b6e024a774ca996fd7858a93c42b2 (127.1.135.195:34569)
I20260923 22:14:53.295032 15826 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.1.135.195@KRBTEST.COM'} at 127.1.135.195:49717
I20260923 22:14:53.300552 16017 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 22:14:53.294904 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49312 (local address 127.1.135.254:37515)
0923 22:14:53.294978 (+ 74us) server_negotiation.cc:206] Beginning negotiation
0923 22:14:53.294980 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0923 22:14:53.294989 (+ 9us) server_negotiation.cc:407] Connection header received
0923 22:14:53.295085 (+ 96us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 22:14:53.295085 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 22:14:53.295106 (+ 21us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 22:14:53.295148 (+ 42us) server_negotiation.cc:226] Negotiated authn=SASL
0923 22:14:53.295723 (+ 575us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 22:14:53.296248 (+ 525us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 22:14:53.296954 (+ 706us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 22:14:53.297161 (+ 207us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 22:14:53.297238 (+ 77us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 22:14:53.298346 (+ 1108us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0923 22:14:53.298347 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0923 22:14:53.298347 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0923 22:14:53.298353 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0923 22:14:53.299412 (+ 1059us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0923 22:14:53.299752 (+ 340us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0923 22:14:53.299754 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0923 22:14:53.299754 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0923 22:14:53.299801 (+ 47us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0923 22:14:53.300051 (+ 250us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0923 22:14:53.300052 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0923 22:14:53.300052 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0923 22:14:53.300176 (+ 124us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0923 22:14:53.300346 (+ 170us) server_negotiation.cc:1035] Waiting for connection context
0923 22:14:53.300382 (+ 36us) server_negotiation.cc:299] Negotiation successful
0923 22:14:53.300451 (+ 69us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":55}
I20260923 22:14:53.301239 1567 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260923 22:14:53.303977 16017 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 22:14:53.301590 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49320 (local address 127.1.135.254:37515)
0923 22:14:53.301646 (+ 56us) server_negotiation.cc:206] Beginning negotiation
0923 22:14:53.301648 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0923 22:14:53.301703 (+ 55us) server_negotiation.cc:407] Connection header received
0923 22:14:53.301782 (+ 79us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 22:14:53.301783 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 22:14:53.301849 (+ 66us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 22:14:53.301921 (+ 72us) server_negotiation.cc:226] Negotiated authn=TOKEN
0923 22:14:53.302024 (+ 103us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 22:14:53.302468 (+ 444us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 22:14:53.303243 (+ 775us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 22:14:53.303437 (+ 194us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 22:14:53.303525 (+ 88us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 22:14:53.303692 (+ 167us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0923 22:14:53.303786 (+ 94us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0923 22:14:53.303876 (+ 90us) server_negotiation.cc:1035] Waiting for connection context
0923 22:14:53.303894 (+ 18us) server_negotiation.cc:299] Negotiation successful
0923 22:14:53.303910 (+ 16us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":35}
I20260923 22:14:53.304226 15826 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin'} at 127.0.0.1:49320:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260923 22:14:53.305780 15826 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260923 22:14:53.307775 15947 tablet_service.cc:1467] Processing CreateTablet for tablet a64cc070467e497dad4bebe37959837b (DEFAULT_TABLE table=test-table [id=db1628a18a854699818f1a42e33e4c0e]), partition=RANGE (key) PARTITION UNBOUNDED
I20260923 22:14:53.307955 15947 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet a64cc070467e497dad4bebe37959837b. 1 dirs total, 0 dirs full, 0 dirs failed
I20260923 22:14:53.309598 15906 tablet_bootstrap.cc:492] T a64cc070467e497dad4bebe37959837b P 2ac26bd2c8eb4b4fb12e08af1af156e2: Bootstrap starting.
I20260923 22:14:53.309921 15906 tablet_bootstrap.cc:654] T a64cc070467e497dad4bebe37959837b P 2ac26bd2c8eb4b4fb12e08af1af156e2: Neither blocks nor log segments found. Creating new log.
I20260923 22:14:53.310757 15906 tablet_bootstrap.cc:492] T a64cc070467e497dad4bebe37959837b P 2ac26bd2c8eb4b4fb12e08af1af156e2: No bootstrap required, opened a new log
I20260923 22:14:53.310814 15906 ts_tablet_manager.cc:1397] T a64cc070467e497dad4bebe37959837b P 2ac26bd2c8eb4b4fb12e08af1af156e2: Time spent bootstrapping tablet: real 0.001s user 0.001s sys 0.000s
I20260923 22:14:53.310959 15906 raft_consensus.cc:348] T a64cc070467e497dad4bebe37959837b P 2ac26bd2c8eb4b4fb12e08af1af156e2 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7d6af16a92f948c98fa0a8c4d04da648" member_type: VOTER last_known_addr { host: "127.1.135.194" port: 46495 } } peers { permanent_uuid: "7f8b6e024a774ca996fd7858a93c42b2" member_type: VOTER last_known_addr { host: "127.1.135.195" port: 34569 } } peers { permanent_uuid: "2ac26bd2c8eb4b4fb12e08af1af156e2" member_type: VOTER last_known_addr { host: "127.1.135.193" port: 37137 } }
I20260923 22:14:53.311030 15906 raft_consensus.cc:374] T a64cc070467e497dad4bebe37959837b P 2ac26bd2c8eb4b4fb12e08af1af156e2 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260923 22:14:53.311066 15906 raft_consensus.cc:729] T a64cc070467e497dad4bebe37959837b P 2ac26bd2c8eb4b4fb12e08af1af156e2 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 2ac26bd2c8eb4b4fb12e08af1af156e2, State: Initialized, Role: FOLLOWER
I20260923 22:14:53.311116 15906 consensus_queue.cc:260] T a64cc070467e497dad4bebe37959837b P 2ac26bd2c8eb4b4fb12e08af1af156e2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7d6af16a92f948c98fa0a8c4d04da648" member_type: VOTER last_known_addr { host: "127.1.135.194" port: 46495 } } peers { permanent_uuid: "7f8b6e024a774ca996fd7858a93c42b2" member_type: VOTER last_known_addr { host: "127.1.135.195" port: 34569 } } peers { permanent_uuid: "2ac26bd2c8eb4b4fb12e08af1af156e2" member_type: VOTER last_known_addr { host: "127.1.135.193" port: 37137 } }
I20260923 22:14:53.311205 15906 ts_tablet_manager.cc:1428] T a64cc070467e497dad4bebe37959837b P 2ac26bd2c8eb4b4fb12e08af1af156e2: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20260923 22:14:53.311859 16289 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 22:14:53.307656 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.135.195:34569 (local address 127.0.0.1:42552)
0923 22:14:53.308014 (+ 358us) negotiation.cc:107] Waiting for socket to connect
0923 22:14:53.308024 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0923 22:14:53.308144 (+ 120us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0923 22:14:53.308341 (+ 197us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0923 22:14:53.308342 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0923 22:14:53.308344 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0923 22:14:53.308376 (+ 32us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 22:14:53.308378 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 22:14:53.309532 (+ 1154us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 22:14:53.309533 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 22:14:53.310643 (+ 1110us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 22:14:53.310646 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 22:14:53.311639 (+ 993us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 22:14:53.311640 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 22:14:53.311699 (+ 59us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 22:14:53.311717 (+ 18us) client_negotiation.cc:769] Sending connection context
0923 22:14:53.311765 (+ 48us) client_negotiation.cc:240] Negotiation successful
0923 22:14:53.311797 (+ 32us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":347,"thread_start_us":49,"threads_started":1}
I20260923 22:14:53.312107 16291 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 22:14:53.307821 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42552 (local address 127.1.135.195:34569)
0923 22:14:53.307972 (+ 151us) server_negotiation.cc:206] Beginning negotiation
0923 22:14:53.307976 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0923 22:14:53.308119 (+ 143us) server_negotiation.cc:407] Connection header received
0923 22:14:53.308234 (+ 115us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 22:14:53.308236 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 22:14:53.308251 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 22:14:53.308331 (+ 80us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0923 22:14:53.308430 (+ 99us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 22:14:53.309417 (+ 987us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 22:14:53.310719 (+ 1302us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 22:14:53.311481 (+ 762us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 22:14:53.311546 (+ 65us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 22:14:53.311594 (+ 48us) server_negotiation.cc:1035] Waiting for connection context
0923 22:14:53.311968 (+ 374us) server_negotiation.cc:299] Negotiation successful
0923 22:14:53.312014 (+ 46us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":129,"thread_start_us":77,"threads_started":1}
I20260923 22:14:53.312292 16021 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 22:14:53.307544 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.135.194:46495 (local address 127.0.0.1:48764)
0923 22:14:53.307738 (+ 194us) negotiation.cc:107] Waiting for socket to connect
0923 22:14:53.307747 (+ 9us) client_negotiation.cc:174] Beginning negotiation
0923 22:14:53.307780 (+ 33us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0923 22:14:53.308110 (+ 330us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0923 22:14:53.308111 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0923 22:14:53.308113 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0923 22:14:53.308152 (+ 39us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 22:14:53.308154 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 22:14:53.309325 (+ 1171us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 22:14:53.309326 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 22:14:53.310462 (+ 1136us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 22:14:53.310465 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 22:14:53.312036 (+ 1571us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 22:14:53.312037 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 22:14:53.312127 (+ 90us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 22:14:53.312130 (+ 3us) client_negotiation.cc:769] Sending connection context
0923 22:14:53.312186 (+ 56us) client_negotiation.cc:240] Negotiation successful
0923 22:14:53.312230 (+ 44us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":181}
I20260923 22:14:53.312474 16290 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 22:14:53.307654 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48764 (local address 127.1.135.194:46495)
0923 22:14:53.307888 (+ 234us) server_negotiation.cc:206] Beginning negotiation
0923 22:14:53.307892 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0923 22:14:53.307902 (+ 10us) server_negotiation.cc:407] Connection header received
0923 22:14:53.307941 (+ 39us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 22:14:53.307943 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 22:14:53.307959 (+ 16us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 22:14:53.308226 (+ 267us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0923 22:14:53.308295 (+ 69us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 22:14:53.309223 (+ 928us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 22:14:53.310549 (+ 1326us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 22:14:53.311937 (+ 1388us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 22:14:53.312029 (+ 92us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 22:14:53.312116 (+ 87us) server_negotiation.cc:1035] Waiting for connection context
0923 22:14:53.312360 (+ 244us) server_negotiation.cc:299] Negotiation successful
0923 22:14:53.312399 (+ 39us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":201,"thread_start_us":81,"threads_started":1}
I20260923 22:14:53.312693 16218 tablet_service.cc:1467] Processing CreateTablet for tablet a64cc070467e497dad4bebe37959837b (DEFAULT_TABLE table=test-table [id=db1628a18a854699818f1a42e33e4c0e]), partition=RANGE (key) PARTITION UNBOUNDED
I20260923 22:14:53.312952 16218 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet a64cc070467e497dad4bebe37959837b. 1 dirs total, 0 dirs full, 0 dirs failed
I20260923 22:14:53.315243 16293 tablet_bootstrap.cc:492] T a64cc070467e497dad4bebe37959837b P 7f8b6e024a774ca996fd7858a93c42b2: Bootstrap starting.
I20260923 22:14:53.315917 16293 tablet_bootstrap.cc:654] T a64cc070467e497dad4bebe37959837b P 7f8b6e024a774ca996fd7858a93c42b2: Neither blocks nor log segments found. Creating new log.
I20260923 22:14:53.315815 16086 tablet_service.cc:1467] Processing CreateTablet for tablet a64cc070467e497dad4bebe37959837b (DEFAULT_TABLE table=test-table [id=db1628a18a854699818f1a42e33e4c0e]), partition=RANGE (key) PARTITION UNBOUNDED
I20260923 22:14:53.316160 16086 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet a64cc070467e497dad4bebe37959837b. 1 dirs total, 0 dirs full, 0 dirs failed
I20260923 22:14:53.316550 16293 log.cc:824] T a64cc070467e497dad4bebe37959837b P 7f8b6e024a774ca996fd7858a93c42b2: Log is configured to *not* fsync() on all Append() calls
I20260923 22:14:53.317243 16293 tablet_bootstrap.cc:492] T a64cc070467e497dad4bebe37959837b P 7f8b6e024a774ca996fd7858a93c42b2: No bootstrap required, opened a new log
I20260923 22:14:53.317322 16293 ts_tablet_manager.cc:1397] T a64cc070467e497dad4bebe37959837b P 7f8b6e024a774ca996fd7858a93c42b2: Time spent bootstrapping tablet: real 0.002s user 0.000s sys 0.001s
I20260923 22:14:53.319181 16293 raft_consensus.cc:348] T a64cc070467e497dad4bebe37959837b P 7f8b6e024a774ca996fd7858a93c42b2 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7d6af16a92f948c98fa0a8c4d04da648" member_type: VOTER last_known_addr { host: "127.1.135.194" port: 46495 } } peers { permanent_uuid: "7f8b6e024a774ca996fd7858a93c42b2" member_type: VOTER last_known_addr { host: "127.1.135.195" port: 34569 } } peers { permanent_uuid: "2ac26bd2c8eb4b4fb12e08af1af156e2" member_type: VOTER last_known_addr { host: "127.1.135.193" port: 37137 } }
I20260923 22:14:53.319350 16293 raft_consensus.cc:374] T a64cc070467e497dad4bebe37959837b P 7f8b6e024a774ca996fd7858a93c42b2 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260923 22:14:53.319399 16293 raft_consensus.cc:729] T a64cc070467e497dad4bebe37959837b P 7f8b6e024a774ca996fd7858a93c42b2 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 7f8b6e024a774ca996fd7858a93c42b2, State: Initialized, Role: FOLLOWER
I20260923 22:14:53.319509 16293 consensus_queue.cc:260] T a64cc070467e497dad4bebe37959837b P 7f8b6e024a774ca996fd7858a93c42b2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7d6af16a92f948c98fa0a8c4d04da648" member_type: VOTER last_known_addr { host: "127.1.135.194" port: 46495 } } peers { permanent_uuid: "7f8b6e024a774ca996fd7858a93c42b2" member_type: VOTER last_known_addr { host: "127.1.135.195" port: 34569 } } peers { permanent_uuid: "2ac26bd2c8eb4b4fb12e08af1af156e2" member_type: VOTER last_known_addr { host: "127.1.135.193" port: 37137 } }
I20260923 22:14:53.319730 16293 ts_tablet_manager.cc:1428] T a64cc070467e497dad4bebe37959837b P 7f8b6e024a774ca996fd7858a93c42b2: Time spent starting tablet: real 0.002s user 0.000s sys 0.003s
I20260923 22:14:53.319772 16284 heartbeater.cc:503] Master 127.1.135.254:37515 was elected leader, sending a full tablet report...
I20260923 22:14:53.319913 16295 tablet_bootstrap.cc:492] T a64cc070467e497dad4bebe37959837b P 7d6af16a92f948c98fa0a8c4d04da648: Bootstrap starting.
I20260923 22:14:53.320582 16295 tablet_bootstrap.cc:654] T a64cc070467e497dad4bebe37959837b P 7d6af16a92f948c98fa0a8c4d04da648: Neither blocks nor log segments found. Creating new log.
I20260923 22:14:53.321014 16295 log.cc:824] T a64cc070467e497dad4bebe37959837b P 7d6af16a92f948c98fa0a8c4d04da648: Log is configured to *not* fsync() on all Append() calls
I20260923 22:14:53.321607 16295 tablet_bootstrap.cc:492] T a64cc070467e497dad4bebe37959837b P 7d6af16a92f948c98fa0a8c4d04da648: No bootstrap required, opened a new log
I20260923 22:14:53.321655 16295 ts_tablet_manager.cc:1397] T a64cc070467e497dad4bebe37959837b P 7d6af16a92f948c98fa0a8c4d04da648: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20260923 22:14:53.322888 16295 raft_consensus.cc:348] T a64cc070467e497dad4bebe37959837b P 7d6af16a92f948c98fa0a8c4d04da648 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7d6af16a92f948c98fa0a8c4d04da648" member_type: VOTER last_known_addr { host: "127.1.135.194" port: 46495 } } peers { permanent_uuid: "7f8b6e024a774ca996fd7858a93c42b2" member_type: VOTER last_known_addr { host: "127.1.135.195" port: 34569 } } peers { permanent_uuid: "2ac26bd2c8eb4b4fb12e08af1af156e2" member_type: VOTER last_known_addr { host: "127.1.135.193" port: 37137 } }
I20260923 22:14:53.323029 16295 raft_consensus.cc:374] T a64cc070467e497dad4bebe37959837b P 7d6af16a92f948c98fa0a8c4d04da648 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260923 22:14:53.323083 16295 raft_consensus.cc:729] T a64cc070467e497dad4bebe37959837b P 7d6af16a92f948c98fa0a8c4d04da648 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 7d6af16a92f948c98fa0a8c4d04da648, State: Initialized, Role: FOLLOWER
I20260923 22:14:53.323210 16295 consensus_queue.cc:260] T a64cc070467e497dad4bebe37959837b P 7d6af16a92f948c98fa0a8c4d04da648 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7d6af16a92f948c98fa0a8c4d04da648" member_type: VOTER last_known_addr { host: "127.1.135.194" port: 46495 } } peers { permanent_uuid: "7f8b6e024a774ca996fd7858a93c42b2" member_type: VOTER last_known_addr { host: "127.1.135.195" port: 34569 } } peers { permanent_uuid: "2ac26bd2c8eb4b4fb12e08af1af156e2" member_type: VOTER last_known_addr { host: "127.1.135.193" port: 37137 } }
I20260923 22:14:53.323447 16295 ts_tablet_manager.cc:1428] T a64cc070467e497dad4bebe37959837b P 7d6af16a92f948c98fa0a8c4d04da648: Time spent starting tablet: real 0.002s user 0.002s sys 0.000s
I20260923 22:14:53.323453 16152 heartbeater.cc:503] Master 127.1.135.254:37515 was elected leader, sending a full tablet report...
I20260923 22:14:53.333418 16019 raft_consensus.cc:482] T a64cc070467e497dad4bebe37959837b P 2ac26bd2c8eb4b4fb12e08af1af156e2 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260923 22:14:53.333503 16019 raft_consensus.cc:504] T a64cc070467e497dad4bebe37959837b P 2ac26bd2c8eb4b4fb12e08af1af156e2 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7d6af16a92f948c98fa0a8c4d04da648" member_type: VOTER last_known_addr { host: "127.1.135.194" port: 46495 } } peers { permanent_uuid: "7f8b6e024a774ca996fd7858a93c42b2" member_type: VOTER last_known_addr { host: "127.1.135.195" port: 34569 } } peers { permanent_uuid: "2ac26bd2c8eb4b4fb12e08af1af156e2" member_type: VOTER last_known_addr { host: "127.1.135.193" port: 37137 } }
I20260923 22:14:53.333745 16019 leader_election.cc:288] T a64cc070467e497dad4bebe37959837b P 2ac26bd2c8eb4b4fb12e08af1af156e2 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 7d6af16a92f948c98fa0a8c4d04da648 (127.1.135.194:46495), 7f8b6e024a774ca996fd7858a93c42b2 (127.1.135.195:34569)
I20260923 22:14:53.336534 16290 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 22:14:53.334006 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.135.193:44559 (local address 127.1.135.194:46495)
0923 22:14:53.334088 (+ 82us) server_negotiation.cc:206] Beginning negotiation
0923 22:14:53.334090 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0923 22:14:53.334099 (+ 9us) server_negotiation.cc:407] Connection header received
0923 22:14:53.334119 (+ 20us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 22:14:53.334120 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 22:14:53.334129 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 22:14:53.334187 (+ 58us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0923 22:14:53.334330 (+ 143us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 22:14:53.334863 (+ 533us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 22:14:53.335828 (+ 965us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 22:14:53.336211 (+ 383us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 22:14:53.336284 (+ 73us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 22:14:53.336291 (+ 7us) server_negotiation.cc:1035] Waiting for connection context
0923 22:14:53.336447 (+ 156us) server_negotiation.cc:299] Negotiation successful
0923 22:14:53.336477 (+ 30us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":62}
I20260923 22:14:53.336546 16016 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 22:14:53.333903 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.135.194:46495 (local address 127.1.135.193:44559)
0923 22:14:53.333959 (+ 56us) negotiation.cc:107] Waiting for socket to connect
0923 22:14:53.333969 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0923 22:14:53.334026 (+ 57us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0923 22:14:53.334190 (+ 164us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0923 22:14:53.334191 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0923 22:14:53.334193 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0923 22:14:53.334244 (+ 51us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 22:14:53.334246 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 22:14:53.334940 (+ 694us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 22:14:53.334941 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 22:14:53.335744 (+ 803us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 22:14:53.335747 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 22:14:53.336291 (+ 544us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 22:14:53.336292 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 22:14:53.336362 (+ 70us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 22:14:53.336381 (+ 19us) client_negotiation.cc:769] Sending connection context
0923 22:14:53.336442 (+ 61us) client_negotiation.cc:240] Negotiation successful
0923 22:14:53.336479 (+ 37us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":35}
I20260923 22:14:53.336634 16299 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 22:14:53.334003 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.135.195:34569 (local address 127.1.135.193:52643)
0923 22:14:53.334169 (+ 166us) negotiation.cc:107] Waiting for socket to connect
0923 22:14:53.334178 (+ 9us) client_negotiation.cc:174] Beginning negotiation
0923 22:14:53.334228 (+ 50us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0923 22:14:53.334390 (+ 162us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0923 22:14:53.334391 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0923 22:14:53.334392 (+ 1us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0923 22:14:53.334468 (+ 76us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 22:14:53.334469 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 22:14:53.335056 (+ 587us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 22:14:53.335057 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 22:14:53.335808 (+ 751us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 22:14:53.335811 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 22:14:53.336383 (+ 572us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 22:14:53.336384 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 22:14:53.336456 (+ 72us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 22:14:53.336464 (+ 8us) client_negotiation.cc:769] Sending connection context
0923 22:14:53.336531 (+ 67us) client_negotiation.cc:240] Negotiation successful
0923 22:14:53.336566 (+ 35us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":151,"thread_start_us":123,"threads_started":1}
I20260923 22:14:53.336627 16291 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 22:14:53.334079 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.135.193:52643 (local address 127.1.135.195:34569)
0923 22:14:53.334133 (+ 54us) server_negotiation.cc:206] Beginning negotiation
0923 22:14:53.334135 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0923 22:14:53.334232 (+ 97us) server_negotiation.cc:407] Connection header received
0923 22:14:53.334300 (+ 68us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 22:14:53.334301 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 22:14:53.334310 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 22:14:53.334391 (+ 81us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0923 22:14:53.334538 (+ 147us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 22:14:53.334995 (+ 457us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 22:14:53.335891 (+ 896us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 22:14:53.336298 (+ 407us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 22:14:53.336379 (+ 81us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 22:14:53.336385 (+ 6us) server_negotiation.cc:1035] Waiting for connection context
0923 22:14:53.336538 (+ 153us) server_negotiation.cc:299] Negotiation successful
0923 22:14:53.336569 (+ 31us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":34}
I20260923 22:14:53.336863 16106 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "a64cc070467e497dad4bebe37959837b" candidate_uuid: "2ac26bd2c8eb4b4fb12e08af1af156e2" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7d6af16a92f948c98fa0a8c4d04da648" is_pre_election: true
I20260923 22:14:53.336871 16238 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "a64cc070467e497dad4bebe37959837b" candidate_uuid: "2ac26bd2c8eb4b4fb12e08af1af156e2" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7f8b6e024a774ca996fd7858a93c42b2" is_pre_election: true
I20260923 22:14:53.336966 16238 raft_consensus.cc:2454] T a64cc070467e497dad4bebe37959837b P 7f8b6e024a774ca996fd7858a93c42b2 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 2ac26bd2c8eb4b4fb12e08af1af156e2 in term 0.
I20260923 22:14:53.336966 16106 raft_consensus.cc:2454] T a64cc070467e497dad4bebe37959837b P 7d6af16a92f948c98fa0a8c4d04da648 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 2ac26bd2c8eb4b4fb12e08af1af156e2 in term 0.
I20260923 22:14:53.337154 15901 leader_election.cc:302] T a64cc070467e497dad4bebe37959837b P 2ac26bd2c8eb4b4fb12e08af1af156e2 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2ac26bd2c8eb4b4fb12e08af1af156e2, 7f8b6e024a774ca996fd7858a93c42b2; no voters:
I20260923 22:14:53.337324 16019 raft_consensus.cc:2781] T a64cc070467e497dad4bebe37959837b P 2ac26bd2c8eb4b4fb12e08af1af156e2 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260923 22:14:53.337396 16019 raft_consensus.cc:482] T a64cc070467e497dad4bebe37959837b P 2ac26bd2c8eb4b4fb12e08af1af156e2 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260923 22:14:53.337419 16019 raft_consensus.cc:3037] T a64cc070467e497dad4bebe37959837b P 2ac26bd2c8eb4b4fb12e08af1af156e2 [term 0 FOLLOWER]: Advancing to term 1
I20260923 22:14:53.338027 16019 raft_consensus.cc:504] T a64cc070467e497dad4bebe37959837b P 2ac26bd2c8eb4b4fb12e08af1af156e2 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7d6af16a92f948c98fa0a8c4d04da648" member_type: VOTER last_known_addr { host: "127.1.135.194" port: 46495 } } peers { permanent_uuid: "7f8b6e024a774ca996fd7858a93c42b2" member_type: VOTER last_known_addr { host: "127.1.135.195" port: 34569 } } peers { permanent_uuid: "2ac26bd2c8eb4b4fb12e08af1af156e2" member_type: VOTER last_known_addr { host: "127.1.135.193" port: 37137 } }
I20260923 22:14:53.338171 16019 leader_election.cc:288] T a64cc070467e497dad4bebe37959837b P 2ac26bd2c8eb4b4fb12e08af1af156e2 [CANDIDATE]: Term 1 election: Requested vote from peers 7d6af16a92f948c98fa0a8c4d04da648 (127.1.135.194:46495), 7f8b6e024a774ca996fd7858a93c42b2 (127.1.135.195:34569)
I20260923 22:14:53.338330 16106 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "a64cc070467e497dad4bebe37959837b" candidate_uuid: "2ac26bd2c8eb4b4fb12e08af1af156e2" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7d6af16a92f948c98fa0a8c4d04da648"
I20260923 22:14:53.338379 16238 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "a64cc070467e497dad4bebe37959837b" candidate_uuid: "2ac26bd2c8eb4b4fb12e08af1af156e2" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7f8b6e024a774ca996fd7858a93c42b2"
I20260923 22:14:53.338434 16106 raft_consensus.cc:3037] T a64cc070467e497dad4bebe37959837b P 7d6af16a92f948c98fa0a8c4d04da648 [term 0 FOLLOWER]: Advancing to term 1
I20260923 22:14:53.338449 16238 raft_consensus.cc:3037] T a64cc070467e497dad4bebe37959837b P 7f8b6e024a774ca996fd7858a93c42b2 [term 0 FOLLOWER]: Advancing to term 1
I20260923 22:14:53.339248 16238 raft_consensus.cc:2454] T a64cc070467e497dad4bebe37959837b P 7f8b6e024a774ca996fd7858a93c42b2 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 2ac26bd2c8eb4b4fb12e08af1af156e2 in term 1.
I20260923 22:14:53.339248 16106 raft_consensus.cc:2454] T a64cc070467e497dad4bebe37959837b P 7d6af16a92f948c98fa0a8c4d04da648 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 2ac26bd2c8eb4b4fb12e08af1af156e2 in term 1.
I20260923 22:14:53.339437 15901 leader_election.cc:302] T a64cc070467e497dad4bebe37959837b P 2ac26bd2c8eb4b4fb12e08af1af156e2 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2ac26bd2c8eb4b4fb12e08af1af156e2, 7d6af16a92f948c98fa0a8c4d04da648; no voters:
I20260923 22:14:53.339532 16019 raft_consensus.cc:2781] T a64cc070467e497dad4bebe37959837b P 2ac26bd2c8eb4b4fb12e08af1af156e2 [term 1 FOLLOWER]: Leader election won for term 1
I20260923 22:14:53.339663 16019 raft_consensus.cc:686] T a64cc070467e497dad4bebe37959837b P 2ac26bd2c8eb4b4fb12e08af1af156e2 [term 1 LEADER]: Becoming Leader. State: Replica: 2ac26bd2c8eb4b4fb12e08af1af156e2, State: Running, Role: LEADER
I20260923 22:14:53.339713 16019 consensus_queue.cc:237] T a64cc070467e497dad4bebe37959837b P 2ac26bd2c8eb4b4fb12e08af1af156e2 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7d6af16a92f948c98fa0a8c4d04da648" member_type: VOTER last_known_addr { host: "127.1.135.194" port: 46495 } } peers { permanent_uuid: "7f8b6e024a774ca996fd7858a93c42b2" member_type: VOTER last_known_addr { host: "127.1.135.195" port: 34569 } } peers { permanent_uuid: "2ac26bd2c8eb4b4fb12e08af1af156e2" member_type: VOTER last_known_addr { host: "127.1.135.193" port: 37137 } }
I20260923 22:14:53.340423 15826 catalog_manager.cc:5495] T a64cc070467e497dad4bebe37959837b P 2ac26bd2c8eb4b4fb12e08af1af156e2 reported cstate change: term changed from 0 to 1, leader changed from <none> to 2ac26bd2c8eb4b4fb12e08af1af156e2 (127.1.135.193). New cstate: current_term: 1 leader_uuid: "2ac26bd2c8eb4b4fb12e08af1af156e2" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7d6af16a92f948c98fa0a8c4d04da648" member_type: VOTER last_known_addr { host: "127.1.135.194" port: 46495 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "7f8b6e024a774ca996fd7858a93c42b2" member_type: VOTER last_known_addr { host: "127.1.135.195" port: 34569 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "2ac26bd2c8eb4b4fb12e08af1af156e2" member_type: VOTER last_known_addr { host: "127.1.135.193" port: 37137 } health_report { overall_health: HEALTHY } } }
I20260923 22:14:53.346504 16022 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 22:14:53.344624 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55688 (local address 127.1.135.193:37137)
0923 22:14:53.344665 (+ 41us) server_negotiation.cc:206] Beginning negotiation
0923 22:14:53.344666 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0923 22:14:53.344673 (+ 7us) server_negotiation.cc:407] Connection header received
0923 22:14:53.344743 (+ 70us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 22:14:53.344744 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 22:14:53.344756 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 22:14:53.344808 (+ 52us) server_negotiation.cc:226] Negotiated authn=TOKEN
0923 22:14:53.344895 (+ 87us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 22:14:53.345300 (+ 405us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 22:14:53.345904 (+ 604us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 22:14:53.346078 (+ 174us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 22:14:53.346124 (+ 46us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 22:14:53.346231 (+ 107us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0923 22:14:53.346305 (+ 74us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0923 22:14:53.346349 (+ 44us) server_negotiation.cc:1035] Waiting for connection context
0923 22:14:53.346408 (+ 59us) server_negotiation.cc:299] Negotiation successful
0923 22:14:53.346421 (+ 13us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":25}
I20260923 22:14:53.347797 16106 raft_consensus.cc:1261] T a64cc070467e497dad4bebe37959837b P 7d6af16a92f948c98fa0a8c4d04da648 [term 1 FOLLOWER]: Refusing update from remote peer 2ac26bd2c8eb4b4fb12e08af1af156e2: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260923 22:14:53.347797 16238 raft_consensus.cc:1261] T a64cc070467e497dad4bebe37959837b P 7f8b6e024a774ca996fd7858a93c42b2 [term 1 FOLLOWER]: Refusing update from remote peer 2ac26bd2c8eb4b4fb12e08af1af156e2: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260923 22:14:53.348107 16019 consensus_queue.cc:1035] T a64cc070467e497dad4bebe37959837b P 2ac26bd2c8eb4b4fb12e08af1af156e2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "7d6af16a92f948c98fa0a8c4d04da648" member_type: VOTER last_known_addr { host: "127.1.135.194" port: 46495 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260923 22:14:53.348217 16300 consensus_queue.cc:1035] T a64cc070467e497dad4bebe37959837b P 2ac26bd2c8eb4b4fb12e08af1af156e2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "7f8b6e024a774ca996fd7858a93c42b2" member_type: VOTER last_known_addr { host: "127.1.135.195" port: 34569 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260923 22:14:53.350402 16303 mvcc.cc:205] Tried to move back new op lower bound from 7332666135950364672 to 7332666135920029696. Current Snapshot: MvccSnapshot[applied={T|T < 7332666135950364672}]
I20260923 22:14:53.350622 16304 mvcc.cc:205] Tried to move back new op lower bound from 7332666135950364672 to 7332666135920029696. Current Snapshot: MvccSnapshot[applied={T|T < 7332666135950364672}]
I20260923 22:14:53.352821 15826 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin'} at 127.0.0.1:49320:
table { table_name: "test-table" } modify_external_catalogs: true
I20260923 22:14:53.352909 15826 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin'} at 127.0.0.1:49320:
table { table_name: "test-table" } modify_external_catalogs: true
I20260923 22:14:53.353418 15826 catalog_manager.cc:5782] T 00000000000000000000000000000000 P d32493cb30cb4eda8ee9e9f9a9c35ce2: Sending DeleteTablet for 3 replicas of tablet a64cc070467e497dad4bebe37959837b
I20260923 22:14:53.353705 15947 tablet_service.cc:1514] Processing DeleteTablet for tablet a64cc070467e497dad4bebe37959837b with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-23 22:14:53 UTC) from {username='kudu', principal='kudu/127.1.135.254@KRBTEST.COM'} at 127.0.0.1:55680
I20260923 22:14:53.353685 16086 tablet_service.cc:1514] Processing DeleteTablet for tablet a64cc070467e497dad4bebe37959837b with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-23 22:14:53 UTC) from {username='kudu', principal='kudu/127.1.135.254@KRBTEST.COM'} at 127.0.0.1:48764
I20260923 22:14:53.353734 1567 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskRiQe_a/build/release/bin/kudu with pid 15864
I20260923 22:14:53.353786 16023 tablet_replica.cc:330] T a64cc070467e497dad4bebe37959837b P 2ac26bd2c8eb4b4fb12e08af1af156e2: stopping tablet replica
I20260923 22:14:53.353838 16023 raft_consensus.cc:2229] T a64cc070467e497dad4bebe37959837b P 2ac26bd2c8eb4b4fb12e08af1af156e2 [term 1 LEADER]: Raft consensus shutting down.
I20260923 22:14:53.353892 16218 tablet_service.cc:1514] Processing DeleteTablet for tablet a64cc070467e497dad4bebe37959837b with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-23 22:14:53 UTC) from {username='kudu', principal='kudu/127.1.135.254@KRBTEST.COM'} at 127.0.0.1:42552
I20260923 22:14:53.353920 16023 raft_consensus.cc:2258] T a64cc070467e497dad4bebe37959837b P 2ac26bd2c8eb4b4fb12e08af1af156e2 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260923 22:14:53.354059 16312 tablet_replica.cc:330] T a64cc070467e497dad4bebe37959837b P 7d6af16a92f948c98fa0a8c4d04da648: stopping tablet replica
I20260923 22:14:53.354321 16313 tablet_replica.cc:330] T a64cc070467e497dad4bebe37959837b P 7f8b6e024a774ca996fd7858a93c42b2: stopping tablet replica
I20260923 22:14:53.354393 16313 raft_consensus.cc:2229] T a64cc070467e497dad4bebe37959837b P 7f8b6e024a774ca996fd7858a93c42b2 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260923 22:14:53.354452 16313 raft_consensus.cc:2258] T a64cc070467e497dad4bebe37959837b P 7f8b6e024a774ca996fd7858a93c42b2 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260923 22:14:53.354573 16312 raft_consensus.cc:2229] T a64cc070467e497dad4bebe37959837b P 7d6af16a92f948c98fa0a8c4d04da648 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260923 22:14:53.354668 16312 raft_consensus.cc:2258] T a64cc070467e497dad4bebe37959837b P 7d6af16a92f948c98fa0a8c4d04da648 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260923 22:14:53.354887 16313 ts_tablet_manager.cc:1905] T a64cc070467e497dad4bebe37959837b P 7f8b6e024a774ca996fd7858a93c42b2: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260923 22:14:53.355026 16312 ts_tablet_manager.cc:1905] T a64cc070467e497dad4bebe37959837b P 7d6af16a92f948c98fa0a8c4d04da648: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260923 22:14:53.356424 16313 ts_tablet_manager.cc:1918] T a64cc070467e497dad4bebe37959837b P 7f8b6e024a774ca996fd7858a93c42b2: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260923 22:14:53.356516 16313 log.cc:1196] T a64cc070467e497dad4bebe37959837b P 7f8b6e024a774ca996fd7858a93c42b2: Deleting WAL directory at /tmp/dist-test-taskRiQe_a/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790201539513534-1567-0/minicluster-data/ts-2/wal/wals/a64cc070467e497dad4bebe37959837b
I20260923 22:14:53.356631 16312 ts_tablet_manager.cc:1918] T a64cc070467e497dad4bebe37959837b P 7d6af16a92f948c98fa0a8c4d04da648: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260923 22:14:53.356704 16312 log.cc:1196] T a64cc070467e497dad4bebe37959837b P 7d6af16a92f948c98fa0a8c4d04da648: Deleting WAL directory at /tmp/dist-test-taskRiQe_a/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790201539513534-1567-0/minicluster-data/ts-1/wal/wals/a64cc070467e497dad4bebe37959837b
I20260923 22:14:53.356827 16313 ts_tablet_manager.cc:1939] T a64cc070467e497dad4bebe37959837b P 7f8b6e024a774ca996fd7858a93c42b2: Deleting consensus metadata
I20260923 22:14:53.357002 16312 ts_tablet_manager.cc:1939] T a64cc070467e497dad4bebe37959837b P 7d6af16a92f948c98fa0a8c4d04da648: Deleting consensus metadata
I20260923 22:14:53.357210 15811 catalog_manager.cc:4841] TS 7f8b6e024a774ca996fd7858a93c42b2 (127.1.135.195:34569): tablet a64cc070467e497dad4bebe37959837b (table test-table [id=db1628a18a854699818f1a42e33e4c0e]) successfully deleted
I20260923 22:14:53.357328 15811 catalog_manager.cc:4841] TS 7d6af16a92f948c98fa0a8c4d04da648 (127.1.135.194:46495): tablet a64cc070467e497dad4bebe37959837b (table test-table [id=db1628a18a854699818f1a42e33e4c0e]) successfully deleted
W20260923 22:14:53.359607 15812 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.1.135.193:37137 (error 108)
I20260923 22:14:53.359848 1567 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskRiQe_a/build/release/bin/kudu with pid 16020
I20260923 22:14:53.360015 16021 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 22:14:53.359912 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.135.193:37137 (local address 127.0.0.1:55692)
0923 22:14:53.359955 (+ 43us) negotiation.cc:107] Waiting for socket to connect
0923 22:14:53.359983 (+ 28us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.135.193:37137: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":26}
W20260923 22:14:53.360498 15812 catalog_manager.cc:4584] TS 2ac26bd2c8eb4b4fb12e08af1af156e2 (127.1.135.193:37137): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet a64cc070467e497dad4bebe37959837b: Network error: Client connection negotiation failed: client connection to 127.1.135.193:37137: connect: Connection refused (error 111)
I20260923 22:14:53.365435 1567 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskRiQe_a/build/release/bin/kudu with pid 16155
I20260923 22:14:53.369711 1567 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskRiQe_a/build/release/bin/kudu with pid 15793
I20260923 22:14:53.374877 1567 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskRiQe_a/build/release/bin/kudu
/tmp/dist-test-taskRiQe_a/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskRiQe_a/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790201539513534-1567-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskRiQe_a/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790201539513534-1567-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskRiQe_a/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790201539513534-1567-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskRiQe_a/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790201539513534-1567-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.1.135.254:37515
--webserver_interface=127.1.135.254
--webserver_port=45941
--keytab_file=/tmp/dist-test-taskRiQe_a/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790201539513534-1567-0/krb5kdc/kudu_127.1.135.254.keytab
--principal=kudu/127.1.135.254
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.1.135.212:38425
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.1.135.254:37515
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskRiQe_a/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790201539513534-1567-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskRiQe_a/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790201539513534-1567-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskRiQe_a/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790201539513534-1567-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260923 22:14:53.452126 16315 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260923 22:14:53.452303 16315 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260923 22:14:53.452323 16315 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260923 22:14:53.453410 16315 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260923 22:14:53.453449 16315 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260923 22:14:53.453462 16315 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260923 22:14:53.453508 16315 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260923 22:14:53.453526 16315 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260923 22:14:53.454885 16315 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.1.135.212:38425
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskRiQe_a/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790201539513534-1567-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskRiQe_a/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790201539513534-1567-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.1.135.254:37515
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskRiQe_a/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790201539513534-1567-0/krb5kdc/kudu_127.1.135.254.keytab
--principal=kudu/127.1.135.254
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.1.135.254:37515
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskRiQe_a/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790201539513534-1567-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.1.135.254
--webserver_port=45941
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.16315
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskRiQe_a/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790201539513534-1567-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 23 Sep 2026 20:53:37 UTC on ccycloud.quasar-rkviyx.root.comops.site
build id 4668083
I20260923 22:14:53.455084 16315 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260923 22:14:53.455281 16315 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260923 22:14:53.459671 16320 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260923 22:14:53.459679 16321 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260923 22:14:53.459833 16315 server_base.cc:1034] running on GCE node
W20260923 22:14:53.459723 16323 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260923 22:14:53.460049 16315 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260923 22:14:53.460327 16315 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260923 22:14:53.461460 16315 hybrid_clock.cc:648] HybridClock initialized: now 1790201693461447 us; error 27 us; skew 500 ppm
Sep 23 22:14:53 dist-test-slave-s2fj krb5kdc[15248](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790201693, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.135.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260923 22:14:53.463685 16315 init.cc:377] Logged in from keytab as kudu/127.1.135.254@KRBTEST.COM (short username kudu)
I20260923 22:14:53.464110 16315 webserver.cc:460] Webserver started at http://127.1.135.254:45941/ using document root <none> and password file <none>
I20260923 22:14:53.464339 16315 fs_manager.cc:359] Metadata directory not provided
I20260923 22:14:53.464401 16315 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260923 22:14:53.465551 16315 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.001s sys 0.000s
I20260923 22:14:53.466128 16330 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260923 22:14:53.466275 16315 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.000s
I20260923 22:14:53.466364 16315 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskRiQe_a/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790201539513534-1567-0/minicluster-data/master-0/data,/tmp/dist-test-taskRiQe_a/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790201539513534-1567-0/minicluster-data/master-0/wal
uuid: "d32493cb30cb4eda8ee9e9f9a9c35ce2"
format_stamp: "Formatted at 2026-09-23 22:14:52 on dist-test-slave-s2fj"
server_key: "8358f9839aaaf6002d5bb63dbf35b23e"
server_key_iv: "887f4f7c2f04783e830210e387213916"
server_key_version: "encryptionkey@0"
I20260923 22:14:53.466668 16315 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskRiQe_a/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790201539513534-1567-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskRiQe_a/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790201539513534-1567-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskRiQe_a/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790201539513534-1567-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260923 22:14:53.494728 16315 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260923 22:14:53.496342 16315 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260923 22:14:53.496487 16315 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260923 22:14:53.499957 16315 rpc_server.cc:307] RPC server started. Bound to: 127.1.135.254:37515
I20260923 22:14:53.500000 16382 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.1.135.254:37515 every 8 connection(s)
I20260923 22:14:53.500350 16315 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskRiQe_a/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790201539513534-1567-0/minicluster-data/master-0/data/info.pb
I20260923 22:14:53.501814 16383 sys_catalog.cc:263] Verifying existing consensus state
I20260923 22:14:53.502398 16383 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P d32493cb30cb4eda8ee9e9f9a9c35ce2: Bootstrap starting.
I20260923 22:14:53.509620 1567 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskRiQe_a/build/release/bin/kudu as pid 16315
I20260923 22:14:53.509879 1567 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskRiQe_a/build/release/bin/kudu
/tmp/dist-test-taskRiQe_a/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskRiQe_a/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790201539513534-1567-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskRiQe_a/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790201539513534-1567-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskRiQe_a/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790201539513534-1567-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskRiQe_a/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790201539513534-1567-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.1.135.193:37137
--local_ip_for_outbound_sockets=127.1.135.193
--tserver_master_addrs=127.1.135.254:37515
--webserver_port=34527
--webserver_interface=127.1.135.193
--keytab_file=/tmp/dist-test-taskRiQe_a/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790201539513534-1567-0/krb5kdc/kudu_127.1.135.193.keytab
--principal=kudu/127.1.135.193
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.1.135.212:38425
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskRiQe_a/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790201539513534-1567-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskRiQe_a/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790201539513534-1567-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskRiQe_a/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790201539513534-1567-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260923 22:14:53.511497 16383 log.cc:824] T 00000000000000000000000000000000 P d32493cb30cb4eda8ee9e9f9a9c35ce2: Log is configured to *not* fsync() on all Append() calls
I20260923 22:14:53.513541 16383 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P d32493cb30cb4eda8ee9e9f9a9c35ce2: Bootstrap replayed 1/1 log segments. Stats: ops{read=14 overwritten=0 applied=14 ignored=0} inserts{seen=7 ignored=0} mutations{seen=9 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260923 22:14:53.513824 16383 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P d32493cb30cb4eda8ee9e9f9a9c35ce2: Bootstrap complete.
I20260923 22:14:53.515877 16383 raft_consensus.cc:348] T 00000000000000000000000000000000 P d32493cb30cb4eda8ee9e9f9a9c35ce2 [term 2 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d32493cb30cb4eda8ee9e9f9a9c35ce2" member_type: VOTER last_known_addr { host: "127.1.135.254" port: 37515 } }
I20260923 22:14:53.516183 16383 raft_consensus.cc:729] T 00000000000000000000000000000000 P d32493cb30cb4eda8ee9e9f9a9c35ce2 [term 2 FOLLOWER]: Becoming Follower/Learner. State: Replica: d32493cb30cb4eda8ee9e9f9a9c35ce2, State: Initialized, Role: FOLLOWER
I20260923 22:14:53.516376 16383 consensus_queue.cc:260] T 00000000000000000000000000000000 P d32493cb30cb4eda8ee9e9f9a9c35ce2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 14, Last appended: 2.14, Last appended by leader: 14, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d32493cb30cb4eda8ee9e9f9a9c35ce2" member_type: VOTER last_known_addr { host: "127.1.135.254" port: 37515 } }
I20260923 22:14:53.516450 16383 raft_consensus.cc:388] T 00000000000000000000000000000000 P d32493cb30cb4eda8ee9e9f9a9c35ce2 [term 2 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260923 22:14:53.516500 16383 raft_consensus.cc:482] T 00000000000000000000000000000000 P d32493cb30cb4eda8ee9e9f9a9c35ce2 [term 2 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260923 22:14:53.516553 16383 raft_consensus.cc:3037] T 00000000000000000000000000000000 P d32493cb30cb4eda8ee9e9f9a9c35ce2 [term 2 FOLLOWER]: Advancing to term 3
I20260923 22:14:53.517460 16383 raft_consensus.cc:504] T 00000000000000000000000000000000 P d32493cb30cb4eda8ee9e9f9a9c35ce2 [term 3 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d32493cb30cb4eda8ee9e9f9a9c35ce2" member_type: VOTER last_known_addr { host: "127.1.135.254" port: 37515 } }
I20260923 22:14:53.517582 16383 leader_election.cc:302] T 00000000000000000000000000000000 P d32493cb30cb4eda8ee9e9f9a9c35ce2 [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: d32493cb30cb4eda8ee9e9f9a9c35ce2; no voters:
I20260923 22:14:53.517742 16383 leader_election.cc:288] T 00000000000000000000000000000000 P d32493cb30cb4eda8ee9e9f9a9c35ce2 [CANDIDATE]: Term 3 election: Requested vote from peers
I20260923 22:14:53.517789 16387 raft_consensus.cc:2781] T 00000000000000000000000000000000 P d32493cb30cb4eda8ee9e9f9a9c35ce2 [term 3 FOLLOWER]: Leader election won for term 3
I20260923 22:14:53.517941 16387 raft_consensus.cc:686] T 00000000000000000000000000000000 P d32493cb30cb4eda8ee9e9f9a9c35ce2 [term 3 LEADER]: Becoming Leader. State: Replica: d32493cb30cb4eda8ee9e9f9a9c35ce2, State: Running, Role: LEADER
I20260923 22:14:53.517975 16383 sys_catalog.cc:564] T 00000000000000000000000000000000 P d32493cb30cb4eda8ee9e9f9a9c35ce2 [sys.catalog]: configured and running, proceeding with master startup.
I20260923 22:14:53.518047 16387 consensus_queue.cc:237] T 00000000000000000000000000000000 P d32493cb30cb4eda8ee9e9f9a9c35ce2 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 14, Committed index: 14, Last appended: 2.14, Last appended by leader: 14, Current term: 3, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d32493cb30cb4eda8ee9e9f9a9c35ce2" member_type: VOTER last_known_addr { host: "127.1.135.254" port: 37515 } }
I20260923 22:14:53.518318 16387 sys_catalog.cc:455] T 00000000000000000000000000000000 P d32493cb30cb4eda8ee9e9f9a9c35ce2 [sys.catalog]: SysCatalogTable state changed. Reason: New leader d32493cb30cb4eda8ee9e9f9a9c35ce2. Latest consensus state: current_term: 3 leader_uuid: "d32493cb30cb4eda8ee9e9f9a9c35ce2" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d32493cb30cb4eda8ee9e9f9a9c35ce2" member_type: VOTER last_known_addr { host: "127.1.135.254" port: 37515 } } }
I20260923 22:14:53.518438 16387 sys_catalog.cc:458] T 00000000000000000000000000000000 P d32493cb30cb4eda8ee9e9f9a9c35ce2 [sys.catalog]: This master's current role is: LEADER
I20260923 22:14:53.518319 16388 sys_catalog.cc:455] T 00000000000000000000000000000000 P d32493cb30cb4eda8ee9e9f9a9c35ce2 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 3 leader_uuid: "d32493cb30cb4eda8ee9e9f9a9c35ce2" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d32493cb30cb4eda8ee9e9f9a9c35ce2" member_type: VOTER last_known_addr { host: "127.1.135.254" port: 37515 } } }
I20260923 22:14:53.518620 16388 sys_catalog.cc:458] T 00000000000000000000000000000000 P d32493cb30cb4eda8ee9e9f9a9c35ce2 [sys.catalog]: This master's current role is: LEADER
I20260923 22:14:53.518640 16394 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260923 22:14:53.519466 16394 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260923 22:14:53.519572 16394 catalog_manager.cc:1259] Loaded cluster ID: 6fbcafe1f67a46a2a305ad3d03ea6e60
I20260923 22:14:53.519608 16394 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
F20260923 22:14:53.519814 16394 catalog_manager.cc:1466] Initializing Kudu internal certificate authority failed: Runtime error: unable to load data from memory: error:0D0680A8:asn1 encoding routines:ASN1_CHECK_TLEN:wrong tag:tasn_dec.c:1220 error:0D06C03A:asn1 encoding routines:ASN1_D2I_EX_PRIMITIVE:nested asn1 error:tasn_dec.c:788 error:0D08303A:asn1 encoding routines:ASN1_TEMPLATE_NOEXP_D2I:nested asn1 error:tasn_dec.c:720:Field=version, Type=RSA error:04093004:rsa routines:OLD_RSA_PRIV_DECODE:RSA lib:rsa_ameth.c:121 error:0D0680A8:asn1 encoding routines:ASN1_CHECK_TLEN:wrong tag:tasn_dec.c:1220 error:0D06C03A:asn1 encoding routines:ASN1_D2I_EX_PRIMITIVE:nested asn1 error:tasn_dec.c:788 error:0D08303A:asn1 encoding routines:ASN1_TEMPLATE_NOEXP_D2I:nested asn1 error:tasn_dec.c:720:Field=version, Type=PKCS8_PRIV_KEY_INFO
*** Check failure stack trace: ***
@ 0x7f34912e19ad google::LogMessage::Fail() at ??:0
@ 0x7f34912e488a google::LogMessage::SendToLog() at ??:0
@ 0x7f34912e14de google::LogMessage::Flush() at ??:0
@ 0x7f34912e2a19 google::LogMessageFatal::~LogMessageFatal() at ??:0
@ 0x7f34992060f1 _ZZN4kudu6master14CatalogManager24PrepareForLeadershipTaskEvENKUlSt8functionIFNS_6StatusEvEERKNS_9consensus13RaftConsensusElPKcE_clES5_S9_lSB_.isra.591 at ??:0
@ 0x7f349920f8b9 kudu::master::CatalogManager::PrepareForLeadershipTask() at ??:0
@ 0x7f34931af0d3 kudu::ThreadPool::DispatchThread() at ??:0
@ 0x7f34931a4954 kudu::Thread::SuperviseThread() at ??:0
@ 0x7f34940086db start_thread at ??:0
@ 0x7f348fddf61f clone at ??:0
W20260923 22:14:53.598304 16385 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260923 22:14:53.598481 16385 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260923 22:14:53.598503 16385 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260923 22:14:53.599624 16385 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260923 22:14:53.599664 16385 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260923 22:14:53.599689 16385 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.1.135.193
I20260923 22:14:53.601153 16385 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.1.135.212:38425
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskRiQe_a/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790201539513534-1567-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskRiQe_a/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790201539513534-1567-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskRiQe_a/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790201539513534-1567-0/krb5kdc/kudu_127.1.135.193.keytab
--principal=kudu/127.1.135.193
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.1.135.193:37137
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskRiQe_a/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790201539513534-1567-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.1.135.193
--webserver_port=34527
--tserver_master_addrs=127.1.135.254:37515
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.16385
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.1.135.193
--log_dir=/tmp/dist-test-taskRiQe_a/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790201539513534-1567-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 23 Sep 2026 20:53:37 UTC on ccycloud.quasar-rkviyx.root.comops.site
build id 4668083
I20260923 22:14:53.601367 16385 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260923 22:14:53.601609 16385 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260923 22:14:53.607671 16420 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260923 22:14:53.607766 16385 server_base.cc:1034] running on GCE node
W20260923 22:14:53.607697 16422 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260923 22:14:53.607898 16419 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260923 22:14:53.608147 16385 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260923 22:14:53.608410 16385 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260923 22:14:53.609566 16385 hybrid_clock.cc:648] HybridClock initialized: now 1790201693609553 us; error 34 us; skew 500 ppm
Sep 23 22:14:53 dist-test-slave-s2fj krb5kdc[15248](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790201693, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.135.193@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260923 22:14:53.612169 16385 init.cc:377] Logged in from keytab as kudu/127.1.135.193@KRBTEST.COM (short username kudu)
I20260923 22:14:53.612586 16385 webserver.cc:460] Webserver started at http://127.1.135.193:34527/ using document root <none> and password file <none>
I20260923 22:14:53.612789 16385 fs_manager.cc:359] Metadata directory not provided
I20260923 22:14:53.612850 16385 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260923 22:14:53.614132 16385 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.001s sys 0.000s
I20260923 22:14:53.614841 16429 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260923 22:14:53.615020 16385 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260923 22:14:53.615109 16385 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskRiQe_a/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790201539513534-1567-0/minicluster-data/ts-0/data,/tmp/dist-test-taskRiQe_a/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790201539513534-1567-0/minicluster-data/ts-0/wal
uuid: "2ac26bd2c8eb4b4fb12e08af1af156e2"
format_stamp: "Formatted at 2026-09-23 22:14:52 on dist-test-slave-s2fj"
server_key: "7ce9468d20e1b5fc5cba598c1a59b91d"
server_key_iv: "af2c59483b907adc33e1ce686719613d"
server_key_version: "encryptionkey@0"
I20260923 22:14:53.615428 16385 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskRiQe_a/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790201539513534-1567-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskRiQe_a/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790201539513534-1567-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskRiQe_a/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790201539513534-1567-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260923 22:14:53.624562 16385 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260923 22:14:53.626778 16385 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260923 22:14:53.626907 16385 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260923 22:14:53.627130 16385 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260923 22:14:53.627661 16436 ts_tablet_manager.cc:536] Loading tablet metadata (0/1 complete)
I20260923 22:14:53.629563 16385 ts_tablet_manager.cc:579] Loaded tablet metadata (1 total tablets, 1 live tablets)
I20260923 22:14:53.629622 16385 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.002s user 0.000s sys 0.000s
I20260923 22:14:53.629662 16385 ts_tablet_manager.cc:594] Registering tablets (0/1 complete)
I20260923 22:14:53.630229 16385 ts_tablet_manager.cc:610] Registered 1 tablets
I20260923 22:14:53.630285 16385 ts_tablet_manager.cc:589] Time spent register tablets: real 0.001s user 0.000s sys 0.000s
I20260923 22:14:53.630398 16436 tablet_bootstrap.cc:492] T a64cc070467e497dad4bebe37959837b P 2ac26bd2c8eb4b4fb12e08af1af156e2: Bootstrap starting.
I20260923 22:14:53.636644 16385 rpc_server.cc:307] RPC server started. Bound to: 127.1.135.193:37137
I20260923 22:14:53.637153 16385 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskRiQe_a/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790201539513534-1567-0/minicluster-data/ts-0/data/info.pb
I20260923 22:14:53.637367 16543 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.1.135.193:37137 every 8 connection(s)
I20260923 22:14:53.641711 16436 log.cc:824] T a64cc070467e497dad4bebe37959837b P 2ac26bd2c8eb4b4fb12e08af1af156e2: Log is configured to *not* fsync() on all Append() calls
I20260923 22:14:53.644531 16436 tablet_bootstrap.cc:492] T a64cc070467e497dad4bebe37959837b P 2ac26bd2c8eb4b4fb12e08af1af156e2: Bootstrap replayed 1/1 log segments. Stats: ops{read=2 overwritten=0 applied=2 ignored=0} inserts{seen=1 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260923 22:14:53.644810 16436 tablet_bootstrap.cc:492] T a64cc070467e497dad4bebe37959837b P 2ac26bd2c8eb4b4fb12e08af1af156e2: Bootstrap complete.
I20260923 22:14:53.644819 1567 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskRiQe_a/build/release/bin/kudu as pid 16385
I20260923 22:14:53.645100 1567 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskRiQe_a/build/release/bin/kudu
/tmp/dist-test-taskRiQe_a/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskRiQe_a/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790201539513534-1567-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskRiQe_a/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790201539513534-1567-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskRiQe_a/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790201539513534-1567-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskRiQe_a/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790201539513534-1567-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.1.135.194:46495
--local_ip_for_outbound_sockets=127.1.135.194
--tserver_master_addrs=127.1.135.254:37515
--webserver_port=36489
--webserver_interface=127.1.135.194
--keytab_file=/tmp/dist-test-taskRiQe_a/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790201539513534-1567-0/krb5kdc/kudu_127.1.135.194.keytab
--principal=kudu/127.1.135.194
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.1.135.212:38425
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskRiQe_a/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790201539513534-1567-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskRiQe_a/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790201539513534-1567-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskRiQe_a/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790201539513534-1567-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260923 22:14:53.646003 16436 ts_tablet_manager.cc:1397] T a64cc070467e497dad4bebe37959837b P 2ac26bd2c8eb4b4fb12e08af1af156e2: Time spent bootstrapping tablet: real 0.016s user 0.007s sys 0.007s
I20260923 22:14:53.647429 16436 raft_consensus.cc:348] T a64cc070467e497dad4bebe37959837b P 2ac26bd2c8eb4b4fb12e08af1af156e2 [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7d6af16a92f948c98fa0a8c4d04da648" member_type: VOTER last_known_addr { host: "127.1.135.194" port: 46495 } } peers { permanent_uuid: "7f8b6e024a774ca996fd7858a93c42b2" member_type: VOTER last_known_addr { host: "127.1.135.195" port: 34569 } } peers { permanent_uuid: "2ac26bd2c8eb4b4fb12e08af1af156e2" member_type: VOTER last_known_addr { host: "127.1.135.193" port: 37137 } }
I20260923 22:14:53.647763 16436 raft_consensus.cc:729] T a64cc070467e497dad4bebe37959837b P 2ac26bd2c8eb4b4fb12e08af1af156e2 [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: 2ac26bd2c8eb4b4fb12e08af1af156e2, State: Initialized, Role: FOLLOWER
I20260923 22:14:53.647934 16436 consensus_queue.cc:260] T a64cc070467e497dad4bebe37959837b P 2ac26bd2c8eb4b4fb12e08af1af156e2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 2, Last appended: 1.2, Last appended by leader: 2, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7d6af16a92f948c98fa0a8c4d04da648" member_type: VOTER last_known_addr { host: "127.1.135.194" port: 46495 } } peers { permanent_uuid: "7f8b6e024a774ca996fd7858a93c42b2" member_type: VOTER last_known_addr { host: "127.1.135.195" port: 34569 } } peers { permanent_uuid: "2ac26bd2c8eb4b4fb12e08af1af156e2" member_type: VOTER last_known_addr { host: "127.1.135.193" port: 37137 } }
I20260923 22:14:53.648222 16436 ts_tablet_manager.cc:1428] T a64cc070467e497dad4bebe37959837b P 2ac26bd2c8eb4b4fb12e08af1af156e2: Time spent starting tablet: real 0.002s user 0.003s sys 0.001s
W20260923 22:14:53.754717 16546 negotiation.cc:337] Failed RPC negotiation. Trace:
0923 22:14:53.640554 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.135.254:37515 (local address 127.1.135.193:47357)
0923 22:14:53.640733 (+ 179us) negotiation.cc:107] Waiting for socket to connect
0923 22:14:53.640773 (+ 40us) client_negotiation.cc:174] Beginning negotiation
0923 22:14:53.641188 (+ 415us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0923 22:14:53.754485 (+113297us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.135.254:37515: BlockingRecv error: recv error from unknown peer: Transport endpoint is not connected (error 107)
Metrics: {"client-negotiator.queue_time_us":139,"thread_start_us":110,"threads_started":1}
W20260923 22:14:53.755056 16544 heartbeater.cc:644] Failed to heartbeat to 127.1.135.254:37515 (0 consecutive failures): Network error: Failed to ping master at 127.1.135.254:37515: Client connection negotiation failed: client connection to 127.1.135.254:37515: BlockingRecv error: recv error from unknown peer: Transport endpoint is not connected (error 107)
I20260923 22:14:53.755419 16546 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 22:14:53.755296 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.135.254:37515 (local address 127.1.135.193:50441)
0923 22:14:53.755352 (+ 56us) negotiation.cc:107] Waiting for socket to connect
0923 22:14:53.755389 (+ 37us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.135.254:37515: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":36}
I20260923 22:14:53.755802 16546 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 22:14:53.755723 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.135.254:37515 (local address 127.1.135.193:58491)
0923 22:14:53.755765 (+ 42us) negotiation.cc:107] Waiting for socket to connect
0923 22:14:53.755787 (+ 22us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.135.254:37515: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":30}
W20260923 22:14:53.755928 16544 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
W20260923 22:14:53.763736 16548 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260923 22:14:53.763909 16548 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260923 22:14:53.763957 16548 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260923 22:14:53.765118 16548 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260923 22:14:53.765158 16548 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260923 22:14:53.765210 16548 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.1.135.194
I20260923 22:14:53.766750 16548 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.1.135.212:38425
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskRiQe_a/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790201539513534-1567-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskRiQe_a/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790201539513534-1567-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskRiQe_a/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790201539513534-1567-0/krb5kdc/kudu_127.1.135.194.keytab
--principal=kudu/127.1.135.194
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.1.135.194:46495
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskRiQe_a/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790201539513534-1567-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.1.135.194
--webserver_port=36489
--tserver_master_addrs=127.1.135.254:37515
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.16548
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.1.135.194
--log_dir=/tmp/dist-test-taskRiQe_a/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790201539513534-1567-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 23 Sep 2026 20:53:37 UTC on ccycloud.quasar-rkviyx.root.comops.site
build id 4668083
I20260923 22:14:53.766983 16548 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260923 22:14:53.767207 16548 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260923 22:14:53.771975 16554 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260923 22:14:53.772104 16557 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260923 22:14:53.772010 16555 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260923 22:14:53.772029 16548 server_base.cc:1034] running on GCE node
I20260923 22:14:53.772337 16548 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260923 22:14:53.772584 16548 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260923 22:14:53.773731 16548 hybrid_clock.cc:648] HybridClock initialized: now 1790201693773717 us; error 24 us; skew 500 ppm
Sep 23 22:14:53 dist-test-slave-s2fj krb5kdc[15248](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790201693, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.135.194@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260923 22:14:53.775961 16548 init.cc:377] Logged in from keytab as kudu/127.1.135.194@KRBTEST.COM (short username kudu)
I20260923 22:14:53.776348 16548 webserver.cc:460] Webserver started at http://127.1.135.194:36489/ using document root <none> and password file <none>
I20260923 22:14:53.776608 16548 fs_manager.cc:359] Metadata directory not provided
I20260923 22:14:53.776676 16548 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260923 22:14:53.777892 16548 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.000s sys 0.001s
I20260923 22:14:53.778443 16564 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260923 22:14:53.778591 16548 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.001s
I20260923 22:14:53.778683 16548 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskRiQe_a/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790201539513534-1567-0/minicluster-data/ts-1/data,/tmp/dist-test-taskRiQe_a/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790201539513534-1567-0/minicluster-data/ts-1/wal
uuid: "7d6af16a92f948c98fa0a8c4d04da648"
format_stamp: "Formatted at 2026-09-23 22:14:52 on dist-test-slave-s2fj"
server_key: "5ed27962f37132d31849db12e425c61f"
server_key_iv: "ce92088ae7b1e9838e4e018406906389"
server_key_version: "encryptionkey@0"
I20260923 22:14:53.778986 16548 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskRiQe_a/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790201539513534-1567-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskRiQe_a/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790201539513534-1567-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskRiQe_a/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790201539513534-1567-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260923 22:14:53.791790 16548 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260923 22:14:53.793421 16548 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260923 22:14:53.793610 16548 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260923 22:14:53.793812 16548 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260923 22:14:53.794071 16548 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260923 22:14:53.794135 16548 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260923 22:14:53.794166 16548 ts_tablet_manager.cc:610] Registered 0 tablets
I20260923 22:14:53.794190 16548 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260923 22:14:53.799503 16548 rpc_server.cc:307] RPC server started. Bound to: 127.1.135.194:46495
I20260923 22:14:53.799603 16677 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.1.135.194:46495 every 8 connection(s)
I20260923 22:14:53.799896 16548 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskRiQe_a/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790201539513534-1567-0/minicluster-data/ts-1/data/info.pb
I20260923 22:14:53.800948 16680 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 22:14:53.800466 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.135.254:37515 (local address 127.1.135.194:52903)
0923 22:14:53.800679 (+ 213us) negotiation.cc:107] Waiting for socket to connect
0923 22:14:53.800769 (+ 90us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.135.254:37515: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":168,"thread_start_us":71,"threads_started":1}
W20260923 22:14:53.801337 16678 heartbeater.cc:644] Failed to heartbeat to 127.1.135.254:37515 (0 consecutive failures): Network error: Failed to ping master at 127.1.135.254:37515: Client connection negotiation failed: client connection to 127.1.135.254:37515: connect: Connection refused (error 111)
I20260923 22:14:53.801584 16680 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 22:14:53.801491 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.135.254:37515 (local address 127.1.135.194:48879)
0923 22:14:53.801541 (+ 50us) negotiation.cc:107] Waiting for socket to connect
0923 22:14:53.801566 (+ 25us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.135.254:37515: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":36}
I20260923 22:14:53.801823 16680 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 22:14:53.801752 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.135.254:37515 (local address 127.1.135.194:35561)
0923 22:14:53.801790 (+ 38us) negotiation.cc:107] Waiting for socket to connect
0923 22:14:53.801810 (+ 20us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.135.254:37515: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":27}
W20260923 22:14:53.801941 16678 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
I20260923 22:14:53.802281 1567 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskRiQe_a/build/release/bin/kudu as pid 16548
I20260923 22:14:53.802469 1567 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskRiQe_a/build/release/bin/kudu
/tmp/dist-test-taskRiQe_a/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskRiQe_a/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790201539513534-1567-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskRiQe_a/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790201539513534-1567-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskRiQe_a/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790201539513534-1567-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskRiQe_a/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790201539513534-1567-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.1.135.195:34569
--local_ip_for_outbound_sockets=127.1.135.195
--tserver_master_addrs=127.1.135.254:37515
--webserver_port=37309
--webserver_interface=127.1.135.195
--keytab_file=/tmp/dist-test-taskRiQe_a/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790201539513534-1567-0/krb5kdc/kudu_127.1.135.195.keytab
--principal=kudu/127.1.135.195
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.1.135.212:38425
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskRiQe_a/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790201539513534-1567-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskRiQe_a/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790201539513534-1567-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskRiQe_a/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790201539513534-1567-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260923 22:14:53.879323 16681 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260923 22:14:53.879483 16681 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260923 22:14:53.879504 16681 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260923 22:14:53.880669 16681 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260923 22:14:53.880707 16681 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260923 22:14:53.880734 16681 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.1.135.195
I20260923 22:14:53.882125 16681 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.1.135.212:38425
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskRiQe_a/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790201539513534-1567-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskRiQe_a/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790201539513534-1567-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskRiQe_a/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790201539513534-1567-0/krb5kdc/kudu_127.1.135.195.keytab
--principal=kudu/127.1.135.195
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.1.135.195:34569
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskRiQe_a/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790201539513534-1567-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.1.135.195
--webserver_port=37309
--tserver_master_addrs=127.1.135.254:37515
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.16681
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.1.135.195
--log_dir=/tmp/dist-test-taskRiQe_a/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790201539513534-1567-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 23 Sep 2026 20:53:37 UTC on ccycloud.quasar-rkviyx.root.comops.site
build id 4668083
I20260923 22:14:53.882278 16681 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260923 22:14:53.882426 16681 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260923 22:14:53.887072 16689 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260923 22:14:53.887058 16686 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260923 22:14:53.887058 16687 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260923 22:14:53.887455 16681 server_base.cc:1034] running on GCE node
I20260923 22:14:53.887673 16681 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260923 22:14:53.887880 16681 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260923 22:14:53.889037 16681 hybrid_clock.cc:648] HybridClock initialized: now 1790201693888995 us; error 52 us; skew 500 ppm
Sep 23 22:14:53 dist-test-slave-s2fj krb5kdc[15248](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790201693, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.135.195@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260923 22:14:53.891287 16681 init.cc:377] Logged in from keytab as kudu/127.1.135.195@KRBTEST.COM (short username kudu)
I20260923 22:14:53.891741 16681 webserver.cc:460] Webserver started at http://127.1.135.195:37309/ using document root <none> and password file <none>
I20260923 22:14:53.891947 16681 fs_manager.cc:359] Metadata directory not provided
I20260923 22:14:53.892011 16681 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260923 22:14:53.893194 16681 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.000s sys 0.001s
I20260923 22:14:53.893769 16696 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260923 22:14:53.893934 16681 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.001s
I20260923 22:14:53.894021 16681 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskRiQe_a/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790201539513534-1567-0/minicluster-data/ts-2/data,/tmp/dist-test-taskRiQe_a/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790201539513534-1567-0/minicluster-data/ts-2/wal
uuid: "7f8b6e024a774ca996fd7858a93c42b2"
format_stamp: "Formatted at 2026-09-23 22:14:52 on dist-test-slave-s2fj"
server_key: "93f7a16cb6e8af959504c1302d41db8e"
server_key_iv: "8b41b00d37b8db1e35715c0c63d26e2d"
server_key_version: "encryptionkey@0"
I20260923 22:14:53.894336 16681 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskRiQe_a/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790201539513534-1567-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskRiQe_a/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790201539513534-1567-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskRiQe_a/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790201539513534-1567-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260923 22:14:53.902631 16681 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260923 22:14:53.904392 16681 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260923 22:14:53.904537 16681 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260923 22:14:53.904732 16681 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260923 22:14:53.905014 16681 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260923 22:14:53.905076 16681 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260923 22:14:53.905112 16681 ts_tablet_manager.cc:610] Registered 0 tablets
I20260923 22:14:53.905140 16681 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260923 22:14:53.910655 16681 rpc_server.cc:307] RPC server started. Bound to: 127.1.135.195:34569
I20260923 22:14:53.910706 16809 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.1.135.195:34569 every 8 connection(s)
I20260923 22:14:53.910997 16681 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskRiQe_a/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790201539513534-1567-0/minicluster-data/ts-2/data/info.pb
I20260923 22:14:53.911926 16812 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 22:14:53.911601 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.135.254:37515 (local address 127.1.135.195:40597)
0923 22:14:53.911746 (+ 145us) negotiation.cc:107] Waiting for socket to connect
0923 22:14:53.911806 (+ 60us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.135.254:37515: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":113,"thread_start_us":77,"threads_started":1}
W20260923 22:14:53.912292 16810 heartbeater.cc:644] Failed to heartbeat to 127.1.135.254:37515 (0 consecutive failures): Network error: Failed to ping master at 127.1.135.254:37515: Client connection negotiation failed: client connection to 127.1.135.254:37515: connect: Connection refused (error 111)
I20260923 22:14:53.912511 16812 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 22:14:53.912451 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.135.254:37515 (local address 127.1.135.195:51537)
0923 22:14:53.912475 (+ 24us) negotiation.cc:107] Waiting for socket to connect
0923 22:14:53.912497 (+ 22us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.135.254:37515: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":14}
I20260923 22:14:53.912761 16812 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 22:14:53.912711 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.135.254:37515 (local address 127.1.135.195:51387)
0923 22:14:53.912735 (+ 24us) negotiation.cc:107] Waiting for socket to connect
0923 22:14:53.912751 (+ 16us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.135.254:37515: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":17}
W20260923 22:14:53.912858 16810 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
I20260923 22:14:53.916841 1567 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskRiQe_a/build/release/bin/kudu as pid 16681
I20260923 22:14:54.756492 16814 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 22:14:54.756253 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.135.254:37515 (local address 127.1.135.193:32957)
0923 22:14:54.756371 (+ 118us) negotiation.cc:107] Waiting for socket to connect
0923 22:14:54.756420 (+ 49us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.135.254:37515: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":101,"thread_start_us":54,"threads_started":1}
I20260923 22:14:54.802397 16815 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 22:14:54.802206 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.135.254:37515 (local address 127.1.135.194:59413)
0923 22:14:54.802321 (+ 115us) negotiation.cc:107] Waiting for socket to connect
0923 22:14:54.802352 (+ 31us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.135.254:37515: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":101,"thread_start_us":59,"threads_started":1}
I20260923 22:14:54.913316 16816 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 22:14:54.913135 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.135.254:37515 (local address 127.1.135.195:34613)
0923 22:14:54.913246 (+ 111us) negotiation.cc:107] Waiting for socket to connect
0923 22:14:54.913275 (+ 29us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.135.254:37515: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":98,"thread_start_us":61,"threads_started":1}
I20260923 22:14:55.056244 16817 raft_consensus.cc:482] T a64cc070467e497dad4bebe37959837b P 2ac26bd2c8eb4b4fb12e08af1af156e2 [term 1 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260923 22:14:55.056367 16817 raft_consensus.cc:504] T a64cc070467e497dad4bebe37959837b P 2ac26bd2c8eb4b4fb12e08af1af156e2 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7d6af16a92f948c98fa0a8c4d04da648" member_type: VOTER last_known_addr { host: "127.1.135.194" port: 46495 } } peers { permanent_uuid: "7f8b6e024a774ca996fd7858a93c42b2" member_type: VOTER last_known_addr { host: "127.1.135.195" port: 34569 } } peers { permanent_uuid: "2ac26bd2c8eb4b4fb12e08af1af156e2" member_type: VOTER last_known_addr { host: "127.1.135.193" port: 37137 } }
I20260923 22:14:55.056756 16817 leader_election.cc:288] T a64cc070467e497dad4bebe37959837b P 2ac26bd2c8eb4b4fb12e08af1af156e2 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 7d6af16a92f948c98fa0a8c4d04da648 (127.1.135.194:46495), 7f8b6e024a774ca996fd7858a93c42b2 (127.1.135.195:34569)
Sep 23 22:14:55 dist-test-slave-s2fj krb5kdc[15248](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790201693, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.135.193@KRBTEST.COM for kudu/127.1.135.194@KRBTEST.COM
Sep 23 22:14:55 dist-test-slave-s2fj krb5kdc[15248](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790201693, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.135.193@KRBTEST.COM for kudu/127.1.135.195@KRBTEST.COM
I20260923 22:14:55.068818 16818 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 22:14:55.056987 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.135.193:55837 (local address 127.1.135.194:46495)
0923 22:14:55.057175 (+ 188us) server_negotiation.cc:206] Beginning negotiation
0923 22:14:55.057184 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0923 22:14:55.057238 (+ 54us) server_negotiation.cc:407] Connection header received
0923 22:14:55.057802 (+ 564us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 22:14:55.057805 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 22:14:55.057863 (+ 58us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 22:14:55.058013 (+ 150us) server_negotiation.cc:226] Negotiated authn=SASL
0923 22:14:55.058955 (+ 942us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 22:14:55.060336 (+ 1381us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 22:14:55.061842 (+ 1506us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 22:14:55.062437 (+ 595us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 22:14:55.062547 (+ 110us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 22:14:55.065333 (+ 2786us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0923 22:14:55.065337 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0923 22:14:55.065338 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0923 22:14:55.065347 (+ 9us) server_negotiation.cc:922] calling sasl_server_start()
0923 22:14:55.067252 (+ 1905us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0923 22:14:55.067642 (+ 390us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0923 22:14:55.067645 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0923 22:14:55.067645 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0923 22:14:55.067732 (+ 87us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0923 22:14:55.067977 (+ 245us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0923 22:14:55.067978 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0923 22:14:55.067978 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0923 22:14:55.068269 (+ 291us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0923 22:14:55.068354 (+ 85us) server_negotiation.cc:1035] Waiting for connection context
0923 22:14:55.068603 (+ 249us) server_negotiation.cc:299] Negotiation successful
0923 22:14:55.068714 (+ 111us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":147,"thread_start_us":88,"threads_started":1}
I20260923 22:14:55.068809 16814 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 22:14:55.056895 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.135.194:46495 (local address 127.1.135.193:55837)
0923 22:14:55.056947 (+ 52us) negotiation.cc:107] Waiting for socket to connect
0923 22:14:55.056958 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0923 22:14:55.057011 (+ 53us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0923 22:14:55.058071 (+ 1060us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0923 22:14:55.058074 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0923 22:14:55.058529 (+ 455us) client_negotiation.cc:189] Negotiated authn=SASL
0923 22:14:55.058807 (+ 278us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 22:14:55.058815 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 22:14:55.060418 (+ 1603us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 22:14:55.060418 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 22:14:55.061743 (+ 1325us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 22:14:55.061745 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 22:14:55.062528 (+ 783us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 22:14:55.062530 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 22:14:55.062679 (+ 149us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 22:14:55.063317 (+ 638us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0923 22:14:55.063322 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0923 22:14:55.065230 (+ 1908us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0923 22:14:55.067339 (+ 2109us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0923 22:14:55.067344 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0923 22:14:55.067346 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0923 22:14:55.067538 (+ 192us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0923 22:14:55.067803 (+ 265us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0923 22:14:55.067803 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0923 22:14:55.067804 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0923 22:14:55.067904 (+ 100us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0923 22:14:55.068337 (+ 433us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0923 22:14:55.068343 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0923 22:14:55.068458 (+ 115us) client_negotiation.cc:769] Sending connection context
0923 22:14:55.068556 (+ 98us) client_negotiation.cc:240] Negotiation successful
0923 22:14:55.068725 (+ 169us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":33,"mutex_wait_us":101}
I20260923 22:14:55.068919 16819 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 22:14:55.057029 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.135.195:34569 (local address 127.1.135.193:35723)
0923 22:14:55.057136 (+ 107us) negotiation.cc:107] Waiting for socket to connect
0923 22:14:55.057145 (+ 9us) client_negotiation.cc:174] Beginning negotiation
0923 22:14:55.057187 (+ 42us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0923 22:14:55.058071 (+ 884us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0923 22:14:55.058074 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0923 22:14:55.058528 (+ 454us) client_negotiation.cc:189] Negotiated authn=SASL
0923 22:14:55.058807 (+ 279us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 22:14:55.058815 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 22:14:55.060421 (+ 1606us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 22:14:55.060422 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 22:14:55.061743 (+ 1321us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 22:14:55.061745 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 22:14:55.062530 (+ 785us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 22:14:55.062530 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 22:14:55.062679 (+ 149us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 22:14:55.063295 (+ 616us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0923 22:14:55.063305 (+ 10us) client_negotiation.cc:656] calling sasl_client_start()
0923 22:14:55.065619 (+ 2314us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0923 22:14:55.067560 (+ 1941us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0923 22:14:55.067561 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0923 22:14:55.067562 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0923 22:14:55.067733 (+ 171us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0923 22:14:55.067984 (+ 251us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0923 22:14:55.067985 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0923 22:14:55.067985 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0923 22:14:55.068050 (+ 65us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0923 22:14:55.068436 (+ 386us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0923 22:14:55.068437 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0923 22:14:55.068479 (+ 42us) client_negotiation.cc:769] Sending connection context
0923 22:14:55.068556 (+ 77us) client_negotiation.cc:240] Negotiation successful
0923 22:14:55.068795 (+ 239us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":97,"mutex_wait_us":204,"thread_start_us":63,"threads_started":1}
I20260923 22:14:55.069248 16820 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 22:14:55.057131 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.135.193:35723 (local address 127.1.135.195:34569)
0923 22:14:55.057267 (+ 136us) server_negotiation.cc:206] Beginning negotiation
0923 22:14:55.057275 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0923 22:14:55.057300 (+ 25us) server_negotiation.cc:407] Connection header received
0923 22:14:55.057802 (+ 502us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 22:14:55.057805 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 22:14:55.057862 (+ 57us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 22:14:55.058013 (+ 151us) server_negotiation.cc:226] Negotiated authn=SASL
0923 22:14:55.058955 (+ 942us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 22:14:55.060309 (+ 1354us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 22:14:55.061844 (+ 1535us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 22:14:55.062437 (+ 593us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 22:14:55.062547 (+ 110us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 22:14:55.065731 (+ 3184us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0923 22:14:55.065735 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0923 22:14:55.065735 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0923 22:14:55.065744 (+ 9us) server_negotiation.cc:922] calling sasl_server_start()
0923 22:14:55.067467 (+ 1723us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0923 22:14:55.067815 (+ 348us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0923 22:14:55.067818 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0923 22:14:55.067818 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0923 22:14:55.067910 (+ 92us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0923 22:14:55.068139 (+ 229us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0923 22:14:55.068140 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0923 22:14:55.068140 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0923 22:14:55.068360 (+ 220us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0923 22:14:55.068441 (+ 81us) server_negotiation.cc:1035] Waiting for connection context
0923 22:14:55.068623 (+ 182us) server_negotiation.cc:299] Negotiation successful
0923 22:14:55.068755 (+ 132us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":103,"thread_start_us":55,"threads_started":1}
W20260923 22:14:55.069687 16431 leader_election.cc:341] T a64cc070467e497dad4bebe37959837b P 2ac26bd2c8eb4b4fb12e08af1af156e2 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 7d6af16a92f948c98fa0a8c4d04da648 (127.1.135.194:46495): Not found: Tablet not found: a64cc070467e497dad4bebe37959837b
W20260923 22:14:55.069780 16431 leader_election.cc:341] T a64cc070467e497dad4bebe37959837b P 2ac26bd2c8eb4b4fb12e08af1af156e2 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 7f8b6e024a774ca996fd7858a93c42b2 (127.1.135.195:34569): Not found: Tablet not found: a64cc070467e497dad4bebe37959837b
I20260923 22:14:55.069813 16431 leader_election.cc:302] T a64cc070467e497dad4bebe37959837b P 2ac26bd2c8eb4b4fb12e08af1af156e2 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 2ac26bd2c8eb4b4fb12e08af1af156e2; no voters: 7d6af16a92f948c98fa0a8c4d04da648, 7f8b6e024a774ca996fd7858a93c42b2
I20260923 22:14:55.069895 16817 raft_consensus.cc:2726] T a64cc070467e497dad4bebe37959837b P 2ac26bd2c8eb4b4fb12e08af1af156e2 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260923 22:14:55.757652 16821 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 22:14:55.757335 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.135.254:37515 (local address 127.1.135.193:60635)
0923 22:14:55.757572 (+ 237us) negotiation.cc:107] Waiting for socket to connect
0923 22:14:55.757608 (+ 36us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.135.254:37515: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":221,"thread_start_us":56,"threads_started":1}
I20260923 22:14:55.803026 16822 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 22:14:55.802802 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.135.254:37515 (local address 127.1.135.194:50809)
0923 22:14:55.802949 (+ 147us) negotiation.cc:107] Waiting for socket to connect
0923 22:14:55.802986 (+ 37us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.135.254:37515: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":131,"thread_start_us":64,"threads_started":1}
I20260923 22:14:55.913892 16823 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 22:14:55.913686 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.135.254:37515 (local address 127.1.135.195:45353)
0923 22:14:55.913810 (+ 124us) negotiation.cc:107] Waiting for socket to connect
0923 22:14:55.913840 (+ 30us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.135.254:37515: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":98,"thread_start_us":54,"threads_started":1}
I20260923 22:14:56.758291 16824 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 22:14:56.758087 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.135.254:37515 (local address 127.1.135.193:33807)
0923 22:14:56.758213 (+ 126us) negotiation.cc:107] Waiting for socket to connect
0923 22:14:56.758246 (+ 33us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.135.254:37515: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":110,"thread_start_us":56,"threads_started":1}
I20260923 22:14:56.803613 16825 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 22:14:56.803425 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.135.254:37515 (local address 127.1.135.194:57073)
0923 22:14:56.803546 (+ 121us) negotiation.cc:107] Waiting for socket to connect
0923 22:14:56.803579 (+ 33us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.135.254:37515: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":106,"thread_start_us":65,"threads_started":1}
I20260923 22:14:56.914471 16826 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 22:14:56.914271 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.135.254:37515 (local address 127.1.135.195:53043)
0923 22:14:56.914408 (+ 137us) negotiation.cc:107] Waiting for socket to connect
0923 22:14:56.914435 (+ 27us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.135.254:37515: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":117,"thread_start_us":88,"threads_started":1}
I20260923 22:14:57.020965 16827 raft_consensus.cc:482] T a64cc070467e497dad4bebe37959837b P 2ac26bd2c8eb4b4fb12e08af1af156e2 [term 1 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260923 22:14:57.021052 16827 raft_consensus.cc:504] T a64cc070467e497dad4bebe37959837b P 2ac26bd2c8eb4b4fb12e08af1af156e2 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7d6af16a92f948c98fa0a8c4d04da648" member_type: VOTER last_known_addr { host: "127.1.135.194" port: 46495 } } peers { permanent_uuid: "7f8b6e024a774ca996fd7858a93c42b2" member_type: VOTER last_known_addr { host: "127.1.135.195" port: 34569 } } peers { permanent_uuid: "2ac26bd2c8eb4b4fb12e08af1af156e2" member_type: VOTER last_known_addr { host: "127.1.135.193" port: 37137 } }
I20260923 22:14:57.021175 16827 leader_election.cc:288] T a64cc070467e497dad4bebe37959837b P 2ac26bd2c8eb4b4fb12e08af1af156e2 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 7d6af16a92f948c98fa0a8c4d04da648 (127.1.135.194:46495), 7f8b6e024a774ca996fd7858a93c42b2 (127.1.135.195:34569)
W20260923 22:14:57.021606 16431 leader_election.cc:341] T a64cc070467e497dad4bebe37959837b P 2ac26bd2c8eb4b4fb12e08af1af156e2 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 7d6af16a92f948c98fa0a8c4d04da648 (127.1.135.194:46495): Not found: Tablet not found: a64cc070467e497dad4bebe37959837b
W20260923 22:14:57.021685 16431 leader_election.cc:341] T a64cc070467e497dad4bebe37959837b P 2ac26bd2c8eb4b4fb12e08af1af156e2 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 7f8b6e024a774ca996fd7858a93c42b2 (127.1.135.195:34569): Not found: Tablet not found: a64cc070467e497dad4bebe37959837b
I20260923 22:14:57.021728 16431 leader_election.cc:302] T a64cc070467e497dad4bebe37959837b P 2ac26bd2c8eb4b4fb12e08af1af156e2 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 2ac26bd2c8eb4b4fb12e08af1af156e2; no voters: 7d6af16a92f948c98fa0a8c4d04da648, 7f8b6e024a774ca996fd7858a93c42b2
I20260923 22:14:57.021826 16827 raft_consensus.cc:2726] T a64cc070467e497dad4bebe37959837b P 2ac26bd2c8eb4b4fb12e08af1af156e2 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260923 22:14:57.758983 16828 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 22:14:57.758792 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.135.254:37515 (local address 127.1.135.193:45045)
0923 22:14:57.758907 (+ 115us) negotiation.cc:107] Waiting for socket to connect
0923 22:14:57.758939 (+ 32us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.135.254:37515: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":101,"thread_start_us":61,"threads_started":1}
I20260923 22:14:57.804209 16829 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 22:14:57.804017 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.135.254:37515 (local address 127.1.135.194:44749)
0923 22:14:57.804146 (+ 129us) negotiation.cc:107] Waiting for socket to connect
0923 22:14:57.804177 (+ 31us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.135.254:37515: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":114,"thread_start_us":89,"threads_started":1}
I20260923 22:14:57.914994 16830 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 22:14:57.914835 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.135.254:37515 (local address 127.1.135.195:52059)
0923 22:14:57.914935 (+ 100us) negotiation.cc:107] Waiting for socket to connect
0923 22:14:57.914963 (+ 28us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.135.254:37515: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":88,"thread_start_us":52,"threads_started":1}
I20260923 22:14:58.759656 16831 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 22:14:58.759437 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.135.254:37515 (local address 127.1.135.193:49013)
0923 22:14:58.759577 (+ 140us) negotiation.cc:107] Waiting for socket to connect
0923 22:14:58.759610 (+ 33us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.135.254:37515: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":125,"thread_start_us":71,"threads_started":1}
I20260923 22:14:58.804755 16832 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 22:14:58.804585 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.135.254:37515 (local address 127.1.135.194:50637)
0923 22:14:58.804692 (+ 107us) negotiation.cc:107] Waiting for socket to connect
0923 22:14:58.804722 (+ 30us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.135.254:37515: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":92,"thread_start_us":52,"threads_started":1}
I20260923 22:14:58.915679 16833 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 22:14:58.915492 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.135.254:37515 (local address 127.1.135.195:42065)
0923 22:14:58.915614 (+ 122us) negotiation.cc:107] Waiting for socket to connect
0923 22:14:58.915645 (+ 31us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.135.254:37515: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":108,"thread_start_us":72,"threads_started":1}
I20260923 22:14:59.078504 16834 raft_consensus.cc:482] T a64cc070467e497dad4bebe37959837b P 2ac26bd2c8eb4b4fb12e08af1af156e2 [term 1 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260923 22:14:59.078624 16834 raft_consensus.cc:504] T a64cc070467e497dad4bebe37959837b P 2ac26bd2c8eb4b4fb12e08af1af156e2 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7d6af16a92f948c98fa0a8c4d04da648" member_type: VOTER last_known_addr { host: "127.1.135.194" port: 46495 } } peers { permanent_uuid: "7f8b6e024a774ca996fd7858a93c42b2" member_type: VOTER last_known_addr { host: "127.1.135.195" port: 34569 } } peers { permanent_uuid: "2ac26bd2c8eb4b4fb12e08af1af156e2" member_type: VOTER last_known_addr { host: "127.1.135.193" port: 37137 } }
I20260923 22:14:59.078799 16834 leader_election.cc:288] T a64cc070467e497dad4bebe37959837b P 2ac26bd2c8eb4b4fb12e08af1af156e2 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 7d6af16a92f948c98fa0a8c4d04da648 (127.1.135.194:46495), 7f8b6e024a774ca996fd7858a93c42b2 (127.1.135.195:34569)
W20260923 22:14:59.079171 16431 leader_election.cc:341] T a64cc070467e497dad4bebe37959837b P 2ac26bd2c8eb4b4fb12e08af1af156e2 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 7d6af16a92f948c98fa0a8c4d04da648 (127.1.135.194:46495): Not found: Tablet not found: a64cc070467e497dad4bebe37959837b
W20260923 22:14:59.079241 16431 leader_election.cc:341] T a64cc070467e497dad4bebe37959837b P 2ac26bd2c8eb4b4fb12e08af1af156e2 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 7f8b6e024a774ca996fd7858a93c42b2 (127.1.135.195:34569): Not found: Tablet not found: a64cc070467e497dad4bebe37959837b
I20260923 22:14:59.079255 16431 leader_election.cc:302] T a64cc070467e497dad4bebe37959837b P 2ac26bd2c8eb4b4fb12e08af1af156e2 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 2ac26bd2c8eb4b4fb12e08af1af156e2; no voters: 7d6af16a92f948c98fa0a8c4d04da648, 7f8b6e024a774ca996fd7858a93c42b2
I20260923 22:14:59.079324 16834 raft_consensus.cc:2726] T a64cc070467e497dad4bebe37959837b P 2ac26bd2c8eb4b4fb12e08af1af156e2 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260923 22:14:59.760339 16835 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 22:14:59.760143 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.135.254:37515 (local address 127.1.135.193:44039)
0923 22:14:59.760264 (+ 121us) negotiation.cc:107] Waiting for socket to connect
0923 22:14:59.760294 (+ 30us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.135.254:37515: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":107,"thread_start_us":69,"threads_started":1}
I20260923 22:14:59.805303 16836 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 22:14:59.805138 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.135.254:37515 (local address 127.1.135.194:46189)
0923 22:14:59.805245 (+ 107us) negotiation.cc:107] Waiting for socket to connect
0923 22:14:59.805271 (+ 26us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.135.254:37515: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":94,"thread_start_us":60,"threads_started":1}
I20260923 22:14:59.916229 16837 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 22:14:59.916050 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.135.254:37515 (local address 127.1.135.195:54593)
0923 22:14:59.916164 (+ 114us) negotiation.cc:107] Waiting for socket to connect
0923 22:14:59.916191 (+ 27us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.135.254:37515: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":101,"thread_start_us":75,"threads_started":1}
I20260923 22:15:00.760938 16838 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 22:15:00.760756 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.135.254:37515 (local address 127.1.135.193:58555)
0923 22:15:00.760864 (+ 108us) negotiation.cc:107] Waiting for socket to connect
0923 22:15:00.760894 (+ 30us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.135.254:37515: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":95,"thread_start_us":57,"threads_started":1}
I20260923 22:15:00.805819 16839 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 22:15:00.805660 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.135.254:37515 (local address 127.1.135.194:54629)
0923 22:15:00.805758 (+ 98us) negotiation.cc:107] Waiting for socket to connect
0923 22:15:00.805787 (+ 29us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.135.254:37515: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":86,"thread_start_us":50,"threads_started":1}
I20260923 22:15:00.916749 16840 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 22:15:00.916551 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.135.254:37515 (local address 127.1.135.195:52003)
0923 22:15:00.916669 (+ 118us) negotiation.cc:107] Waiting for socket to connect
0923 22:15:00.916699 (+ 30us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.135.254:37515: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":104,"thread_start_us":52,"threads_started":1}
I20260923 22:15:01.203402 16841 raft_consensus.cc:482] T a64cc070467e497dad4bebe37959837b P 2ac26bd2c8eb4b4fb12e08af1af156e2 [term 1 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260923 22:15:01.203529 16841 raft_consensus.cc:504] T a64cc070467e497dad4bebe37959837b P 2ac26bd2c8eb4b4fb12e08af1af156e2 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7d6af16a92f948c98fa0a8c4d04da648" member_type: VOTER last_known_addr { host: "127.1.135.194" port: 46495 } } peers { permanent_uuid: "7f8b6e024a774ca996fd7858a93c42b2" member_type: VOTER last_known_addr { host: "127.1.135.195" port: 34569 } } peers { permanent_uuid: "2ac26bd2c8eb4b4fb12e08af1af156e2" member_type: VOTER last_known_addr { host: "127.1.135.193" port: 37137 } }
I20260923 22:15:01.203735 16841 leader_election.cc:288] T a64cc070467e497dad4bebe37959837b P 2ac26bd2c8eb4b4fb12e08af1af156e2 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 7d6af16a92f948c98fa0a8c4d04da648 (127.1.135.194:46495), 7f8b6e024a774ca996fd7858a93c42b2 (127.1.135.195:34569)
W20260923 22:15:01.204099 16431 leader_election.cc:341] T a64cc070467e497dad4bebe37959837b P 2ac26bd2c8eb4b4fb12e08af1af156e2 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 7d6af16a92f948c98fa0a8c4d04da648 (127.1.135.194:46495): Not found: Tablet not found: a64cc070467e497dad4bebe37959837b
W20260923 22:15:01.204176 16431 leader_election.cc:341] T a64cc070467e497dad4bebe37959837b P 2ac26bd2c8eb4b4fb12e08af1af156e2 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 7f8b6e024a774ca996fd7858a93c42b2 (127.1.135.195:34569): Not found: Tablet not found: a64cc070467e497dad4bebe37959837b
I20260923 22:15:01.204190 16431 leader_election.cc:302] T a64cc070467e497dad4bebe37959837b P 2ac26bd2c8eb4b4fb12e08af1af156e2 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 2ac26bd2c8eb4b4fb12e08af1af156e2; no voters: 7d6af16a92f948c98fa0a8c4d04da648, 7f8b6e024a774ca996fd7858a93c42b2
I20260923 22:15:01.204305 16841 raft_consensus.cc:2726] T a64cc070467e497dad4bebe37959837b P 2ac26bd2c8eb4b4fb12e08af1af156e2 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260923 22:15:01.761660 16842 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 22:15:01.761431 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.135.254:37515 (local address 127.1.135.193:43767)
0923 22:15:01.761566 (+ 135us) negotiation.cc:107] Waiting for socket to connect
0923 22:15:01.761607 (+ 41us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.135.254:37515: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":115,"thread_start_us":87,"threads_started":1}
I20260923 22:15:01.806389 16843 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 22:15:01.806212 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.135.254:37515 (local address 127.1.135.194:47887)
0923 22:15:01.806321 (+ 109us) negotiation.cc:107] Waiting for socket to connect
0923 22:15:01.806357 (+ 36us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.135.254:37515: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":92,"thread_start_us":53,"threads_started":1}
I20260923 22:15:01.917481 16844 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 22:15:01.917273 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.135.254:37515 (local address 127.1.135.195:57419)
0923 22:15:01.917389 (+ 116us) negotiation.cc:107] Waiting for socket to connect
0923 22:15:01.917427 (+ 38us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.135.254:37515: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":99,"thread_start_us":57,"threads_started":1}
I20260923 22:15:02.762428 16845 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 22:15:02.762187 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.135.254:37515 (local address 127.1.135.193:33021)
0923 22:15:02.762336 (+ 149us) negotiation.cc:107] Waiting for socket to connect
0923 22:15:02.762381 (+ 45us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.135.254:37515: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":129,"thread_start_us":63,"threads_started":1}
I20260923 22:15:02.807009 16846 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 22:15:02.806830 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.135.254:37515 (local address 127.1.135.194:39453)
0923 22:15:02.806936 (+ 106us) negotiation.cc:107] Waiting for socket to connect
0923 22:15:02.806978 (+ 42us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.135.254:37515: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":88,"thread_start_us":52,"threads_started":1}
I20260923 22:15:02.918107 16847 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 22:15:02.917924 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.135.254:37515 (local address 127.1.135.195:50397)
0923 22:15:02.918034 (+ 110us) negotiation.cc:107] Waiting for socket to connect
0923 22:15:02.918074 (+ 40us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.135.254:37515: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":92,"thread_start_us":53,"threads_started":1}
I20260923 22:15:03.763178 16848 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 22:15:03.762941 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.135.254:37515 (local address 127.1.135.193:52547)
0923 22:15:03.763084 (+ 143us) negotiation.cc:107] Waiting for socket to connect
0923 22:15:03.763115 (+ 31us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.135.254:37515: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":128,"thread_start_us":91,"threads_started":1}
I20260923 22:15:03.807576 16849 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 22:15:03.807396 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.135.254:37515 (local address 127.1.135.194:39491)
0923 22:15:03.807514 (+ 118us) negotiation.cc:107] Waiting for socket to connect
0923 22:15:03.807544 (+ 30us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.135.254:37515: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":104,"thread_start_us":66,"threads_started":1}
I20260923 22:15:03.918715 16850 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 22:15:03.918541 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.135.254:37515 (local address 127.1.135.195:56999)
0923 22:15:03.918653 (+ 112us) negotiation.cc:107] Waiting for socket to connect
0923 22:15:03.918681 (+ 28us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.135.254:37515: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":98,"thread_start_us":57,"threads_started":1}
I20260923 22:15:04.027578 16851 raft_consensus.cc:482] T a64cc070467e497dad4bebe37959837b P 2ac26bd2c8eb4b4fb12e08af1af156e2 [term 1 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260923 22:15:04.027662 16851 raft_consensus.cc:504] T a64cc070467e497dad4bebe37959837b P 2ac26bd2c8eb4b4fb12e08af1af156e2 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7d6af16a92f948c98fa0a8c4d04da648" member_type: VOTER last_known_addr { host: "127.1.135.194" port: 46495 } } peers { permanent_uuid: "7f8b6e024a774ca996fd7858a93c42b2" member_type: VOTER last_known_addr { host: "127.1.135.195" port: 34569 } } peers { permanent_uuid: "2ac26bd2c8eb4b4fb12e08af1af156e2" member_type: VOTER last_known_addr { host: "127.1.135.193" port: 37137 } }
I20260923 22:15:04.027791 16851 leader_election.cc:288] T a64cc070467e497dad4bebe37959837b P 2ac26bd2c8eb4b4fb12e08af1af156e2 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 7d6af16a92f948c98fa0a8c4d04da648 (127.1.135.194:46495), 7f8b6e024a774ca996fd7858a93c42b2 (127.1.135.195:34569)
W20260923 22:15:04.028388 16431 leader_election.cc:341] T a64cc070467e497dad4bebe37959837b P 2ac26bd2c8eb4b4fb12e08af1af156e2 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 7d6af16a92f948c98fa0a8c4d04da648 (127.1.135.194:46495): Not found: Tablet not found: a64cc070467e497dad4bebe37959837b
W20260923 22:15:04.028472 16431 leader_election.cc:341] T a64cc070467e497dad4bebe37959837b P 2ac26bd2c8eb4b4fb12e08af1af156e2 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 7f8b6e024a774ca996fd7858a93c42b2 (127.1.135.195:34569): Not found: Tablet not found: a64cc070467e497dad4bebe37959837b
I20260923 22:15:04.028515 16431 leader_election.cc:302] T a64cc070467e497dad4bebe37959837b P 2ac26bd2c8eb4b4fb12e08af1af156e2 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 2ac26bd2c8eb4b4fb12e08af1af156e2; no voters: 7d6af16a92f948c98fa0a8c4d04da648, 7f8b6e024a774ca996fd7858a93c42b2
I20260923 22:15:04.028612 16851 raft_consensus.cc:2726] T a64cc070467e497dad4bebe37959837b P 2ac26bd2c8eb4b4fb12e08af1af156e2 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260923 22:15:04.763836 16852 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 22:15:04.763629 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.135.254:37515 (local address 127.1.135.193:33379)
0923 22:15:04.763760 (+ 131us) negotiation.cc:107] Waiting for socket to connect
0923 22:15:04.763788 (+ 28us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.135.254:37515: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":118,"thread_start_us":77,"threads_started":1}
I20260923 22:15:04.808238 16853 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 22:15:04.808011 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.135.254:37515 (local address 127.1.135.194:49757)
0923 22:15:04.808172 (+ 161us) negotiation.cc:107] Waiting for socket to connect
0923 22:15:04.808203 (+ 31us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.135.254:37515: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":146,"thread_start_us":107,"threads_started":1}
I20260923 22:15:04.919270 16854 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 22:15:04.919098 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.135.254:37515 (local address 127.1.135.195:44779)
0923 22:15:04.919203 (+ 105us) negotiation.cc:107] Waiting for socket to connect
0923 22:15:04.919235 (+ 32us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.135.254:37515: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":91,"thread_start_us":52,"threads_started":1}
I20260923 22:15:05.764564 16855 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 22:15:05.764368 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.135.254:37515 (local address 127.1.135.193:52333)
0923 22:15:05.764483 (+ 115us) negotiation.cc:107] Waiting for socket to connect
0923 22:15:05.764520 (+ 37us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.135.254:37515: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":101,"thread_start_us":56,"threads_started":1}
I20260923 22:15:05.808823 16856 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 22:15:05.808615 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.135.254:37515 (local address 127.1.135.194:51171)
0923 22:15:05.808752 (+ 137us) negotiation.cc:107] Waiting for socket to connect
0923 22:15:05.808789 (+ 37us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.135.254:37515: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":103,"thread_start_us":63,"threads_started":1}
I20260923 22:15:05.919823 16857 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 22:15:05.919655 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.135.254:37515 (local address 127.1.135.195:40793)
0923 22:15:05.919757 (+ 102us) negotiation.cc:107] Waiting for socket to connect
0923 22:15:05.919789 (+ 32us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.135.254:37515: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":89,"thread_start_us":54,"threads_started":1}
I20260923 22:15:06.765203 16858 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 22:15:06.764981 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.135.254:37515 (local address 127.1.135.193:41081)
0923 22:15:06.765104 (+ 123us) negotiation.cc:107] Waiting for socket to connect
0923 22:15:06.765157 (+ 53us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.135.254:37515: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":106,"thread_start_us":57,"threads_started":1}
I20260923 22:15:06.809443 16859 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 22:15:06.809241 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.135.254:37515 (local address 127.1.135.194:47079)
0923 22:15:06.809365 (+ 124us) negotiation.cc:107] Waiting for socket to connect
0923 22:15:06.809395 (+ 30us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.135.254:37515: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":111,"thread_start_us":71,"threads_started":1}
I20260923 22:15:06.920372 16860 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 22:15:06.920200 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.135.254:37515 (local address 127.1.135.195:49321)
0923 22:15:06.920305 (+ 105us) negotiation.cc:107] Waiting for socket to connect
0923 22:15:06.920337 (+ 32us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.135.254:37515: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":91,"thread_start_us":53,"threads_started":1}
I20260923 22:15:07.765920 16861 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 22:15:07.765691 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.135.254:37515 (local address 127.1.135.193:51529)
0923 22:15:07.765837 (+ 146us) negotiation.cc:107] Waiting for socket to connect
0923 22:15:07.765873 (+ 36us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.135.254:37515: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":128,"thread_start_us":57,"threads_started":1}
I20260923 22:15:07.810038 16862 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 22:15:07.809821 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.135.254:37515 (local address 127.1.135.194:38859)
0923 22:15:07.809963 (+ 142us) negotiation.cc:107] Waiting for socket to connect
0923 22:15:07.809998 (+ 35us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.135.254:37515: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":114,"thread_start_us":59,"threads_started":1}
I20260923 22:15:07.920985 16863 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 22:15:07.920783 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.135.254:37515 (local address 127.1.135.195:49041)
0923 22:15:07.920908 (+ 125us) negotiation.cc:107] Waiting for socket to connect
0923 22:15:07.920941 (+ 33us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.135.254:37515: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":109,"thread_start_us":62,"threads_started":1}
I20260923 22:15:08.766748 16864 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 22:15:08.766544 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.135.254:37515 (local address 127.1.135.193:49337)
0923 22:15:08.766671 (+ 127us) negotiation.cc:107] Waiting for socket to connect
0923 22:15:08.766703 (+ 32us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.135.254:37515: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":112,"thread_start_us":69,"threads_started":1}
I20260923 22:15:08.810724 16865 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 22:15:08.810526 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.135.254:37515 (local address 127.1.135.194:48287)
0923 22:15:08.810646 (+ 120us) negotiation.cc:107] Waiting for socket to connect
0923 22:15:08.810682 (+ 36us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.135.254:37515: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":102,"thread_start_us":57,"threads_started":1}
I20260923 22:15:08.921676 16866 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 22:15:08.921489 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.135.254:37515 (local address 127.1.135.195:45143)
0923 22:15:08.921607 (+ 118us) negotiation.cc:107] Waiting for socket to connect
0923 22:15:08.921637 (+ 30us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.135.254:37515: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":104,"thread_start_us":59,"threads_started":1}
I20260923 22:15:08.927158 1567 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskRiQe_a/build/release/bin/kudu with pid 16385
I20260923 22:15:08.931615 1567 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskRiQe_a/build/release/bin/kudu with pid 16548
I20260923 22:15:08.935642 1567 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskRiQe_a/build/release/bin/kudu with pid 16681
2026-09-23T22:15:08Z chronyd exiting
[ OK ] SecurityITest.TestEnableAndDisableEncryptedIPKICert (18011 ms)
[ RUN ] SecurityITest.TestEnableAndDisableEncryptedTSK
Loading random data
Initializing database '/tmp/dist-test-taskRiQe_a/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790201539513534-1567-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Sep 23 22:15:08 dist-test-slave-s2fj krb5kdc[16869](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Sep 23 22:15:08 dist-test-slave-s2fj krb5kdc[16869](info): set up 2 sockets
Sep 23 22:15:08 dist-test-slave-s2fj krb5kdc[16869](info): commencing operation
krb5kdc: starting...
W20260923 22:15:09.956276 1567 mini_kdc.cc:121] Time spent starting KDC: real 1.001s user 0.000s sys 0.005s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskRiQe_a/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790201539513534-1567-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskRiQe_a/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790201539513534-1567-0/krb5kdc/test-user.keytab.
Sep 23 22:15:10 dist-test-slave-s2fj krb5kdc[16869](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790201710, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-09-23T22:15:10Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-09-23T22:15:10Z Disabled control of system clock
WARNING: no policy specified for kudu/127.1.135.254@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.1.135.254@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.1.135.254 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskRiQe_a/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790201539513534-1567-0/krb5kdc/kudu_127.1.135.254.keytab.
Entry for principal kudu/127.1.135.254 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskRiQe_a/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790201539513534-1567-0/krb5kdc/kudu_127.1.135.254.keytab.
I20260923 22:15:10.073252 1567 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskRiQe_a/build/release/bin/kudu
/tmp/dist-test-taskRiQe_a/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskRiQe_a/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790201539513534-1567-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskRiQe_a/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790201539513534-1567-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskRiQe_a/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790201539513534-1567-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskRiQe_a/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790201539513534-1567-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.1.135.254:45507
--webserver_interface=127.1.135.254
--webserver_port=0
--keytab_file=/tmp/dist-test-taskRiQe_a/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790201539513534-1567-0/krb5kdc/kudu_127.1.135.254.keytab
--principal=kudu/127.1.135.254
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.1.135.212:34139
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.1.135.254:45507
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--authn_token_validity_seconds=20
--authz_token_validity_seconds=20
--tsk_rotation_seconds=20 with env {KRB5CCNAME=/tmp/dist-test-taskRiQe_a/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790201539513534-1567-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskRiQe_a/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790201539513534-1567-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskRiQe_a/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790201539513534-1567-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260923 22:15:10.151674 16882 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260923 22:15:10.151871 16882 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260923 22:15:10.151892 16882 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260923 22:15:10.153036 16882 flags.cc:425] Enabled experimental flag: --authn_token_validity_seconds=20
W20260923 22:15:10.153079 16882 flags.cc:425] Enabled experimental flag: --authz_token_validity_seconds=20
W20260923 22:15:10.153092 16882 flags.cc:425] Enabled experimental flag: --tsk_rotation_seconds=20
W20260923 22:15:10.153103 16882 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260923 22:15:10.153158 16882 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260923 22:15:10.153177 16882 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260923 22:15:10.153199 16882 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260923 22:15:10.153225 16882 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260923 22:15:10.154688 16882 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.1.135.212:34139
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskRiQe_a/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790201539513534-1567-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskRiQe_a/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790201539513534-1567-0/minicluster-data/master-0/wal
--authn_token_validity_seconds=20
--authz_token_validity_seconds=20
--tsk_rotation_seconds=20
--ipki_ca_key_size=768
--master_addresses=127.1.135.254:45507
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskRiQe_a/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790201539513534-1567-0/krb5kdc/kudu_127.1.135.254.keytab
--principal=kudu/127.1.135.254
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.1.135.254:45507
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskRiQe_a/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790201539513534-1567-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.1.135.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.16882
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskRiQe_a/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790201539513534-1567-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 23 Sep 2026 20:53:37 UTC on ccycloud.quasar-rkviyx.root.comops.site
build id 4668083
I20260923 22:15:10.154937 16882 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260923 22:15:10.155162 16882 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260923 22:15:10.159759 16888 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260923 22:15:10.159965 16882 server_base.cc:1034] running on GCE node
W20260923 22:15:10.159763 16890 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260923 22:15:10.159780 16887 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260923 22:15:10.160377 16882 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260923 22:15:10.160650 16882 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260923 22:15:10.161816 16882 hybrid_clock.cc:648] HybridClock initialized: now 1790201710161801 us; error 26 us; skew 500 ppm
Sep 23 22:15:10 dist-test-slave-s2fj krb5kdc[16869](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790201710, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.135.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260923 22:15:10.164198 16882 init.cc:377] Logged in from keytab as kudu/127.1.135.254@KRBTEST.COM (short username kudu)
I20260923 22:15:10.164631 16882 webserver.cc:460] Webserver started at http://127.1.135.254:34177/ using document root <none> and password file <none>
I20260923 22:15:10.164801 16882 fs_manager.cc:359] Metadata directory not provided
I20260923 22:15:10.164839 16882 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260923 22:15:10.164932 16882 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260923 22:15:10.165886 16882 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskRiQe_a/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790201539513534-1567-0/minicluster-data/master-0/data/instance:
uuid: "2d2671e82fe14a6fadf679b384125ea3"
format_stamp: "Formatted at 2026-09-23 22:15:10 on dist-test-slave-s2fj"
server_key: "b4b3416d8d0a070e64cead2cd4d90893"
server_key_iv: "6a4a527f9c3e1de279d16239eceaa956"
server_key_version: "encryptionkey@0"
I20260923 22:15:10.166172 16882 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskRiQe_a/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790201539513534-1567-0/minicluster-data/master-0/wal/instance:
uuid: "2d2671e82fe14a6fadf679b384125ea3"
format_stamp: "Formatted at 2026-09-23 22:15:10 on dist-test-slave-s2fj"
server_key: "b4b3416d8d0a070e64cead2cd4d90893"
server_key_iv: "6a4a527f9c3e1de279d16239eceaa956"
server_key_version: "encryptionkey@0"
I20260923 22:15:10.167411 16882 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260923 22:15:10.168047 16897 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260923 22:15:10.168254 16882 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260923 22:15:10.168339 16882 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskRiQe_a/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790201539513534-1567-0/minicluster-data/master-0/data,/tmp/dist-test-taskRiQe_a/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790201539513534-1567-0/minicluster-data/master-0/wal
uuid: "2d2671e82fe14a6fadf679b384125ea3"
format_stamp: "Formatted at 2026-09-23 22:15:10 on dist-test-slave-s2fj"
server_key: "b4b3416d8d0a070e64cead2cd4d90893"
server_key_iv: "6a4a527f9c3e1de279d16239eceaa956"
server_key_version: "encryptionkey@0"
I20260923 22:15:10.168407 16882 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskRiQe_a/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790201539513534-1567-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskRiQe_a/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790201539513534-1567-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskRiQe_a/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790201539513534-1567-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260923 22:15:10.179670 16882 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260923 22:15:10.181473 16882 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260923 22:15:10.181613 16882 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260923 22:15:10.185312 16882 rpc_server.cc:307] RPC server started. Bound to: 127.1.135.254:45507
I20260923 22:15:10.185616 16949 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.1.135.254:45507 every 8 connection(s)
I20260923 22:15:10.185680 16882 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskRiQe_a/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790201539513534-1567-0/minicluster-data/master-0/data/info.pb
I20260923 22:15:10.186326 16950 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260923 22:15:10.187583 1567 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskRiQe_a/build/release/bin/kudu as pid 16882
I20260923 22:15:10.187698 1567 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskRiQe_a/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790201539513534-1567-0/minicluster-data/master-0/wal/instance
I20260923 22:15:10.187892 1567 external_mini_cluster.cc:1428] Setting key 9e996b47a7202d244ee48706fef322b9
I20260923 22:15:10.188828 16950 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 2d2671e82fe14a6fadf679b384125ea3: Bootstrap starting.
I20260923 22:15:10.189510 16950 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 2d2671e82fe14a6fadf679b384125ea3: Neither blocks nor log segments found. Creating new log.
I20260923 22:15:10.189839 16950 log.cc:824] T 00000000000000000000000000000000 P 2d2671e82fe14a6fadf679b384125ea3: Log is configured to *not* fsync() on all Append() calls
I20260923 22:15:10.190506 16950 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 2d2671e82fe14a6fadf679b384125ea3: No bootstrap required, opened a new log
I20260923 22:15:10.192845 16950 raft_consensus.cc:348] T 00000000000000000000000000000000 P 2d2671e82fe14a6fadf679b384125ea3 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2d2671e82fe14a6fadf679b384125ea3" member_type: VOTER last_known_addr { host: "127.1.135.254" port: 45507 } }
I20260923 22:15:10.193009 16950 raft_consensus.cc:374] T 00000000000000000000000000000000 P 2d2671e82fe14a6fadf679b384125ea3 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260923 22:15:10.193044 16950 raft_consensus.cc:729] T 00000000000000000000000000000000 P 2d2671e82fe14a6fadf679b384125ea3 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 2d2671e82fe14a6fadf679b384125ea3, State: Initialized, Role: FOLLOWER
I20260923 22:15:10.193215 16950 consensus_queue.cc:260] T 00000000000000000000000000000000 P 2d2671e82fe14a6fadf679b384125ea3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2d2671e82fe14a6fadf679b384125ea3" member_type: VOTER last_known_addr { host: "127.1.135.254" port: 45507 } }
I20260923 22:15:10.193310 16950 raft_consensus.cc:388] T 00000000000000000000000000000000 P 2d2671e82fe14a6fadf679b384125ea3 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260923 22:15:10.193342 16950 raft_consensus.cc:482] T 00000000000000000000000000000000 P 2d2671e82fe14a6fadf679b384125ea3 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260923 22:15:10.193390 16950 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 2d2671e82fe14a6fadf679b384125ea3 [term 0 FOLLOWER]: Advancing to term 1
I20260923 22:15:10.194136 16950 raft_consensus.cc:504] T 00000000000000000000000000000000 P 2d2671e82fe14a6fadf679b384125ea3 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2d2671e82fe14a6fadf679b384125ea3" member_type: VOTER last_known_addr { host: "127.1.135.254" port: 45507 } }
I20260923 22:15:10.194255 16950 leader_election.cc:302] T 00000000000000000000000000000000 P 2d2671e82fe14a6fadf679b384125ea3 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 2d2671e82fe14a6fadf679b384125ea3; no voters:
I20260923 22:15:10.194427 16950 leader_election.cc:288] T 00000000000000000000000000000000 P 2d2671e82fe14a6fadf679b384125ea3 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260923 22:15:10.194476 16955 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 2d2671e82fe14a6fadf679b384125ea3 [term 1 FOLLOWER]: Leader election won for term 1
I20260923 22:15:10.194618 16955 raft_consensus.cc:686] T 00000000000000000000000000000000 P 2d2671e82fe14a6fadf679b384125ea3 [term 1 LEADER]: Becoming Leader. State: Replica: 2d2671e82fe14a6fadf679b384125ea3, State: Running, Role: LEADER
I20260923 22:15:10.194693 16950 sys_catalog.cc:564] T 00000000000000000000000000000000 P 2d2671e82fe14a6fadf679b384125ea3 [sys.catalog]: configured and running, proceeding with master startup.
I20260923 22:15:10.194741 16955 consensus_queue.cc:237] T 00000000000000000000000000000000 P 2d2671e82fe14a6fadf679b384125ea3 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2d2671e82fe14a6fadf679b384125ea3" member_type: VOTER last_known_addr { host: "127.1.135.254" port: 45507 } }
Sep 23 22:15:10 dist-test-slave-s2fj krb5kdc[16869](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790201710, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.1.135.254@KRBTEST.COM
I20260923 22:15:10.195061 16957 sys_catalog.cc:455] T 00000000000000000000000000000000 P 2d2671e82fe14a6fadf679b384125ea3 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "2d2671e82fe14a6fadf679b384125ea3" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2d2671e82fe14a6fadf679b384125ea3" member_type: VOTER last_known_addr { host: "127.1.135.254" port: 45507 } } }
I20260923 22:15:10.195142 16957 sys_catalog.cc:458] T 00000000000000000000000000000000 P 2d2671e82fe14a6fadf679b384125ea3 [sys.catalog]: This master's current role is: LEADER
I20260923 22:15:10.195051 16955 sys_catalog.cc:455] T 00000000000000000000000000000000 P 2d2671e82fe14a6fadf679b384125ea3 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 2d2671e82fe14a6fadf679b384125ea3. Latest consensus state: current_term: 1 leader_uuid: "2d2671e82fe14a6fadf679b384125ea3" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2d2671e82fe14a6fadf679b384125ea3" member_type: VOTER last_known_addr { host: "127.1.135.254" port: 45507 } } }
I20260923 22:15:10.195431 16955 sys_catalog.cc:458] T 00000000000000000000000000000000 P 2d2671e82fe14a6fadf679b384125ea3 [sys.catalog]: This master's current role is: LEADER
I20260923 22:15:10.195809 16959 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260923 22:15:10.196658 16959 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260923 22:15:10.198405 16959 catalog_manager.cc:1347] Generated new cluster ID: 88dd55e093394f6881df1895285fec40
I20260923 22:15:10.198462 16959 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260923 22:15:10.199452 16953 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 22:15:10.188446 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41638 (local address 127.1.135.254:45507)
0923 22:15:10.188613 (+ 167us) server_negotiation.cc:206] Beginning negotiation
0923 22:15:10.188620 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0923 22:15:10.188644 (+ 24us) server_negotiation.cc:407] Connection header received
0923 22:15:10.189058 (+ 414us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 22:15:10.189065 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 22:15:10.189110 (+ 45us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 22:15:10.189220 (+ 110us) server_negotiation.cc:226] Negotiated authn=SASL
0923 22:15:10.190030 (+ 810us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 22:15:10.190948 (+ 918us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 22:15:10.192024 (+ 1076us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 22:15:10.192465 (+ 441us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 22:15:10.192541 (+ 76us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 22:15:10.195747 (+ 3206us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0923 22:15:10.195751 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0923 22:15:10.195752 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0923 22:15:10.195764 (+ 12us) server_negotiation.cc:922] calling sasl_server_start()
0923 22:15:10.197848 (+ 2084us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0923 22:15:10.198248 (+ 400us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0923 22:15:10.198251 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0923 22:15:10.198251 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0923 22:15:10.198345 (+ 94us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0923 22:15:10.198546 (+ 201us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0923 22:15:10.198547 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0923 22:15:10.198547 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0923 22:15:10.198833 (+ 286us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0923 22:15:10.198972 (+ 139us) server_negotiation.cc:1035] Waiting for connection context
0923 22:15:10.199122 (+ 150us) server_negotiation.cc:299] Negotiation successful
0923 22:15:10.199259 (+ 137us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":115,"thread_start_us":80,"threads_started":1}
I20260923 22:15:10.243108 16959 catalog_manager.cc:1370] Generated new certificate authority record
I20260923 22:15:10.243837 16959 catalog_manager.cc:1504] Loading token signing keys...
I20260923 22:15:10.258317 16959 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 2d2671e82fe14a6fadf679b384125ea3: Generated new TSK 0
I20260923 22:15:10.258530 16959 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.1.135.193@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.1.135.193@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.1.135.193 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskRiQe_a/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790201539513534-1567-0/krb5kdc/kudu_127.1.135.193.keytab.
Entry for principal kudu/127.1.135.193 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskRiQe_a/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790201539513534-1567-0/krb5kdc/kudu_127.1.135.193.keytab.
I20260923 22:15:10.284560 1567 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskRiQe_a/build/release/bin/kudu
/tmp/dist-test-taskRiQe_a/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskRiQe_a/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790201539513534-1567-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskRiQe_a/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790201539513534-1567-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskRiQe_a/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790201539513534-1567-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskRiQe_a/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790201539513534-1567-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.1.135.193:0
--local_ip_for_outbound_sockets=127.1.135.193
--webserver_interface=127.1.135.193
--webserver_port=0
--tserver_master_addrs=127.1.135.254:45507
--keytab_file=/tmp/dist-test-taskRiQe_a/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790201539513534-1567-0/krb5kdc/kudu_127.1.135.193.keytab
--principal=kudu/127.1.135.193
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.1.135.212:34139
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskRiQe_a/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790201539513534-1567-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskRiQe_a/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790201539513534-1567-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskRiQe_a/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790201539513534-1567-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260923 22:15:10.365576 16976 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260923 22:15:10.365778 16976 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260923 22:15:10.365801 16976 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260923 22:15:10.366878 16976 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260923 22:15:10.366914 16976 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260923 22:15:10.366945 16976 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.1.135.193
I20260923 22:15:10.368364 16976 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.1.135.212:34139
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskRiQe_a/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790201539513534-1567-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskRiQe_a/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790201539513534-1567-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskRiQe_a/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790201539513534-1567-0/krb5kdc/kudu_127.1.135.193.keytab
--principal=kudu/127.1.135.193
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.1.135.193:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskRiQe_a/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790201539513534-1567-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.1.135.193
--webserver_port=0
--tserver_master_addrs=127.1.135.254:45507
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.16976
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.1.135.193
--log_dir=/tmp/dist-test-taskRiQe_a/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790201539513534-1567-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 23 Sep 2026 20:53:37 UTC on ccycloud.quasar-rkviyx.root.comops.site
build id 4668083
I20260923 22:15:10.368554 16976 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260923 22:15:10.368788 16976 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260923 22:15:10.373653 16981 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260923 22:15:10.373684 16982 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260923 22:15:10.373842 16976 server_base.cc:1034] running on GCE node
W20260923 22:15:10.373661 16984 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260923 22:15:10.374153 16976 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260923 22:15:10.374395 16976 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260923 22:15:10.375535 16976 hybrid_clock.cc:648] HybridClock initialized: now 1790201710375523 us; error 29 us; skew 500 ppm
Sep 23 22:15:10 dist-test-slave-s2fj krb5kdc[16869](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790201710, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.135.193@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260923 22:15:10.378086 16976 init.cc:377] Logged in from keytab as kudu/127.1.135.193@KRBTEST.COM (short username kudu)
I20260923 22:15:10.378454 16976 webserver.cc:460] Webserver started at http://127.1.135.193:35971/ using document root <none> and password file <none>
I20260923 22:15:10.378639 16976 fs_manager.cc:359] Metadata directory not provided
I20260923 22:15:10.378679 16976 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260923 22:15:10.378764 16976 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260923 22:15:10.379781 16976 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskRiQe_a/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790201539513534-1567-0/minicluster-data/ts-0/data/instance:
uuid: "de2e7140d2c742e6983e06beafcffd7a"
format_stamp: "Formatted at 2026-09-23 22:15:10 on dist-test-slave-s2fj"
server_key: "c01fe50084c85daf4a825bec04f9a216"
server_key_iv: "b82882840d6cb85ee11076a203f25d2f"
server_key_version: "encryptionkey@0"
I20260923 22:15:10.380190 16976 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskRiQe_a/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790201539513534-1567-0/minicluster-data/ts-0/wal/instance:
uuid: "de2e7140d2c742e6983e06beafcffd7a"
format_stamp: "Formatted at 2026-09-23 22:15:10 on dist-test-slave-s2fj"
server_key: "c01fe50084c85daf4a825bec04f9a216"
server_key_iv: "b82882840d6cb85ee11076a203f25d2f"
server_key_version: "encryptionkey@0"
I20260923 22:15:10.381458 16976 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.001s sys 0.000s
I20260923 22:15:10.382210 16991 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260923 22:15:10.382427 16976 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260923 22:15:10.382545 16976 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskRiQe_a/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790201539513534-1567-0/minicluster-data/ts-0/data,/tmp/dist-test-taskRiQe_a/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790201539513534-1567-0/minicluster-data/ts-0/wal
uuid: "de2e7140d2c742e6983e06beafcffd7a"
format_stamp: "Formatted at 2026-09-23 22:15:10 on dist-test-slave-s2fj"
server_key: "c01fe50084c85daf4a825bec04f9a216"
server_key_iv: "b82882840d6cb85ee11076a203f25d2f"
server_key_version: "encryptionkey@0"
I20260923 22:15:10.382623 16976 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskRiQe_a/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790201539513534-1567-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskRiQe_a/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790201539513534-1567-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskRiQe_a/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790201539513534-1567-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260923 22:15:10.399518 16976 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260923 22:15:10.401623 16976 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260923 22:15:10.401808 16976 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260923 22:15:10.402050 16976 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260923 22:15:10.402382 16976 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260923 22:15:10.402441 16976 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260923 22:15:10.402486 16976 ts_tablet_manager.cc:610] Registered 0 tablets
I20260923 22:15:10.402513 16976 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260923 22:15:10.408568 16976 rpc_server.cc:307] RPC server started. Bound to: 127.1.135.193:46309
I20260923 22:15:10.408612 17104 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.1.135.193:46309 every 8 connection(s)
I20260923 22:15:10.408957 16976 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskRiQe_a/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790201539513534-1567-0/minicluster-data/ts-0/data/info.pb
I20260923 22:15:10.409325 1567 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskRiQe_a/build/release/bin/kudu as pid 16976
I20260923 22:15:10.409443 1567 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskRiQe_a/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790201539513534-1567-0/minicluster-data/ts-0/wal/instance
I20260923 22:15:10.409687 1567 external_mini_cluster.cc:1428] Setting key ea35cf2aaee2778560a871c62ed3883c
Sep 23 22:15:10 dist-test-slave-s2fj krb5kdc[16869](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790201710, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.135.193@KRBTEST.COM for kudu/127.1.135.254@KRBTEST.COM
WARNING: no policy specified for kudu/127.1.135.194@KRBTEST.COM; defaulting to no policy
I20260923 22:15:10.419339 16953 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 22:15:10.409775 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.135.193:59995 (local address 127.1.135.254:45507)
0923 22:15:10.409877 (+ 102us) server_negotiation.cc:206] Beginning negotiation
0923 22:15:10.409880 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0923 22:15:10.410082 (+ 202us) server_negotiation.cc:407] Connection header received
0923 22:15:10.410674 (+ 592us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 22:15:10.410675 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 22:15:10.410688 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 22:15:10.410756 (+ 68us) server_negotiation.cc:226] Negotiated authn=SASL
0923 22:15:10.411749 (+ 993us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 22:15:10.412347 (+ 598us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 22:15:10.413350 (+ 1003us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 22:15:10.413583 (+ 233us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 22:15:10.413672 (+ 89us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 22:15:10.416241 (+ 2569us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0923 22:15:10.416242 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0923 22:15:10.416243 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0923 22:15:10.416251 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0923 22:15:10.417875 (+ 1624us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0923 22:15:10.418337 (+ 462us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0923 22:15:10.418338 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0923 22:15:10.418339 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0923 22:15:10.418402 (+ 63us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0923 22:15:10.418661 (+ 259us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0923 22:15:10.418663 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0923 22:15:10.418663 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0923 22:15:10.418787 (+ 124us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0923 22:15:10.418868 (+ 81us) server_negotiation.cc:1035] Waiting for connection context
0923 22:15:10.419146 (+ 278us) server_negotiation.cc:299] Negotiation successful
0923 22:15:10.419240 (+ 94us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":73}
I20260923 22:15:10.419535 17107 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 22:15:10.409756 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.135.254:45507 (local address 127.1.135.193:59995)
0923 22:15:10.409968 (+ 212us) negotiation.cc:107] Waiting for socket to connect
0923 22:15:10.410024 (+ 56us) client_negotiation.cc:174] Beginning negotiation
0923 22:15:10.410569 (+ 545us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0923 22:15:10.410816 (+ 247us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0923 22:15:10.410819 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0923 22:15:10.411409 (+ 590us) client_negotiation.cc:189] Negotiated authn=SASL
0923 22:15:10.411647 (+ 238us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 22:15:10.411652 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 22:15:10.412441 (+ 789us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 22:15:10.412443 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 22:15:10.413253 (+ 810us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 22:15:10.413255 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 22:15:10.413683 (+ 428us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 22:15:10.413684 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 22:15:10.413819 (+ 135us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 22:15:10.414418 (+ 599us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0923 22:15:10.414428 (+ 10us) client_negotiation.cc:656] calling sasl_client_start()
0923 22:15:10.416146 (+ 1718us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0923 22:15:10.417970 (+ 1824us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0923 22:15:10.417977 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0923 22:15:10.417979 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0923 22:15:10.418250 (+ 271us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0923 22:15:10.418474 (+ 224us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0923 22:15:10.418475 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0923 22:15:10.418475 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0923 22:15:10.418590 (+ 115us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0923 22:15:10.418858 (+ 268us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0923 22:15:10.418865 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0923 22:15:10.419013 (+ 148us) client_negotiation.cc:769] Sending connection context
0923 22:15:10.419126 (+ 113us) client_negotiation.cc:240] Negotiation successful
0923 22:15:10.419292 (+ 166us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":161,"thread_start_us":97,"threads_started":1}
I20260923 22:15:10.420086 17105 heartbeater.cc:347] Connected to a master server at 127.1.135.254:45507
I20260923 22:15:10.420208 17105 heartbeater.cc:464] Registering TS with master...
I20260923 22:15:10.420400 17105 heartbeater.cc:511] Master 127.1.135.254:45507 requested a full tablet report, sending...
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.1.135.194@KRBTEST.COM" created.
I20260923 22:15:10.420984 16914 ts_manager.cc:194] Registered new tserver with Master: de2e7140d2c742e6983e06beafcffd7a (127.1.135.193:46309)
I20260923 22:15:10.421589 16914 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.1.135.193@KRBTEST.COM'} at 127.1.135.193:59995
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.1.135.194 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskRiQe_a/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790201539513534-1567-0/krb5kdc/kudu_127.1.135.194.keytab.
Entry for principal kudu/127.1.135.194 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskRiQe_a/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790201539513534-1567-0/krb5kdc/kudu_127.1.135.194.keytab.
I20260923 22:15:10.435078 1567 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskRiQe_a/build/release/bin/kudu
/tmp/dist-test-taskRiQe_a/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskRiQe_a/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790201539513534-1567-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskRiQe_a/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790201539513534-1567-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskRiQe_a/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790201539513534-1567-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskRiQe_a/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790201539513534-1567-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.1.135.194:0
--local_ip_for_outbound_sockets=127.1.135.194
--webserver_interface=127.1.135.194
--webserver_port=0
--tserver_master_addrs=127.1.135.254:45507
--keytab_file=/tmp/dist-test-taskRiQe_a/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790201539513534-1567-0/krb5kdc/kudu_127.1.135.194.keytab
--principal=kudu/127.1.135.194
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.1.135.212:34139
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskRiQe_a/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790201539513534-1567-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskRiQe_a/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790201539513534-1567-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskRiQe_a/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790201539513534-1567-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260923 22:15:10.518141 17110 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260923 22:15:10.518357 17110 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260923 22:15:10.518381 17110 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260923 22:15:10.519629 17110 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260923 22:15:10.519671 17110 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260923 22:15:10.519698 17110 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.1.135.194
I20260923 22:15:10.521189 17110 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.1.135.212:34139
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskRiQe_a/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790201539513534-1567-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskRiQe_a/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790201539513534-1567-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskRiQe_a/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790201539513534-1567-0/krb5kdc/kudu_127.1.135.194.keytab
--principal=kudu/127.1.135.194
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.1.135.194:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskRiQe_a/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790201539513534-1567-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.1.135.194
--webserver_port=0
--tserver_master_addrs=127.1.135.254:45507
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.17110
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.1.135.194
--log_dir=/tmp/dist-test-taskRiQe_a/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790201539513534-1567-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 23 Sep 2026 20:53:37 UTC on ccycloud.quasar-rkviyx.root.comops.site
build id 4668083
I20260923 22:15:10.521448 17110 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260923 22:15:10.521687 17110 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260923 22:15:10.526728 17116 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260923 22:15:10.526728 17115 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260923 22:15:10.526748 17118 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260923 22:15:10.526964 17110 server_base.cc:1034] running on GCE node
I20260923 22:15:10.527170 17110 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260923 22:15:10.527462 17110 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260923 22:15:10.528625 17110 hybrid_clock.cc:648] HybridClock initialized: now 1790201710528597 us; error 38 us; skew 500 ppm
Sep 23 22:15:10 dist-test-slave-s2fj krb5kdc[16869](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790201710, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.135.194@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260923 22:15:10.536929 17110 init.cc:377] Logged in from keytab as kudu/127.1.135.194@KRBTEST.COM (short username kudu)
I20260923 22:15:10.537503 17110 webserver.cc:460] Webserver started at http://127.1.135.194:34741/ using document root <none> and password file <none>
I20260923 22:15:10.537774 17110 fs_manager.cc:359] Metadata directory not provided
I20260923 22:15:10.537846 17110 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260923 22:15:10.537998 17110 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260923 22:15:10.539099 17110 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskRiQe_a/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790201539513534-1567-0/minicluster-data/ts-1/data/instance:
uuid: "1d61ec2a894f4c76afcd904fdb097bfc"
format_stamp: "Formatted at 2026-09-23 22:15:10 on dist-test-slave-s2fj"
server_key: "61fd161f646230d3a8ca49c7ce067153"
server_key_iv: "55c6a41dda98e464a8572f3994d180a6"
server_key_version: "encryptionkey@0"
I20260923 22:15:10.539472 17110 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskRiQe_a/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790201539513534-1567-0/minicluster-data/ts-1/wal/instance:
uuid: "1d61ec2a894f4c76afcd904fdb097bfc"
format_stamp: "Formatted at 2026-09-23 22:15:10 on dist-test-slave-s2fj"
server_key: "61fd161f646230d3a8ca49c7ce067153"
server_key_iv: "55c6a41dda98e464a8572f3994d180a6"
server_key_version: "encryptionkey@0"
I20260923 22:15:10.540890 17110 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.001s
I20260923 22:15:10.541648 17125 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260923 22:15:10.541837 17110 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260923 22:15:10.541946 17110 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskRiQe_a/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790201539513534-1567-0/minicluster-data/ts-1/data,/tmp/dist-test-taskRiQe_a/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790201539513534-1567-0/minicluster-data/ts-1/wal
uuid: "1d61ec2a894f4c76afcd904fdb097bfc"
format_stamp: "Formatted at 2026-09-23 22:15:10 on dist-test-slave-s2fj"
server_key: "61fd161f646230d3a8ca49c7ce067153"
server_key_iv: "55c6a41dda98e464a8572f3994d180a6"
server_key_version: "encryptionkey@0"
I20260923 22:15:10.542019 17110 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskRiQe_a/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790201539513534-1567-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskRiQe_a/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790201539513534-1567-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskRiQe_a/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790201539513534-1567-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260923 22:15:10.557976 17110 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260923 22:15:10.559830 17110 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260923 22:15:10.559996 17110 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260923 22:15:10.560269 17110 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260923 22:15:10.560593 17110 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260923 22:15:10.560645 17110 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260923 22:15:10.560686 17110 ts_tablet_manager.cc:610] Registered 0 tablets
I20260923 22:15:10.560716 17110 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260923 22:15:10.566874 17110 rpc_server.cc:307] RPC server started. Bound to: 127.1.135.194:33393
I20260923 22:15:10.566915 17238 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.1.135.194:33393 every 8 connection(s)
I20260923 22:15:10.567279 17110 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskRiQe_a/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790201539513534-1567-0/minicluster-data/ts-1/data/info.pb
I20260923 22:15:10.570147 1567 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskRiQe_a/build/release/bin/kudu as pid 17110
I20260923 22:15:10.570271 1567 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskRiQe_a/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790201539513534-1567-0/minicluster-data/ts-1/wal/instance
I20260923 22:15:10.570537 1567 external_mini_cluster.cc:1428] Setting key 4bd73c354e481af982e063ede42c5b79
Sep 23 22:15:10 dist-test-slave-s2fj krb5kdc[16869](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790201710, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.135.194@KRBTEST.COM for kudu/127.1.135.254@KRBTEST.COM
I20260923 22:15:10.577368 16953 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 22:15:10.568018 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.135.194:34249 (local address 127.1.135.254:45507)
0923 22:15:10.568111 (+ 93us) server_negotiation.cc:206] Beginning negotiation
0923 22:15:10.568113 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0923 22:15:10.568292 (+ 179us) server_negotiation.cc:407] Connection header received
0923 22:15:10.568748 (+ 456us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 22:15:10.568749 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 22:15:10.568759 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 22:15:10.568810 (+ 51us) server_negotiation.cc:226] Negotiated authn=SASL
0923 22:15:10.569709 (+ 899us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 22:15:10.570231 (+ 522us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 22:15:10.571666 (+ 1435us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 22:15:10.571851 (+ 185us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 22:15:10.571920 (+ 69us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 22:15:10.574254 (+ 2334us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0923 22:15:10.574256 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0923 22:15:10.574256 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0923 22:15:10.574264 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0923 22:15:10.575905 (+ 1641us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0923 22:15:10.576357 (+ 452us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0923 22:15:10.576358 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0923 22:15:10.576359 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0923 22:15:10.576424 (+ 65us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0923 22:15:10.576666 (+ 242us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0923 22:15:10.576667 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0923 22:15:10.576667 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0923 22:15:10.576824 (+ 157us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0923 22:15:10.576907 (+ 83us) server_negotiation.cc:1035] Waiting for connection context
0923 22:15:10.577168 (+ 261us) server_negotiation.cc:299] Negotiation successful
0923 22:15:10.577251 (+ 83us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":59}
I20260923 22:15:10.577374 17241 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 22:15:10.567966 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.135.254:45507 (local address 127.1.135.194:34249)
0923 22:15:10.568161 (+ 195us) negotiation.cc:107] Waiting for socket to connect
0923 22:15:10.568199 (+ 38us) client_negotiation.cc:174] Beginning negotiation
0923 22:15:10.568669 (+ 470us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0923 22:15:10.568863 (+ 194us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0923 22:15:10.568866 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0923 22:15:10.569444 (+ 578us) client_negotiation.cc:189] Negotiated authn=SASL
0923 22:15:10.569633 (+ 189us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 22:15:10.569637 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 22:15:10.570328 (+ 691us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 22:15:10.570329 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 22:15:10.571583 (+ 1254us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 22:15:10.571586 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 22:15:10.571953 (+ 367us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 22:15:10.571954 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 22:15:10.572132 (+ 178us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 22:15:10.572637 (+ 505us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0923 22:15:10.572648 (+ 11us) client_negotiation.cc:656] calling sasl_client_start()
0923 22:15:10.574159 (+ 1511us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0923 22:15:10.575997 (+ 1838us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0923 22:15:10.576002 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0923 22:15:10.576004 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0923 22:15:10.576285 (+ 281us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0923 22:15:10.576496 (+ 211us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0923 22:15:10.576497 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0923 22:15:10.576497 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0923 22:15:10.576590 (+ 93us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0923 22:15:10.576893 (+ 303us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0923 22:15:10.576897 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0923 22:15:10.577025 (+ 128us) client_negotiation.cc:769] Sending connection context
0923 22:15:10.577099 (+ 74us) client_negotiation.cc:240] Negotiation successful
0923 22:15:10.577215 (+ 116us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":148,"thread_start_us":109,"threads_started":1}
I20260923 22:15:10.577958 17239 heartbeater.cc:347] Connected to a master server at 127.1.135.254:45507
I20260923 22:15:10.578050 17239 heartbeater.cc:464] Registering TS with master...
I20260923 22:15:10.578239 17239 heartbeater.cc:511] Master 127.1.135.254:45507 requested a full tablet report, sending...
I20260923 22:15:10.578598 16914 ts_manager.cc:194] Registered new tserver with Master: 1d61ec2a894f4c76afcd904fdb097bfc (127.1.135.194:33393)
I20260923 22:15:10.578989 16914 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.1.135.194@KRBTEST.COM'} at 127.1.135.194:34249
WARNING: no policy specified for kudu/127.1.135.195@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.1.135.195@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.1.135.195 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskRiQe_a/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790201539513534-1567-0/krb5kdc/kudu_127.1.135.195.keytab.
Entry for principal kudu/127.1.135.195 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskRiQe_a/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790201539513534-1567-0/krb5kdc/kudu_127.1.135.195.keytab.
I20260923 22:15:10.596786 1567 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskRiQe_a/build/release/bin/kudu
/tmp/dist-test-taskRiQe_a/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskRiQe_a/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790201539513534-1567-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskRiQe_a/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790201539513534-1567-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskRiQe_a/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790201539513534-1567-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskRiQe_a/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790201539513534-1567-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.1.135.195:0
--local_ip_for_outbound_sockets=127.1.135.195
--webserver_interface=127.1.135.195
--webserver_port=0
--tserver_master_addrs=127.1.135.254:45507
--keytab_file=/tmp/dist-test-taskRiQe_a/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790201539513534-1567-0/krb5kdc/kudu_127.1.135.195.keytab
--principal=kudu/127.1.135.195
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.1.135.212:34139
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskRiQe_a/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790201539513534-1567-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskRiQe_a/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790201539513534-1567-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskRiQe_a/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790201539513534-1567-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260923 22:15:10.676450 17244 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260923 22:15:10.676669 17244 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260923 22:15:10.676704 17244 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260923 22:15:10.677812 17244 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260923 22:15:10.677881 17244 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260923 22:15:10.677927 17244 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.1.135.195
I20260923 22:15:10.679363 17244 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.1.135.212:34139
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskRiQe_a/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790201539513534-1567-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskRiQe_a/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790201539513534-1567-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskRiQe_a/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790201539513534-1567-0/krb5kdc/kudu_127.1.135.195.keytab
--principal=kudu/127.1.135.195
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.1.135.195:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskRiQe_a/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790201539513534-1567-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.1.135.195
--webserver_port=0
--tserver_master_addrs=127.1.135.254:45507
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.17244
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.1.135.195
--log_dir=/tmp/dist-test-taskRiQe_a/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790201539513534-1567-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 23 Sep 2026 20:53:37 UTC on ccycloud.quasar-rkviyx.root.comops.site
build id 4668083
I20260923 22:15:10.679618 17244 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260923 22:15:10.679840 17244 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260923 22:15:10.684758 17250 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260923 22:15:10.684763 17252 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260923 22:15:10.684854 17249 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260923 22:15:10.684938 17244 server_base.cc:1034] running on GCE node
I20260923 22:15:10.685171 17244 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260923 22:15:10.685401 17244 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260923 22:15:10.686537 17244 hybrid_clock.cc:648] HybridClock initialized: now 1790201710686525 us; error 27 us; skew 500 ppm
Sep 23 22:15:10 dist-test-slave-s2fj krb5kdc[16869](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790201710, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.135.195@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260923 22:15:10.689038 17244 init.cc:377] Logged in from keytab as kudu/127.1.135.195@KRBTEST.COM (short username kudu)
I20260923 22:15:10.689546 17244 webserver.cc:460] Webserver started at http://127.1.135.195:37405/ using document root <none> and password file <none>
I20260923 22:15:10.689781 17244 fs_manager.cc:359] Metadata directory not provided
I20260923 22:15:10.689828 17244 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260923 22:15:10.689988 17244 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260923 22:15:10.691054 17244 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskRiQe_a/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790201539513534-1567-0/minicluster-data/ts-2/data/instance:
uuid: "43f20a4dc81f48e3bdb7a2281b8505c2"
format_stamp: "Formatted at 2026-09-23 22:15:10 on dist-test-slave-s2fj"
server_key: "c319eb09e0c64ba992b15b7f068e6c56"
server_key_iv: "df4df41ccb3bbb369e9bb5805ddba31c"
server_key_version: "encryptionkey@0"
I20260923 22:15:10.691418 17244 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskRiQe_a/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790201539513534-1567-0/minicluster-data/ts-2/wal/instance:
uuid: "43f20a4dc81f48e3bdb7a2281b8505c2"
format_stamp: "Formatted at 2026-09-23 22:15:10 on dist-test-slave-s2fj"
server_key: "c319eb09e0c64ba992b15b7f068e6c56"
server_key_iv: "df4df41ccb3bbb369e9bb5805ddba31c"
server_key_version: "encryptionkey@0"
I20260923 22:15:10.692764 17244 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260923 22:15:10.693463 17259 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260923 22:15:10.693657 17244 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260923 22:15:10.693734 17244 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskRiQe_a/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790201539513534-1567-0/minicluster-data/ts-2/data,/tmp/dist-test-taskRiQe_a/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790201539513534-1567-0/minicluster-data/ts-2/wal
uuid: "43f20a4dc81f48e3bdb7a2281b8505c2"
format_stamp: "Formatted at 2026-09-23 22:15:10 on dist-test-slave-s2fj"
server_key: "c319eb09e0c64ba992b15b7f068e6c56"
server_key_iv: "df4df41ccb3bbb369e9bb5805ddba31c"
server_key_version: "encryptionkey@0"
I20260923 22:15:10.693787 17244 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskRiQe_a/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790201539513534-1567-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskRiQe_a/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790201539513534-1567-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskRiQe_a/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790201539513534-1567-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260923 22:15:10.712678 17244 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260923 22:15:10.714965 17244 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260923 22:15:10.715076 17244 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260923 22:15:10.715252 17244 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260923 22:15:10.715497 17244 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260923 22:15:10.715525 17244 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260923 22:15:10.715547 17244 ts_tablet_manager.cc:610] Registered 0 tablets
I20260923 22:15:10.715559 17244 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260923 22:15:10.721012 17244 rpc_server.cc:307] RPC server started. Bound to: 127.1.135.195:44883
I20260923 22:15:10.721088 17372 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.1.135.195:44883 every 8 connection(s)
I20260923 22:15:10.721392 17244 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskRiQe_a/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790201539513534-1567-0/minicluster-data/ts-2/data/info.pb
I20260923 22:15:10.721875 1567 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskRiQe_a/build/release/bin/kudu as pid 17244
I20260923 22:15:10.721998 1567 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskRiQe_a/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790201539513534-1567-0/minicluster-data/ts-2/wal/instance
I20260923 22:15:10.722213 1567 external_mini_cluster.cc:1428] Setting key e933c123caec6183b89b71552ca4467c
Sep 23 22:15:10 dist-test-slave-s2fj krb5kdc[16869](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790201710, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.135.195@KRBTEST.COM for kudu/127.1.135.254@KRBTEST.COM
I20260923 22:15:10.730623 16953 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 22:15:10.722124 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.135.195:43673 (local address 127.1.135.254:45507)
0923 22:15:10.722175 (+ 51us) server_negotiation.cc:206] Beginning negotiation
0923 22:15:10.722177 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0923 22:15:10.722400 (+ 223us) server_negotiation.cc:407] Connection header received
0923 22:15:10.723069 (+ 669us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 22:15:10.723070 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 22:15:10.723081 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 22:15:10.723144 (+ 63us) server_negotiation.cc:226] Negotiated authn=SASL
0923 22:15:10.724055 (+ 911us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 22:15:10.724631 (+ 576us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 22:15:10.725583 (+ 952us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 22:15:10.725762 (+ 179us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 22:15:10.725814 (+ 52us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 22:15:10.727843 (+ 2029us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0923 22:15:10.727844 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0923 22:15:10.727844 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0923 22:15:10.727852 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0923 22:15:10.729279 (+ 1427us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0923 22:15:10.729701 (+ 422us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0923 22:15:10.729703 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0923 22:15:10.729703 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0923 22:15:10.729762 (+ 59us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0923 22:15:10.730015 (+ 253us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0923 22:15:10.730016 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0923 22:15:10.730016 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0923 22:15:10.730126 (+ 110us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0923 22:15:10.730194 (+ 68us) server_negotiation.cc:1035] Waiting for connection context
0923 22:15:10.730460 (+ 266us) server_negotiation.cc:299] Negotiation successful
0923 22:15:10.730532 (+ 72us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":26}
I20260923 22:15:10.730811 17375 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 22:15:10.722105 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.135.254:45507 (local address 127.1.135.195:43673)
0923 22:15:10.722263 (+ 158us) negotiation.cc:107] Waiting for socket to connect
0923 22:15:10.722300 (+ 37us) client_negotiation.cc:174] Beginning negotiation
0923 22:15:10.722964 (+ 664us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0923 22:15:10.723215 (+ 251us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0923 22:15:10.723219 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0923 22:15:10.723770 (+ 551us) client_negotiation.cc:189] Negotiated authn=SASL
0923 22:15:10.723976 (+ 206us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 22:15:10.723981 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 22:15:10.724727 (+ 746us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 22:15:10.724728 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 22:15:10.725508 (+ 780us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 22:15:10.725510 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 22:15:10.725834 (+ 324us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 22:15:10.725835 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 22:15:10.725930 (+ 95us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 22:15:10.726358 (+ 428us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0923 22:15:10.726366 (+ 8us) client_negotiation.cc:656] calling sasl_client_start()
0923 22:15:10.727743 (+ 1377us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0923 22:15:10.729392 (+ 1649us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0923 22:15:10.729399 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0923 22:15:10.729401 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0923 22:15:10.729614 (+ 213us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0923 22:15:10.729827 (+ 213us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0923 22:15:10.729827 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0923 22:15:10.729828 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0923 22:15:10.729944 (+ 116us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0923 22:15:10.730197 (+ 253us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0923 22:15:10.730203 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0923 22:15:10.730334 (+ 131us) client_negotiation.cc:769] Sending connection context
0923 22:15:10.730438 (+ 104us) client_negotiation.cc:240] Negotiation successful
0923 22:15:10.730590 (+ 152us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":107,"thread_start_us":80,"threads_started":1}
I20260923 22:15:10.731293 17373 heartbeater.cc:347] Connected to a master server at 127.1.135.254:45507
I20260923 22:15:10.731402 17373 heartbeater.cc:464] Registering TS with master...
I20260923 22:15:10.731570 17373 heartbeater.cc:511] Master 127.1.135.254:45507 requested a full tablet report, sending...
I20260923 22:15:10.731913 16914 ts_manager.cc:194] Registered new tserver with Master: 43f20a4dc81f48e3bdb7a2281b8505c2 (127.1.135.195:44883)
I20260923 22:15:10.732314 16914 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.1.135.195@KRBTEST.COM'} at 127.1.135.195:43673
I20260923 22:15:10.733955 1567 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260923 22:15:10.740660 16953 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 22:15:10.735200 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41644 (local address 127.1.135.254:45507)
0923 22:15:10.735250 (+ 50us) server_negotiation.cc:206] Beginning negotiation
0923 22:15:10.735251 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0923 22:15:10.735354 (+ 103us) server_negotiation.cc:407] Connection header received
0923 22:15:10.735408 (+ 54us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 22:15:10.735409 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 22:15:10.735416 (+ 7us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 22:15:10.735467 (+ 51us) server_negotiation.cc:226] Negotiated authn=SASL
0923 22:15:10.735976 (+ 509us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 22:15:10.736481 (+ 505us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 22:15:10.737157 (+ 676us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 22:15:10.737323 (+ 166us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 22:15:10.737369 (+ 46us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 22:15:10.738449 (+ 1080us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0923 22:15:10.738451 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0923 22:15:10.738451 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0923 22:15:10.738457 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0923 22:15:10.739758 (+ 1301us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0923 22:15:10.740038 (+ 280us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0923 22:15:10.740039 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0923 22:15:10.740039 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0923 22:15:10.740136 (+ 97us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0923 22:15:10.740266 (+ 130us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0923 22:15:10.740267 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0923 22:15:10.740267 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0923 22:15:10.740386 (+ 119us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0923 22:15:10.740461 (+ 75us) server_negotiation.cc:1035] Waiting for connection context
0923 22:15:10.740552 (+ 91us) server_negotiation.cc:299] Negotiation successful
0923 22:15:10.740604 (+ 52us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":32}
I20260923 22:15:10.742290 16914 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:41644:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260923 22:15:10.743852 16914 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260923 22:15:10.750284 17384 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 22:15:10.746244 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38252 (local address 127.1.135.195:44883)
0923 22:15:10.746593 (+ 349us) server_negotiation.cc:206] Beginning negotiation
0923 22:15:10.746599 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0923 22:15:10.746609 (+ 10us) server_negotiation.cc:407] Connection header received
0923 22:15:10.746650 (+ 41us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 22:15:10.746652 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 22:15:10.746669 (+ 17us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 22:15:10.746751 (+ 82us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0923 22:15:10.746921 (+ 170us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 22:15:10.748045 (+ 1124us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 22:15:10.749186 (+ 1141us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 22:15:10.749882 (+ 696us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 22:15:10.749961 (+ 79us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 22:15:10.750021 (+ 60us) server_negotiation.cc:1035] Waiting for connection context
0923 22:15:10.750164 (+ 143us) server_negotiation.cc:299] Negotiation successful
0923 22:15:10.750203 (+ 39us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":317,"thread_start_us":80,"threads_started":1}
I20260923 22:15:10.750298 17385 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 22:15:10.746303 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.135.194:33393 (local address 127.0.0.1:50384)
0923 22:15:10.746486 (+ 183us) negotiation.cc:107] Waiting for socket to connect
0923 22:15:10.746497 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0923 22:15:10.746550 (+ 53us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0923 22:15:10.747011 (+ 461us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0923 22:15:10.747011 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0923 22:15:10.747014 (+ 3us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0923 22:15:10.747062 (+ 48us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 22:15:10.747063 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 22:15:10.747925 (+ 862us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 22:15:10.747926 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 22:15:10.749213 (+ 1287us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 22:15:10.749215 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 22:15:10.750041 (+ 826us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 22:15:10.750042 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 22:15:10.750089 (+ 47us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 22:15:10.750092 (+ 3us) client_negotiation.cc:769] Sending connection context
0923 22:15:10.750141 (+ 49us) client_negotiation.cc:240] Negotiation successful
0923 22:15:10.750208 (+ 67us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":171,"thread_start_us":55,"threads_started":1}
I20260923 22:15:10.750298 17382 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 22:15:10.746100 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.135.195:44883 (local address 127.0.0.1:38252)
0923 22:15:10.746386 (+ 286us) negotiation.cc:107] Waiting for socket to connect
0923 22:15:10.746399 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0923 22:15:10.746462 (+ 63us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0923 22:15:10.746769 (+ 307us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0923 22:15:10.746772 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0923 22:15:10.746774 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0923 22:15:10.746835 (+ 61us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 22:15:10.746837 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 22:15:10.748203 (+ 1366us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 22:15:10.748204 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 22:15:10.749104 (+ 900us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 22:15:10.749106 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 22:15:10.749967 (+ 861us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 22:15:10.749968 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 22:15:10.750051 (+ 83us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 22:15:10.750066 (+ 15us) client_negotiation.cc:769] Sending connection context
0923 22:15:10.750141 (+ 75us) client_negotiation.cc:240] Negotiation successful
0923 22:15:10.750208 (+ 67us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":276,"mutex_wait_us":38,"thread_start_us":49,"threads_started":1}
I20260923 22:15:10.750819 17386 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 22:15:10.746491 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50384 (local address 127.1.135.194:33393)
0923 22:15:10.746653 (+ 162us) server_negotiation.cc:206] Beginning negotiation
0923 22:15:10.746657 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0923 22:15:10.746667 (+ 10us) server_negotiation.cc:407] Connection header received
0923 22:15:10.746704 (+ 37us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 22:15:10.746706 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 22:15:10.746720 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 22:15:10.746922 (+ 202us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0923 22:15:10.747141 (+ 219us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 22:15:10.747815 (+ 674us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 22:15:10.749289 (+ 1474us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 22:15:10.749969 (+ 680us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 22:15:10.750036 (+ 67us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 22:15:10.750083 (+ 47us) server_negotiation.cc:1035] Waiting for connection context
0923 22:15:10.750710 (+ 627us) server_negotiation.cc:299] Negotiation successful
0923 22:15:10.750761 (+ 51us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":126,"thread_start_us":82,"threads_started":1}
I20260923 22:15:10.751175 17307 tablet_service.cc:1467] Processing CreateTablet for tablet 95b5c4cf21e84292bd035af291af6641 (DEFAULT_TABLE table=test-table [id=2348099ad6364419b484d26fbd3ac660]), partition=RANGE (key) PARTITION UNBOUNDED
I20260923 22:15:10.751410 17381 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 22:15:10.746056 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.135.193:46309 (local address 127.0.0.1:47414)
0923 22:15:10.746354 (+ 298us) negotiation.cc:107] Waiting for socket to connect
0923 22:15:10.746397 (+ 43us) client_negotiation.cc:174] Beginning negotiation
0923 22:15:10.746466 (+ 69us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0923 22:15:10.746812 (+ 346us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0923 22:15:10.746813 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0923 22:15:10.746814 (+ 1us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0923 22:15:10.746851 (+ 37us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 22:15:10.746852 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 22:15:10.748154 (+ 1302us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 22:15:10.748155 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 22:15:10.749228 (+ 1073us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 22:15:10.749230 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 22:15:10.751210 (+ 1980us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 22:15:10.751211 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 22:15:10.751272 (+ 61us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 22:15:10.751276 (+ 4us) client_negotiation.cc:769] Sending connection context
0923 22:15:10.751305 (+ 29us) client_negotiation.cc:240] Negotiation successful
0923 22:15:10.751345 (+ 40us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":275,"mutex_wait_us":6,"thread_start_us":84,"threads_started":1}
I20260923 22:15:10.751371 17173 tablet_service.cc:1467] Processing CreateTablet for tablet 95b5c4cf21e84292bd035af291af6641 (DEFAULT_TABLE table=test-table [id=2348099ad6364419b484d26fbd3ac660]), partition=RANGE (key) PARTITION UNBOUNDED
I20260923 22:15:10.751549 17307 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 95b5c4cf21e84292bd035af291af6641. 1 dirs total, 0 dirs full, 0 dirs failed
I20260923 22:15:10.751607 17173 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 95b5c4cf21e84292bd035af291af6641. 1 dirs total, 0 dirs full, 0 dirs failed
I20260923 22:15:10.752009 17383 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 22:15:10.746167 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47414 (local address 127.1.135.193:46309)
0923 22:15:10.746366 (+ 199us) server_negotiation.cc:206] Beginning negotiation
0923 22:15:10.746371 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0923 22:15:10.746634 (+ 263us) server_negotiation.cc:407] Connection header received
0923 22:15:10.746684 (+ 50us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 22:15:10.746685 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 22:15:10.746698 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 22:15:10.746933 (+ 235us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0923 22:15:10.747013 (+ 80us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 22:15:10.748054 (+ 1041us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 22:15:10.750144 (+ 2090us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 22:15:10.751107 (+ 963us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 22:15:10.751780 (+ 673us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 22:15:10.751840 (+ 60us) server_negotiation.cc:1035] Waiting for connection context
0923 22:15:10.751865 (+ 25us) server_negotiation.cc:299] Negotiation successful
0923 22:15:10.751910 (+ 45us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":158,"thread_start_us":92,"threads_started":1}
I20260923 22:15:10.752709 17039 tablet_service.cc:1467] Processing CreateTablet for tablet 95b5c4cf21e84292bd035af291af6641 (DEFAULT_TABLE table=test-table [id=2348099ad6364419b484d26fbd3ac660]), partition=RANGE (key) PARTITION UNBOUNDED
I20260923 22:15:10.753026 17039 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 95b5c4cf21e84292bd035af291af6641. 1 dirs total, 0 dirs full, 0 dirs failed
I20260923 22:15:10.754201 17387 tablet_bootstrap.cc:492] T 95b5c4cf21e84292bd035af291af6641 P 1d61ec2a894f4c76afcd904fdb097bfc: Bootstrap starting.
I20260923 22:15:10.754820 17388 tablet_bootstrap.cc:492] T 95b5c4cf21e84292bd035af291af6641 P 43f20a4dc81f48e3bdb7a2281b8505c2: Bootstrap starting.
I20260923 22:15:10.754945 17387 tablet_bootstrap.cc:654] T 95b5c4cf21e84292bd035af291af6641 P 1d61ec2a894f4c76afcd904fdb097bfc: Neither blocks nor log segments found. Creating new log.
I20260923 22:15:10.755502 17388 tablet_bootstrap.cc:654] T 95b5c4cf21e84292bd035af291af6641 P 43f20a4dc81f48e3bdb7a2281b8505c2: Neither blocks nor log segments found. Creating new log.
I20260923 22:15:10.755534 17387 log.cc:824] T 95b5c4cf21e84292bd035af291af6641 P 1d61ec2a894f4c76afcd904fdb097bfc: Log is configured to *not* fsync() on all Append() calls
I20260923 22:15:10.756101 17388 log.cc:824] T 95b5c4cf21e84292bd035af291af6641 P 43f20a4dc81f48e3bdb7a2281b8505c2: Log is configured to *not* fsync() on all Append() calls
I20260923 22:15:10.756335 17387 tablet_bootstrap.cc:492] T 95b5c4cf21e84292bd035af291af6641 P 1d61ec2a894f4c76afcd904fdb097bfc: No bootstrap required, opened a new log
I20260923 22:15:10.756402 17387 ts_tablet_manager.cc:1397] T 95b5c4cf21e84292bd035af291af6641 P 1d61ec2a894f4c76afcd904fdb097bfc: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20260923 22:15:10.756795 17388 tablet_bootstrap.cc:492] T 95b5c4cf21e84292bd035af291af6641 P 43f20a4dc81f48e3bdb7a2281b8505c2: No bootstrap required, opened a new log
I20260923 22:15:10.756872 17388 ts_tablet_manager.cc:1397] T 95b5c4cf21e84292bd035af291af6641 P 43f20a4dc81f48e3bdb7a2281b8505c2: Time spent bootstrapping tablet: real 0.002s user 0.002s sys 0.000s
I20260923 22:15:10.758315 17387 raft_consensus.cc:348] T 95b5c4cf21e84292bd035af291af6641 P 1d61ec2a894f4c76afcd904fdb097bfc [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "de2e7140d2c742e6983e06beafcffd7a" member_type: VOTER last_known_addr { host: "127.1.135.193" port: 46309 } } peers { permanent_uuid: "1d61ec2a894f4c76afcd904fdb097bfc" member_type: VOTER last_known_addr { host: "127.1.135.194" port: 33393 } } peers { permanent_uuid: "43f20a4dc81f48e3bdb7a2281b8505c2" member_type: VOTER last_known_addr { host: "127.1.135.195" port: 44883 } }
I20260923 22:15:10.758491 17387 raft_consensus.cc:374] T 95b5c4cf21e84292bd035af291af6641 P 1d61ec2a894f4c76afcd904fdb097bfc [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260923 22:15:10.758533 17387 raft_consensus.cc:729] T 95b5c4cf21e84292bd035af291af6641 P 1d61ec2a894f4c76afcd904fdb097bfc [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 1d61ec2a894f4c76afcd904fdb097bfc, State: Initialized, Role: FOLLOWER
I20260923 22:15:10.758647 17387 consensus_queue.cc:260] T 95b5c4cf21e84292bd035af291af6641 P 1d61ec2a894f4c76afcd904fdb097bfc [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "de2e7140d2c742e6983e06beafcffd7a" member_type: VOTER last_known_addr { host: "127.1.135.193" port: 46309 } } peers { permanent_uuid: "1d61ec2a894f4c76afcd904fdb097bfc" member_type: VOTER last_known_addr { host: "127.1.135.194" port: 33393 } } peers { permanent_uuid: "43f20a4dc81f48e3bdb7a2281b8505c2" member_type: VOTER last_known_addr { host: "127.1.135.195" port: 44883 } }
I20260923 22:15:10.758917 17387 ts_tablet_manager.cc:1428] T 95b5c4cf21e84292bd035af291af6641 P 1d61ec2a894f4c76afcd904fdb097bfc: Time spent starting tablet: real 0.002s user 0.000s sys 0.003s
I20260923 22:15:10.758944 17239 heartbeater.cc:503] Master 127.1.135.254:45507 was elected leader, sending a full tablet report...
I20260923 22:15:10.758874 17388 raft_consensus.cc:348] T 95b5c4cf21e84292bd035af291af6641 P 43f20a4dc81f48e3bdb7a2281b8505c2 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "de2e7140d2c742e6983e06beafcffd7a" member_type: VOTER last_known_addr { host: "127.1.135.193" port: 46309 } } peers { permanent_uuid: "1d61ec2a894f4c76afcd904fdb097bfc" member_type: VOTER last_known_addr { host: "127.1.135.194" port: 33393 } } peers { permanent_uuid: "43f20a4dc81f48e3bdb7a2281b8505c2" member_type: VOTER last_known_addr { host: "127.1.135.195" port: 44883 } }
I20260923 22:15:10.759042 17388 raft_consensus.cc:374] T 95b5c4cf21e84292bd035af291af6641 P 43f20a4dc81f48e3bdb7a2281b8505c2 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260923 22:15:10.759089 17388 raft_consensus.cc:729] T 95b5c4cf21e84292bd035af291af6641 P 43f20a4dc81f48e3bdb7a2281b8505c2 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 43f20a4dc81f48e3bdb7a2281b8505c2, State: Initialized, Role: FOLLOWER
I20260923 22:15:10.759095 17390 tablet_bootstrap.cc:492] T 95b5c4cf21e84292bd035af291af6641 P de2e7140d2c742e6983e06beafcffd7a: Bootstrap starting.
I20260923 22:15:10.759219 17388 consensus_queue.cc:260] T 95b5c4cf21e84292bd035af291af6641 P 43f20a4dc81f48e3bdb7a2281b8505c2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "de2e7140d2c742e6983e06beafcffd7a" member_type: VOTER last_known_addr { host: "127.1.135.193" port: 46309 } } peers { permanent_uuid: "1d61ec2a894f4c76afcd904fdb097bfc" member_type: VOTER last_known_addr { host: "127.1.135.194" port: 33393 } } peers { permanent_uuid: "43f20a4dc81f48e3bdb7a2281b8505c2" member_type: VOTER last_known_addr { host: "127.1.135.195" port: 44883 } }
I20260923 22:15:10.759521 17388 ts_tablet_manager.cc:1428] T 95b5c4cf21e84292bd035af291af6641 P 43f20a4dc81f48e3bdb7a2281b8505c2: Time spent starting tablet: real 0.003s user 0.003s sys 0.000s
I20260923 22:15:10.759647 17373 heartbeater.cc:503] Master 127.1.135.254:45507 was elected leader, sending a full tablet report...
I20260923 22:15:10.759729 17390 tablet_bootstrap.cc:654] T 95b5c4cf21e84292bd035af291af6641 P de2e7140d2c742e6983e06beafcffd7a: Neither blocks nor log segments found. Creating new log.
I20260923 22:15:10.760332 17390 log.cc:824] T 95b5c4cf21e84292bd035af291af6641 P de2e7140d2c742e6983e06beafcffd7a: Log is configured to *not* fsync() on all Append() calls
I20260923 22:15:10.761020 17390 tablet_bootstrap.cc:492] T 95b5c4cf21e84292bd035af291af6641 P de2e7140d2c742e6983e06beafcffd7a: No bootstrap required, opened a new log
I20260923 22:15:10.761101 17390 ts_tablet_manager.cc:1397] T 95b5c4cf21e84292bd035af291af6641 P de2e7140d2c742e6983e06beafcffd7a: Time spent bootstrapping tablet: real 0.002s user 0.000s sys 0.002s
I20260923 22:15:10.762344 17390 raft_consensus.cc:348] T 95b5c4cf21e84292bd035af291af6641 P de2e7140d2c742e6983e06beafcffd7a [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "de2e7140d2c742e6983e06beafcffd7a" member_type: VOTER last_known_addr { host: "127.1.135.193" port: 46309 } } peers { permanent_uuid: "1d61ec2a894f4c76afcd904fdb097bfc" member_type: VOTER last_known_addr { host: "127.1.135.194" port: 33393 } } peers { permanent_uuid: "43f20a4dc81f48e3bdb7a2281b8505c2" member_type: VOTER last_known_addr { host: "127.1.135.195" port: 44883 } }
I20260923 22:15:10.762555 17390 raft_consensus.cc:374] T 95b5c4cf21e84292bd035af291af6641 P de2e7140d2c742e6983e06beafcffd7a [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260923 22:15:10.762607 17390 raft_consensus.cc:729] T 95b5c4cf21e84292bd035af291af6641 P de2e7140d2c742e6983e06beafcffd7a [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: de2e7140d2c742e6983e06beafcffd7a, State: Initialized, Role: FOLLOWER
I20260923 22:15:10.762720 17390 consensus_queue.cc:260] T 95b5c4cf21e84292bd035af291af6641 P de2e7140d2c742e6983e06beafcffd7a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "de2e7140d2c742e6983e06beafcffd7a" member_type: VOTER last_known_addr { host: "127.1.135.193" port: 46309 } } peers { permanent_uuid: "1d61ec2a894f4c76afcd904fdb097bfc" member_type: VOTER last_known_addr { host: "127.1.135.194" port: 33393 } } peers { permanent_uuid: "43f20a4dc81f48e3bdb7a2281b8505c2" member_type: VOTER last_known_addr { host: "127.1.135.195" port: 44883 } }
I20260923 22:15:10.763002 17390 ts_tablet_manager.cc:1428] T 95b5c4cf21e84292bd035af291af6641 P de2e7140d2c742e6983e06beafcffd7a: Time spent starting tablet: real 0.002s user 0.000s sys 0.002s
I20260923 22:15:10.763072 17105 heartbeater.cc:503] Master 127.1.135.254:45507 was elected leader, sending a full tablet report...
W20260923 22:15:10.817862 17240 tablet.cc:2369] T 95b5c4cf21e84292bd035af291af6641 P 1d61ec2a894f4c76afcd904fdb097bfc: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260923 22:15:10.909865 17106 tablet.cc:2369] T 95b5c4cf21e84292bd035af291af6641 P de2e7140d2c742e6983e06beafcffd7a: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260923 22:15:10.915709 17395 raft_consensus.cc:482] T 95b5c4cf21e84292bd035af291af6641 P de2e7140d2c742e6983e06beafcffd7a [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260923 22:15:10.915812 17395 raft_consensus.cc:504] T 95b5c4cf21e84292bd035af291af6641 P de2e7140d2c742e6983e06beafcffd7a [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "de2e7140d2c742e6983e06beafcffd7a" member_type: VOTER last_known_addr { host: "127.1.135.193" port: 46309 } } peers { permanent_uuid: "1d61ec2a894f4c76afcd904fdb097bfc" member_type: VOTER last_known_addr { host: "127.1.135.194" port: 33393 } } peers { permanent_uuid: "43f20a4dc81f48e3bdb7a2281b8505c2" member_type: VOTER last_known_addr { host: "127.1.135.195" port: 44883 } }
I20260923 22:15:10.916148 17395 leader_election.cc:288] T 95b5c4cf21e84292bd035af291af6641 P de2e7140d2c742e6983e06beafcffd7a [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 1d61ec2a894f4c76afcd904fdb097bfc (127.1.135.194:33393), 43f20a4dc81f48e3bdb7a2281b8505c2 (127.1.135.195:44883)
I20260923 22:15:10.919178 17384 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 22:15:10.916471 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.135.193:60733 (local address 127.1.135.195:44883)
0923 22:15:10.916514 (+ 43us) server_negotiation.cc:206] Beginning negotiation
0923 22:15:10.916516 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0923 22:15:10.916524 (+ 8us) server_negotiation.cc:407] Connection header received
0923 22:15:10.916555 (+ 31us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 22:15:10.916556 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 22:15:10.916566 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 22:15:10.916640 (+ 74us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0923 22:15:10.916856 (+ 216us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 22:15:10.917356 (+ 500us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 22:15:10.918299 (+ 943us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 22:15:10.918714 (+ 415us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 22:15:10.918775 (+ 61us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 22:15:10.918781 (+ 6us) server_negotiation.cc:1035] Waiting for connection context
0923 22:15:10.919064 (+ 283us) server_negotiation.cc:299] Negotiation successful
0923 22:15:10.919107 (+ 43us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":25}
I20260923 22:15:10.919185 17107 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 22:15:10.916343 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.135.195:44883 (local address 127.1.135.193:60733)
0923 22:15:10.916411 (+ 68us) negotiation.cc:107] Waiting for socket to connect
0923 22:15:10.916423 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0923 22:15:10.916467 (+ 44us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0923 22:15:10.916694 (+ 227us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0923 22:15:10.916695 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0923 22:15:10.916697 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0923 22:15:10.916778 (+ 81us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 22:15:10.916779 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 22:15:10.917438 (+ 659us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 22:15:10.917439 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 22:15:10.918188 (+ 749us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 22:15:10.918190 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 22:15:10.918899 (+ 709us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 22:15:10.918900 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 22:15:10.918981 (+ 81us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 22:15:10.918985 (+ 4us) client_negotiation.cc:769] Sending connection context
0923 22:15:10.919039 (+ 54us) client_negotiation.cc:240] Negotiation successful
0923 22:15:10.919098 (+ 59us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":50,"mutex_wait_us":2}
I20260923 22:15:10.919216 17396 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 22:15:10.916343 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.135.194:33393 (local address 127.1.135.193:38009)
0923 22:15:10.916560 (+ 217us) negotiation.cc:107] Waiting for socket to connect
0923 22:15:10.916571 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0923 22:15:10.916629 (+ 58us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0923 22:15:10.916791 (+ 162us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0923 22:15:10.916792 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0923 22:15:10.916793 (+ 1us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0923 22:15:10.916881 (+ 88us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 22:15:10.916883 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 22:15:10.917483 (+ 600us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 22:15:10.917484 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 22:15:10.918211 (+ 727us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 22:15:10.918212 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 22:15:10.918956 (+ 744us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 22:15:10.918957 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 22:15:10.919011 (+ 54us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 22:15:10.919014 (+ 3us) client_negotiation.cc:769] Sending connection context
0923 22:15:10.919089 (+ 75us) client_negotiation.cc:240] Negotiation successful
0923 22:15:10.919123 (+ 34us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":205,"mutex_wait_us":99,"thread_start_us":76,"threads_started":1}
I20260923 22:15:10.919215 17386 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 22:15:10.916412 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.135.193:38009 (local address 127.1.135.194:33393)
0923 22:15:10.916484 (+ 72us) server_negotiation.cc:206] Beginning negotiation
0923 22:15:10.916486 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0923 22:15:10.916639 (+ 153us) server_negotiation.cc:407] Connection header received
0923 22:15:10.916710 (+ 71us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 22:15:10.916711 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 22:15:10.916721 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 22:15:10.916780 (+ 59us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0923 22:15:10.916955 (+ 175us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 22:15:10.917383 (+ 428us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 22:15:10.918364 (+ 981us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 22:15:10.918872 (+ 508us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 22:15:10.918944 (+ 72us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 22:15:10.918953 (+ 9us) server_negotiation.cc:1035] Waiting for connection context
0923 22:15:10.919110 (+ 157us) server_negotiation.cc:299] Negotiation successful
0923 22:15:10.919150 (+ 40us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":45}
I20260923 22:15:10.919524 17327 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "95b5c4cf21e84292bd035af291af6641" candidate_uuid: "de2e7140d2c742e6983e06beafcffd7a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "43f20a4dc81f48e3bdb7a2281b8505c2" is_pre_election: true
I20260923 22:15:10.919588 17193 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "95b5c4cf21e84292bd035af291af6641" candidate_uuid: "de2e7140d2c742e6983e06beafcffd7a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1d61ec2a894f4c76afcd904fdb097bfc" is_pre_election: true
I20260923 22:15:10.919695 17327 raft_consensus.cc:2454] T 95b5c4cf21e84292bd035af291af6641 P 43f20a4dc81f48e3bdb7a2281b8505c2 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate de2e7140d2c742e6983e06beafcffd7a in term 0.
I20260923 22:15:10.919766 17193 raft_consensus.cc:2454] T 95b5c4cf21e84292bd035af291af6641 P 1d61ec2a894f4c76afcd904fdb097bfc [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate de2e7140d2c742e6983e06beafcffd7a in term 0.
I20260923 22:15:10.919986 16995 leader_election.cc:302] T 95b5c4cf21e84292bd035af291af6641 P de2e7140d2c742e6983e06beafcffd7a [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 43f20a4dc81f48e3bdb7a2281b8505c2, de2e7140d2c742e6983e06beafcffd7a; no voters:
I20260923 22:15:10.920148 17395 raft_consensus.cc:2781] T 95b5c4cf21e84292bd035af291af6641 P de2e7140d2c742e6983e06beafcffd7a [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260923 22:15:10.920207 17395 raft_consensus.cc:482] T 95b5c4cf21e84292bd035af291af6641 P de2e7140d2c742e6983e06beafcffd7a [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260923 22:15:10.920223 17395 raft_consensus.cc:3037] T 95b5c4cf21e84292bd035af291af6641 P de2e7140d2c742e6983e06beafcffd7a [term 0 FOLLOWER]: Advancing to term 1
I20260923 22:15:10.920871 17395 raft_consensus.cc:504] T 95b5c4cf21e84292bd035af291af6641 P de2e7140d2c742e6983e06beafcffd7a [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "de2e7140d2c742e6983e06beafcffd7a" member_type: VOTER last_known_addr { host: "127.1.135.193" port: 46309 } } peers { permanent_uuid: "1d61ec2a894f4c76afcd904fdb097bfc" member_type: VOTER last_known_addr { host: "127.1.135.194" port: 33393 } } peers { permanent_uuid: "43f20a4dc81f48e3bdb7a2281b8505c2" member_type: VOTER last_known_addr { host: "127.1.135.195" port: 44883 } }
I20260923 22:15:10.921013 17395 leader_election.cc:288] T 95b5c4cf21e84292bd035af291af6641 P de2e7140d2c742e6983e06beafcffd7a [CANDIDATE]: Term 1 election: Requested vote from peers 1d61ec2a894f4c76afcd904fdb097bfc (127.1.135.194:33393), 43f20a4dc81f48e3bdb7a2281b8505c2 (127.1.135.195:44883)
I20260923 22:15:10.921152 17193 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "95b5c4cf21e84292bd035af291af6641" candidate_uuid: "de2e7140d2c742e6983e06beafcffd7a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1d61ec2a894f4c76afcd904fdb097bfc"
I20260923 22:15:10.921176 17327 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "95b5c4cf21e84292bd035af291af6641" candidate_uuid: "de2e7140d2c742e6983e06beafcffd7a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "43f20a4dc81f48e3bdb7a2281b8505c2"
I20260923 22:15:10.921235 17193 raft_consensus.cc:3037] T 95b5c4cf21e84292bd035af291af6641 P 1d61ec2a894f4c76afcd904fdb097bfc [term 0 FOLLOWER]: Advancing to term 1
I20260923 22:15:10.921243 17327 raft_consensus.cc:3037] T 95b5c4cf21e84292bd035af291af6641 P 43f20a4dc81f48e3bdb7a2281b8505c2 [term 0 FOLLOWER]: Advancing to term 1
I20260923 22:15:10.922122 17327 raft_consensus.cc:2454] T 95b5c4cf21e84292bd035af291af6641 P 43f20a4dc81f48e3bdb7a2281b8505c2 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate de2e7140d2c742e6983e06beafcffd7a in term 1.
I20260923 22:15:10.922122 17193 raft_consensus.cc:2454] T 95b5c4cf21e84292bd035af291af6641 P 1d61ec2a894f4c76afcd904fdb097bfc [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate de2e7140d2c742e6983e06beafcffd7a in term 1.
I20260923 22:15:10.922298 16994 leader_election.cc:302] T 95b5c4cf21e84292bd035af291af6641 P de2e7140d2c742e6983e06beafcffd7a [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1d61ec2a894f4c76afcd904fdb097bfc, de2e7140d2c742e6983e06beafcffd7a; no voters:
I20260923 22:15:10.922447 17395 raft_consensus.cc:2781] T 95b5c4cf21e84292bd035af291af6641 P de2e7140d2c742e6983e06beafcffd7a [term 1 FOLLOWER]: Leader election won for term 1
I20260923 22:15:10.922610 17395 raft_consensus.cc:686] T 95b5c4cf21e84292bd035af291af6641 P de2e7140d2c742e6983e06beafcffd7a [term 1 LEADER]: Becoming Leader. State: Replica: de2e7140d2c742e6983e06beafcffd7a, State: Running, Role: LEADER
I20260923 22:15:10.922713 17395 consensus_queue.cc:237] T 95b5c4cf21e84292bd035af291af6641 P de2e7140d2c742e6983e06beafcffd7a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "de2e7140d2c742e6983e06beafcffd7a" member_type: VOTER last_known_addr { host: "127.1.135.193" port: 46309 } } peers { permanent_uuid: "1d61ec2a894f4c76afcd904fdb097bfc" member_type: VOTER last_known_addr { host: "127.1.135.194" port: 33393 } } peers { permanent_uuid: "43f20a4dc81f48e3bdb7a2281b8505c2" member_type: VOTER last_known_addr { host: "127.1.135.195" port: 44883 } }
I20260923 22:15:10.923581 16913 catalog_manager.cc:5495] T 95b5c4cf21e84292bd035af291af6641 P de2e7140d2c742e6983e06beafcffd7a reported cstate change: term changed from 0 to 1, leader changed from <none> to de2e7140d2c742e6983e06beafcffd7a (127.1.135.193). New cstate: current_term: 1 leader_uuid: "de2e7140d2c742e6983e06beafcffd7a" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "de2e7140d2c742e6983e06beafcffd7a" member_type: VOTER last_known_addr { host: "127.1.135.193" port: 46309 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "1d61ec2a894f4c76afcd904fdb097bfc" member_type: VOTER last_known_addr { host: "127.1.135.194" port: 33393 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "43f20a4dc81f48e3bdb7a2281b8505c2" member_type: VOTER last_known_addr { host: "127.1.135.195" port: 44883 } health_report { overall_health: UNKNOWN } } }
I20260923 22:15:10.930718 17383 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 22:15:10.928370 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47428 (local address 127.1.135.193:46309)
0923 22:15:10.928410 (+ 40us) server_negotiation.cc:206] Beginning negotiation
0923 22:15:10.928412 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0923 22:15:10.928434 (+ 22us) server_negotiation.cc:407] Connection header received
0923 22:15:10.928508 (+ 74us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 22:15:10.928508 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 22:15:10.928530 (+ 22us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 22:15:10.928597 (+ 67us) server_negotiation.cc:226] Negotiated authn=TOKEN
0923 22:15:10.928752 (+ 155us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 22:15:10.929199 (+ 447us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 22:15:10.929952 (+ 753us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 22:15:10.930129 (+ 177us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 22:15:10.930229 (+ 100us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 22:15:10.930407 (+ 178us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0923 22:15:10.930504 (+ 97us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0923 22:15:10.930597 (+ 93us) server_negotiation.cc:1035] Waiting for connection context
0923 22:15:10.930616 (+ 19us) server_negotiation.cc:299] Negotiation successful
0923 22:15:10.930634 (+ 18us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":20}
I20260923 22:15:10.932323 17327 raft_consensus.cc:1261] T 95b5c4cf21e84292bd035af291af6641 P 43f20a4dc81f48e3bdb7a2281b8505c2 [term 1 FOLLOWER]: Refusing update from remote peer de2e7140d2c742e6983e06beafcffd7a: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260923 22:15:10.932323 17193 raft_consensus.cc:1261] T 95b5c4cf21e84292bd035af291af6641 P 1d61ec2a894f4c76afcd904fdb097bfc [term 1 FOLLOWER]: Refusing update from remote peer de2e7140d2c742e6983e06beafcffd7a: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260923 22:15:10.932684 17397 consensus_queue.cc:1035] T 95b5c4cf21e84292bd035af291af6641 P de2e7140d2c742e6983e06beafcffd7a [LEADER]: Connected to new peer: Peer: permanent_uuid: "43f20a4dc81f48e3bdb7a2281b8505c2" member_type: VOTER last_known_addr { host: "127.1.135.195" port: 44883 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260923 22:15:10.932812 17395 consensus_queue.cc:1035] T 95b5c4cf21e84292bd035af291af6641 P de2e7140d2c742e6983e06beafcffd7a [LEADER]: Connected to new peer: Peer: permanent_uuid: "1d61ec2a894f4c76afcd904fdb097bfc" member_type: VOTER last_known_addr { host: "127.1.135.194" port: 33393 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260923 22:15:10.934985 17400 mvcc.cc:205] Tried to move back new op lower bound from 7332666207976165376 to 7332666207940100096. Current Snapshot: MvccSnapshot[applied={T|T < 7332666207976165376}]
I20260923 22:15:10.935221 17401 mvcc.cc:205] Tried to move back new op lower bound from 7332666207976165376 to 7332666207940100096. Current Snapshot: MvccSnapshot[applied={T|T < 7332666207976165376}]
I20260923 22:15:10.937448 16913 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:41644:
table { table_name: "test-table" } modify_external_catalogs: true
I20260923 22:15:10.937552 16913 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:41644:
table { table_name: "test-table" } modify_external_catalogs: true
I20260923 22:15:10.937542 17403 mvcc.cc:205] Tried to move back new op lower bound from 7332666207976165376 to 7332666207940100096. Current Snapshot: MvccSnapshot[applied={T|T < 7332666207976165376}]
I20260923 22:15:10.938254 16913 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 2d2671e82fe14a6fadf679b384125ea3: Sending DeleteTablet for 3 replicas of tablet 95b5c4cf21e84292bd035af291af6641
I20260923 22:15:10.938582 17039 tablet_service.cc:1514] Processing DeleteTablet for tablet 95b5c4cf21e84292bd035af291af6641 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-23 22:15:10 UTC) from {username='kudu', principal='kudu/127.1.135.254@KRBTEST.COM'} at 127.0.0.1:47414
I20260923 22:15:10.938624 17307 tablet_service.cc:1514] Processing DeleteTablet for tablet 95b5c4cf21e84292bd035af291af6641 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-23 22:15:10 UTC) from {username='kudu', principal='kudu/127.1.135.254@KRBTEST.COM'} at 127.0.0.1:38252
I20260923 22:15:10.938616 17173 tablet_service.cc:1514] Processing DeleteTablet for tablet 95b5c4cf21e84292bd035af291af6641 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-23 22:15:10 UTC) from {username='kudu', principal='kudu/127.1.135.254@KRBTEST.COM'} at 127.0.0.1:50384
I20260923 22:15:10.938862 17410 tablet_replica.cc:330] T 95b5c4cf21e84292bd035af291af6641 P de2e7140d2c742e6983e06beafcffd7a: stopping tablet replica
I20260923 22:15:10.938889 1567 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskRiQe_a/build/release/bin/kudu with pid 16976
I20260923 22:15:10.938894 17412 tablet_replica.cc:330] T 95b5c4cf21e84292bd035af291af6641 P 1d61ec2a894f4c76afcd904fdb097bfc: stopping tablet replica
I20260923 22:15:10.939524 17412 raft_consensus.cc:2229] T 95b5c4cf21e84292bd035af291af6641 P 1d61ec2a894f4c76afcd904fdb097bfc [term 1 FOLLOWER]: Raft consensus shutting down.
I20260923 22:15:10.939635 17412 raft_consensus.cc:2258] T 95b5c4cf21e84292bd035af291af6641 P 1d61ec2a894f4c76afcd904fdb097bfc [term 1 FOLLOWER]: Raft consensus is shut down!
I20260923 22:15:10.940045 17412 ts_tablet_manager.cc:1905] T 95b5c4cf21e84292bd035af291af6641 P 1d61ec2a894f4c76afcd904fdb097bfc: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260923 22:15:10.941218 17411 tablet_replica.cc:330] T 95b5c4cf21e84292bd035af291af6641 P 43f20a4dc81f48e3bdb7a2281b8505c2: stopping tablet replica
I20260923 22:15:10.941329 17411 raft_consensus.cc:2229] T 95b5c4cf21e84292bd035af291af6641 P 43f20a4dc81f48e3bdb7a2281b8505c2 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260923 22:15:10.941432 17411 raft_consensus.cc:2258] T 95b5c4cf21e84292bd035af291af6641 P 43f20a4dc81f48e3bdb7a2281b8505c2 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260923 22:15:10.941450 17412 ts_tablet_manager.cc:1918] T 95b5c4cf21e84292bd035af291af6641 P 1d61ec2a894f4c76afcd904fdb097bfc: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260923 22:15:10.941530 17412 log.cc:1196] T 95b5c4cf21e84292bd035af291af6641 P 1d61ec2a894f4c76afcd904fdb097bfc: Deleting WAL directory at /tmp/dist-test-taskRiQe_a/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790201539513534-1567-0/minicluster-data/ts-1/wal/wals/95b5c4cf21e84292bd035af291af6641
I20260923 22:15:10.941795 17412 ts_tablet_manager.cc:1939] T 95b5c4cf21e84292bd035af291af6641 P 1d61ec2a894f4c76afcd904fdb097bfc: Deleting consensus metadata
I20260923 22:15:10.941910 17411 ts_tablet_manager.cc:1905] T 95b5c4cf21e84292bd035af291af6641 P 43f20a4dc81f48e3bdb7a2281b8505c2: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260923 22:15:10.942209 16900 catalog_manager.cc:4841] TS 1d61ec2a894f4c76afcd904fdb097bfc (127.1.135.194:33393): tablet 95b5c4cf21e84292bd035af291af6641 (table test-table [id=2348099ad6364419b484d26fbd3ac660]) successfully deleted
I20260923 22:15:10.943740 17411 ts_tablet_manager.cc:1918] T 95b5c4cf21e84292bd035af291af6641 P 43f20a4dc81f48e3bdb7a2281b8505c2: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260923 22:15:10.943816 17411 log.cc:1196] T 95b5c4cf21e84292bd035af291af6641 P 43f20a4dc81f48e3bdb7a2281b8505c2: Deleting WAL directory at /tmp/dist-test-taskRiQe_a/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790201539513534-1567-0/minicluster-data/ts-2/wal/wals/95b5c4cf21e84292bd035af291af6641
I20260923 22:15:10.944104 17411 ts_tablet_manager.cc:1939] T 95b5c4cf21e84292bd035af291af6641 P 43f20a4dc81f48e3bdb7a2281b8505c2: Deleting consensus metadata
I20260923 22:15:10.944475 16901 catalog_manager.cc:4841] TS 43f20a4dc81f48e3bdb7a2281b8505c2 (127.1.135.195:44883): tablet 95b5c4cf21e84292bd035af291af6641 (table test-table [id=2348099ad6364419b484d26fbd3ac660]) successfully deleted
W20260923 22:15:10.945573 16900 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.1.135.193:46309 (error 108)
I20260923 22:15:10.945813 1567 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskRiQe_a/build/release/bin/kudu with pid 17110
I20260923 22:15:10.945994 17381 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 22:15:10.945872 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.135.193:46309 (local address 127.0.0.1:47442)
0923 22:15:10.945928 (+ 56us) negotiation.cc:107] Waiting for socket to connect
0923 22:15:10.945959 (+ 31us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.135.193:46309: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":34}
W20260923 22:15:10.946619 16900 catalog_manager.cc:4584] TS de2e7140d2c742e6983e06beafcffd7a (127.1.135.193:46309): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet 95b5c4cf21e84292bd035af291af6641: Network error: Client connection negotiation failed: client connection to 127.1.135.193:46309: connect: Connection refused (error 111)
I20260923 22:15:10.950474 1567 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskRiQe_a/build/release/bin/kudu with pid 17244
I20260923 22:15:10.955107 1567 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskRiQe_a/build/release/bin/kudu with pid 16882
I20260923 22:15:30.960379 1567 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskRiQe_a/build/release/bin/kudu
/tmp/dist-test-taskRiQe_a/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskRiQe_a/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790201539513534-1567-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskRiQe_a/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790201539513534-1567-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskRiQe_a/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790201539513534-1567-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskRiQe_a/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790201539513534-1567-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.1.135.254:45507
--webserver_interface=127.1.135.254
--webserver_port=34177
--keytab_file=/tmp/dist-test-taskRiQe_a/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790201539513534-1567-0/krb5kdc/kudu_127.1.135.254.keytab
--principal=kudu/127.1.135.254
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.1.135.212:34139
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.1.135.254:45507
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--authn_token_validity_seconds=20
--authz_token_validity_seconds=20
--tsk_rotation_seconds=20
--tsk_private_key_password_cmd=echo foo with env {KRB5CCNAME=/tmp/dist-test-taskRiQe_a/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790201539513534-1567-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskRiQe_a/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790201539513534-1567-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskRiQe_a/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790201539513534-1567-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260923 22:15:31.038733 17414 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260923 22:15:31.038951 17414 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260923 22:15:31.038987 17414 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260923 22:15:31.040194 17414 flags.cc:425] Enabled experimental flag: --authn_token_validity_seconds=20
W20260923 22:15:31.040244 17414 flags.cc:425] Enabled experimental flag: --authz_token_validity_seconds=20
W20260923 22:15:31.040267 17414 flags.cc:425] Enabled experimental flag: --tsk_rotation_seconds=20
W20260923 22:15:31.040285 17414 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260923 22:15:31.040316 17414 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260923 22:15:31.040347 17414 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260923 22:15:31.040375 17414 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260923 22:15:31.040397 17414 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260923 22:15:31.041780 17414 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.1.135.212:34139
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskRiQe_a/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790201539513534-1567-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskRiQe_a/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790201539513534-1567-0/minicluster-data/master-0/wal
--authn_token_validity_seconds=20
--authz_token_validity_seconds=20
--tsk_private_key_password_cmd=echo foo
--tsk_rotation_seconds=20
--ipki_ca_key_size=768
--master_addresses=127.1.135.254:45507
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskRiQe_a/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790201539513534-1567-0/krb5kdc/kudu_127.1.135.254.keytab
--principal=kudu/127.1.135.254
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.1.135.254:45507
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskRiQe_a/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790201539513534-1567-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.1.135.254
--webserver_port=34177
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.17414
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskRiQe_a/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790201539513534-1567-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 23 Sep 2026 20:53:37 UTC on ccycloud.quasar-rkviyx.root.comops.site
build id 4668083
I20260923 22:15:31.042086 17414 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260923 22:15:31.042294 17414 file_cache.cc:493] Constructed file cache file cache with capacity 419430
I20260923 22:15:31.050710 17414 server_base.cc:1034] running on GCE node
W20260923 22:15:31.050745 17420 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260923 22:15:31.050776 17421 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260923 22:15:31.050773 17423 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260923 22:15:31.051115 17414 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260923 22:15:31.051411 17414 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260923 22:15:31.052567 17414 hybrid_clock.cc:648] HybridClock initialized: now 1790201731052547 us; error 36 us; skew 500 ppm
Sep 23 22:15:31 dist-test-slave-s2fj krb5kdc[16869](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790201731, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.135.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260923 22:15:31.055004 17414 init.cc:377] Logged in from keytab as kudu/127.1.135.254@KRBTEST.COM (short username kudu)
I20260923 22:15:31.055493 17414 webserver.cc:460] Webserver started at http://127.1.135.254:34177/ using document root <none> and password file <none>
I20260923 22:15:31.055761 17414 fs_manager.cc:359] Metadata directory not provided
I20260923 22:15:31.055846 17414 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260923 22:15:31.057271 17414 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.001s sys 0.000s
I20260923 22:15:31.057945 17430 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260923 22:15:31.058116 17414 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260923 22:15:31.058223 17414 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskRiQe_a/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790201539513534-1567-0/minicluster-data/master-0/data,/tmp/dist-test-taskRiQe_a/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790201539513534-1567-0/minicluster-data/master-0/wal
uuid: "2d2671e82fe14a6fadf679b384125ea3"
format_stamp: "Formatted at 2026-09-23 22:15:10 on dist-test-slave-s2fj"
server_key: "b4b3416d8d0a070e64cead2cd4d90893"
server_key_iv: "6a4a527f9c3e1de279d16239eceaa956"
server_key_version: "encryptionkey@0"
I20260923 22:15:31.058595 17414 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskRiQe_a/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790201539513534-1567-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskRiQe_a/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790201539513534-1567-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskRiQe_a/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790201539513534-1567-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260923 22:15:31.080629 17414 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260923 22:15:31.082516 17414 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260923 22:15:31.082684 17414 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260923 22:15:31.086743 17414 rpc_server.cc:307] RPC server started. Bound to: 127.1.135.254:45507
I20260923 22:15:31.086777 17482 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.1.135.254:45507 every 8 connection(s)
I20260923 22:15:31.087138 17414 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskRiQe_a/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790201539513534-1567-0/minicluster-data/master-0/data/info.pb
I20260923 22:15:31.088593 17483 sys_catalog.cc:263] Verifying existing consensus state
I20260923 22:15:31.089143 17483 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 2d2671e82fe14a6fadf679b384125ea3: Bootstrap starting.
I20260923 22:15:31.095103 1567 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskRiQe_a/build/release/bin/kudu as pid 17414
I20260923 22:15:31.095389 1567 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskRiQe_a/build/release/bin/kudu
/tmp/dist-test-taskRiQe_a/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskRiQe_a/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790201539513534-1567-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskRiQe_a/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790201539513534-1567-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskRiQe_a/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790201539513534-1567-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskRiQe_a/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790201539513534-1567-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.1.135.193:46309
--local_ip_for_outbound_sockets=127.1.135.193
--tserver_master_addrs=127.1.135.254:45507
--webserver_port=35971
--webserver_interface=127.1.135.193
--keytab_file=/tmp/dist-test-taskRiQe_a/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790201539513534-1567-0/krb5kdc/kudu_127.1.135.193.keytab
--principal=kudu/127.1.135.193
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.1.135.212:34139
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskRiQe_a/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790201539513534-1567-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskRiQe_a/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790201539513534-1567-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskRiQe_a/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790201539513534-1567-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260923 22:15:31.097114 17483 log.cc:824] T 00000000000000000000000000000000 P 2d2671e82fe14a6fadf679b384125ea3: Log is configured to *not* fsync() on all Append() calls
I20260923 22:15:31.098942 17483 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 2d2671e82fe14a6fadf679b384125ea3: Bootstrap replayed 1/1 log segments. Stats: ops{read=8 overwritten=0 applied=8 ignored=0} inserts{seen=5 ignored=0} mutations{seen=4 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260923 22:15:31.099236 17483 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 2d2671e82fe14a6fadf679b384125ea3: Bootstrap complete.
I20260923 22:15:31.101466 17483 raft_consensus.cc:348] T 00000000000000000000000000000000 P 2d2671e82fe14a6fadf679b384125ea3 [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2d2671e82fe14a6fadf679b384125ea3" member_type: VOTER last_known_addr { host: "127.1.135.254" port: 45507 } }
I20260923 22:15:31.101698 17483 raft_consensus.cc:729] T 00000000000000000000000000000000 P 2d2671e82fe14a6fadf679b384125ea3 [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: 2d2671e82fe14a6fadf679b384125ea3, State: Initialized, Role: FOLLOWER
I20260923 22:15:31.101882 17483 consensus_queue.cc:260] T 00000000000000000000000000000000 P 2d2671e82fe14a6fadf679b384125ea3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 8, Last appended: 1.8, Last appended by leader: 8, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2d2671e82fe14a6fadf679b384125ea3" member_type: VOTER last_known_addr { host: "127.1.135.254" port: 45507 } }
I20260923 22:15:31.101982 17483 raft_consensus.cc:388] T 00000000000000000000000000000000 P 2d2671e82fe14a6fadf679b384125ea3 [term 1 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260923 22:15:31.102041 17483 raft_consensus.cc:482] T 00000000000000000000000000000000 P 2d2671e82fe14a6fadf679b384125ea3 [term 1 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260923 22:15:31.102092 17483 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 2d2671e82fe14a6fadf679b384125ea3 [term 1 FOLLOWER]: Advancing to term 2
I20260923 22:15:31.103040 17483 raft_consensus.cc:504] T 00000000000000000000000000000000 P 2d2671e82fe14a6fadf679b384125ea3 [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2d2671e82fe14a6fadf679b384125ea3" member_type: VOTER last_known_addr { host: "127.1.135.254" port: 45507 } }
I20260923 22:15:31.103155 17483 leader_election.cc:302] T 00000000000000000000000000000000 P 2d2671e82fe14a6fadf679b384125ea3 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 2d2671e82fe14a6fadf679b384125ea3; no voters:
I20260923 22:15:31.103343 17483 leader_election.cc:288] T 00000000000000000000000000000000 P 2d2671e82fe14a6fadf679b384125ea3 [CANDIDATE]: Term 2 election: Requested vote from peers
I20260923 22:15:31.103377 17487 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 2d2671e82fe14a6fadf679b384125ea3 [term 2 FOLLOWER]: Leader election won for term 2
I20260923 22:15:31.103488 17487 raft_consensus.cc:686] T 00000000000000000000000000000000 P 2d2671e82fe14a6fadf679b384125ea3 [term 2 LEADER]: Becoming Leader. State: Replica: 2d2671e82fe14a6fadf679b384125ea3, State: Running, Role: LEADER
I20260923 22:15:31.103591 17483 sys_catalog.cc:564] T 00000000000000000000000000000000 P 2d2671e82fe14a6fadf679b384125ea3 [sys.catalog]: configured and running, proceeding with master startup.
I20260923 22:15:31.103570 17487 consensus_queue.cc:237] T 00000000000000000000000000000000 P 2d2671e82fe14a6fadf679b384125ea3 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 1.8, Last appended by leader: 8, Current term: 2, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2d2671e82fe14a6fadf679b384125ea3" member_type: VOTER last_known_addr { host: "127.1.135.254" port: 45507 } }
I20260923 22:15:31.103838 17488 sys_catalog.cc:455] T 00000000000000000000000000000000 P 2d2671e82fe14a6fadf679b384125ea3 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 2 leader_uuid: "2d2671e82fe14a6fadf679b384125ea3" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2d2671e82fe14a6fadf679b384125ea3" member_type: VOTER last_known_addr { host: "127.1.135.254" port: 45507 } } }
I20260923 22:15:31.103932 17488 sys_catalog.cc:458] T 00000000000000000000000000000000 P 2d2671e82fe14a6fadf679b384125ea3 [sys.catalog]: This master's current role is: LEADER
I20260923 22:15:31.104173 17489 sys_catalog.cc:455] T 00000000000000000000000000000000 P 2d2671e82fe14a6fadf679b384125ea3 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 2d2671e82fe14a6fadf679b384125ea3. Latest consensus state: current_term: 2 leader_uuid: "2d2671e82fe14a6fadf679b384125ea3" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2d2671e82fe14a6fadf679b384125ea3" member_type: VOTER last_known_addr { host: "127.1.135.254" port: 45507 } } }
I20260923 22:15:31.104245 17489 sys_catalog.cc:458] T 00000000000000000000000000000000 P 2d2671e82fe14a6fadf679b384125ea3 [sys.catalog]: This master's current role is: LEADER
I20260923 22:15:31.104354 17492 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260923 22:15:31.105132 17492 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260923 22:15:31.105222 17492 catalog_manager.cc:1259] Loaded cluster ID: 88dd55e093394f6881df1895285fec40
I20260923 22:15:31.105245 17492 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260923 22:15:31.106477 17492 catalog_manager.cc:1504] Loading token signing keys...
I20260923 22:15:31.106640 17492 catalog_manager.cc:5879] T 00000000000000000000000000000000 P 2d2671e82fe14a6fadf679b384125ea3: Loaded TSK: 0
I20260923 22:15:31.116127 17492 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 2d2671e82fe14a6fadf679b384125ea3: Generated new TSK 1
I20260923 22:15:31.116202 17492 catalog_manager.cc:1514] Initializing in-progress tserver states...
W20260923 22:15:31.178951 17485 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260923 22:15:31.179220 17485 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260923 22:15:31.179266 17485 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260923 22:15:31.181106 17485 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260923 22:15:31.181162 17485 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260923 22:15:31.181202 17485 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.1.135.193
I20260923 22:15:31.183820 17485 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.1.135.212:34139
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskRiQe_a/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790201539513534-1567-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskRiQe_a/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790201539513534-1567-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskRiQe_a/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790201539513534-1567-0/krb5kdc/kudu_127.1.135.193.keytab
--principal=kudu/127.1.135.193
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.1.135.193:46309
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskRiQe_a/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790201539513534-1567-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.1.135.193
--webserver_port=35971
--tserver_master_addrs=127.1.135.254:45507
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.17485
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.1.135.193
--log_dir=/tmp/dist-test-taskRiQe_a/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790201539513534-1567-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 23 Sep 2026 20:53:37 UTC on ccycloud.quasar-rkviyx.root.comops.site
build id 4668083
I20260923 22:15:31.184046 17485 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260923 22:15:31.184321 17485 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260923 22:15:31.190532 17513 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260923 22:15:31.190565 17511 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260923 22:15:31.190801 17510 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260923 22:15:31.190793 17485 server_base.cc:1034] running on GCE node
I20260923 22:15:31.191088 17485 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260923 22:15:31.191344 17485 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260923 22:15:31.192525 17485 hybrid_clock.cc:648] HybridClock initialized: now 1790201731192508 us; error 44 us; skew 500 ppm
Sep 23 22:15:31 dist-test-slave-s2fj krb5kdc[16869](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790201731, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.135.193@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260923 22:15:31.195245 17485 init.cc:377] Logged in from keytab as kudu/127.1.135.193@KRBTEST.COM (short username kudu)
I20260923 22:15:31.195679 17485 webserver.cc:460] Webserver started at http://127.1.135.193:35971/ using document root <none> and password file <none>
I20260923 22:15:31.195930 17485 fs_manager.cc:359] Metadata directory not provided
I20260923 22:15:31.196010 17485 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260923 22:15:31.197392 17485 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.001s sys 0.000s
I20260923 22:15:31.198084 17520 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260923 22:15:31.198251 17485 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260923 22:15:31.198356 17485 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskRiQe_a/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790201539513534-1567-0/minicluster-data/ts-0/data,/tmp/dist-test-taskRiQe_a/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790201539513534-1567-0/minicluster-data/ts-0/wal
uuid: "de2e7140d2c742e6983e06beafcffd7a"
format_stamp: "Formatted at 2026-09-23 22:15:10 on dist-test-slave-s2fj"
server_key: "c01fe50084c85daf4a825bec04f9a216"
server_key_iv: "b82882840d6cb85ee11076a203f25d2f"
server_key_version: "encryptionkey@0"
I20260923 22:15:31.198678 17485 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskRiQe_a/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790201539513534-1567-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskRiQe_a/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790201539513534-1567-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskRiQe_a/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790201539513534-1567-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260923 22:15:31.212028 17485 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260923 22:15:31.214227 17485 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260923 22:15:31.214396 17485 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260923 22:15:31.214633 17485 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260923 22:15:31.215104 17527 ts_tablet_manager.cc:536] Loading tablet metadata (0/1 complete)
I20260923 22:15:31.216535 17485 ts_tablet_manager.cc:579] Loaded tablet metadata (1 total tablets, 1 live tablets)
I20260923 22:15:31.216583 17485 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.002s user 0.000s sys 0.000s
I20260923 22:15:31.216615 17485 ts_tablet_manager.cc:594] Registering tablets (0/1 complete)
I20260923 22:15:31.217223 17485 ts_tablet_manager.cc:610] Registered 1 tablets
I20260923 22:15:31.217267 17485 ts_tablet_manager.cc:589] Time spent register tablets: real 0.001s user 0.000s sys 0.000s
I20260923 22:15:31.217308 17527 tablet_bootstrap.cc:492] T 95b5c4cf21e84292bd035af291af6641 P de2e7140d2c742e6983e06beafcffd7a: Bootstrap starting.
I20260923 22:15:31.224721 17485 rpc_server.cc:307] RPC server started. Bound to: 127.1.135.193:46309
I20260923 22:15:31.225188 17485 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskRiQe_a/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790201539513534-1567-0/minicluster-data/ts-0/data/info.pb
I20260923 22:15:31.228018 17527 log.cc:824] T 95b5c4cf21e84292bd035af291af6641 P de2e7140d2c742e6983e06beafcffd7a: Log is configured to *not* fsync() on all Append() calls
I20260923 22:15:31.228631 17634 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.1.135.193:46309 every 8 connection(s)
I20260923 22:15:31.230244 1567 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskRiQe_a/build/release/bin/kudu as pid 17485
I20260923 22:15:31.230558 1567 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskRiQe_a/build/release/bin/kudu
/tmp/dist-test-taskRiQe_a/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskRiQe_a/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790201539513534-1567-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskRiQe_a/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790201539513534-1567-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskRiQe_a/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790201539513534-1567-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskRiQe_a/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790201539513534-1567-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.1.135.194:33393
--local_ip_for_outbound_sockets=127.1.135.194
--tserver_master_addrs=127.1.135.254:45507
--webserver_port=34741
--webserver_interface=127.1.135.194
--keytab_file=/tmp/dist-test-taskRiQe_a/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790201539513534-1567-0/krb5kdc/kudu_127.1.135.194.keytab
--principal=kudu/127.1.135.194
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.1.135.212:34139
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskRiQe_a/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790201539513534-1567-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskRiQe_a/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790201539513534-1567-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskRiQe_a/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790201539513534-1567-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260923 22:15:31.230571 17527 tablet_bootstrap.cc:492] T 95b5c4cf21e84292bd035af291af6641 P de2e7140d2c742e6983e06beafcffd7a: Bootstrap replayed 1/1 log segments. Stats: ops{read=2 overwritten=0 applied=2 ignored=0} inserts{seen=1 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260923 22:15:31.230880 17527 tablet_bootstrap.cc:492] T 95b5c4cf21e84292bd035af291af6641 P de2e7140d2c742e6983e06beafcffd7a: Bootstrap complete.
I20260923 22:15:31.231950 17527 ts_tablet_manager.cc:1397] T 95b5c4cf21e84292bd035af291af6641 P de2e7140d2c742e6983e06beafcffd7a: Time spent bootstrapping tablet: real 0.015s user 0.002s sys 0.011s
I20260923 22:15:31.233238 17527 raft_consensus.cc:348] T 95b5c4cf21e84292bd035af291af6641 P de2e7140d2c742e6983e06beafcffd7a [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "de2e7140d2c742e6983e06beafcffd7a" member_type: VOTER last_known_addr { host: "127.1.135.193" port: 46309 } } peers { permanent_uuid: "1d61ec2a894f4c76afcd904fdb097bfc" member_type: VOTER last_known_addr { host: "127.1.135.194" port: 33393 } } peers { permanent_uuid: "43f20a4dc81f48e3bdb7a2281b8505c2" member_type: VOTER last_known_addr { host: "127.1.135.195" port: 44883 } }
I20260923 22:15:31.233566 17527 raft_consensus.cc:729] T 95b5c4cf21e84292bd035af291af6641 P de2e7140d2c742e6983e06beafcffd7a [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: de2e7140d2c742e6983e06beafcffd7a, State: Initialized, Role: FOLLOWER
I20260923 22:15:31.233708 17527 consensus_queue.cc:260] T 95b5c4cf21e84292bd035af291af6641 P de2e7140d2c742e6983e06beafcffd7a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 2, Last appended: 1.2, Last appended by leader: 2, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "de2e7140d2c742e6983e06beafcffd7a" member_type: VOTER last_known_addr { host: "127.1.135.193" port: 46309 } } peers { permanent_uuid: "1d61ec2a894f4c76afcd904fdb097bfc" member_type: VOTER last_known_addr { host: "127.1.135.194" port: 33393 } } peers { permanent_uuid: "43f20a4dc81f48e3bdb7a2281b8505c2" member_type: VOTER last_known_addr { host: "127.1.135.195" port: 44883 } }
I20260923 22:15:31.234035 17527 ts_tablet_manager.cc:1428] T 95b5c4cf21e84292bd035af291af6641 P de2e7140d2c742e6983e06beafcffd7a: Time spent starting tablet: real 0.002s user 0.004s sys 0.000s
Sep 23 22:15:31 dist-test-slave-s2fj krb5kdc[16869](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790201731, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.135.193@KRBTEST.COM for kudu/127.1.135.254@KRBTEST.COM
I20260923 22:15:31.241533 17638 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 22:15:31.225976 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.135.254:45507 (local address 127.1.135.193:47831)
0923 22:15:31.228822 (+ 2846us) negotiation.cc:107] Waiting for socket to connect
0923 22:15:31.228874 (+ 52us) client_negotiation.cc:174] Beginning negotiation
0923 22:15:31.230396 (+ 1522us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0923 22:15:31.231297 (+ 901us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0923 22:15:31.231300 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0923 22:15:31.231679 (+ 379us) client_negotiation.cc:189] Negotiated authn=SASL
0923 22:15:31.231983 (+ 304us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 22:15:31.231988 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 22:15:31.233566 (+ 1578us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 22:15:31.233567 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 22:15:31.235042 (+ 1475us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 22:15:31.235046 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 22:15:31.235609 (+ 563us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 22:15:31.235610 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 22:15:31.235764 (+ 154us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 22:15:31.236377 (+ 613us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0923 22:15:31.236388 (+ 11us) client_negotiation.cc:656] calling sasl_client_start()
0923 22:15:31.238042 (+ 1654us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0923 22:15:31.239924 (+ 1882us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0923 22:15:31.239931 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0923 22:15:31.239933 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0923 22:15:31.240244 (+ 311us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0923 22:15:31.240487 (+ 243us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0923 22:15:31.240488 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0923 22:15:31.240488 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0923 22:15:31.240597 (+ 109us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0923 22:15:31.240954 (+ 357us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0923 22:15:31.240959 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0923 22:15:31.241086 (+ 127us) client_negotiation.cc:769] Sending connection context
0923 22:15:31.241180 (+ 94us) client_negotiation.cc:240] Negotiation successful
0923 22:15:31.241342 (+ 162us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":2804,"thread_start_us":107,"threads_started":1}
I20260923 22:15:31.241534 17637 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 22:15:31.226014 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.135.193:47831 (local address 127.1.135.254:45507)
0923 22:15:31.228226 (+ 2212us) server_negotiation.cc:206] Beginning negotiation
0923 22:15:31.228235 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0923 22:15:31.229024 (+ 789us) server_negotiation.cc:407] Connection header received
0923 22:15:31.230960 (+ 1936us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 22:15:31.230968 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 22:15:31.231030 (+ 62us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 22:15:31.231161 (+ 131us) server_negotiation.cc:226] Negotiated authn=SASL
0923 22:15:31.232202 (+ 1041us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 22:15:31.233461 (+ 1259us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 22:15:31.235132 (+ 1671us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 22:15:31.235512 (+ 380us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 22:15:31.235597 (+ 85us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 22:15:31.238149 (+ 2552us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0923 22:15:31.238153 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0923 22:15:31.238153 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0923 22:15:31.238172 (+ 19us) server_negotiation.cc:922] calling sasl_server_start()
0923 22:15:31.239801 (+ 1629us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0923 22:15:31.240328 (+ 527us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0923 22:15:31.240331 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0923 22:15:31.240332 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0923 22:15:31.240418 (+ 86us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0923 22:15:31.240668 (+ 250us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0923 22:15:31.240669 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0923 22:15:31.240669 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0923 22:15:31.240874 (+ 205us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0923 22:15:31.240962 (+ 88us) server_negotiation.cc:1035] Waiting for connection context
0923 22:15:31.241251 (+ 289us) server_negotiation.cc:299] Negotiation successful
0923 22:15:31.241365 (+ 114us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":2152,"thread_start_us":71,"threads_started":1}
I20260923 22:15:31.242331 17635 heartbeater.cc:347] Connected to a master server at 127.1.135.254:45507
I20260923 22:15:31.242422 17635 heartbeater.cc:464] Registering TS with master...
I20260923 22:15:31.242579 17635 heartbeater.cc:511] Master 127.1.135.254:45507 requested a full tablet report, sending...
I20260923 22:15:31.243285 17447 ts_manager.cc:194] Registered new tserver with Master: de2e7140d2c742e6983e06beafcffd7a (127.1.135.193:46309)
I20260923 22:15:31.244411 17447 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.1.135.193@KRBTEST.COM'} at 127.1.135.193:47831
I20260923 22:15:31.245289 17635 heartbeater.cc:503] Master 127.1.135.254:45507 was elected leader, sending a full tablet report...
Sep 23 22:15:31 dist-test-slave-s2fj krb5kdc[16869](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790201731, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.135.254@KRBTEST.COM for kudu/127.1.135.193@KRBTEST.COM
I20260923 22:15:31.251550 17642 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 22:15:31.243850 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.135.193:46309 (local address 127.0.0.1:34466)
0923 22:15:31.244030 (+ 180us) negotiation.cc:107] Waiting for socket to connect
0923 22:15:31.244102 (+ 72us) client_negotiation.cc:174] Beginning negotiation
0923 22:15:31.244158 (+ 56us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0923 22:15:31.244351 (+ 193us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0923 22:15:31.244356 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0923 22:15:31.244661 (+ 305us) client_negotiation.cc:189] Negotiated authn=SASL
0923 22:15:31.244711 (+ 50us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 22:15:31.244713 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 22:15:31.245965 (+ 1252us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 22:15:31.245966 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 22:15:31.246513 (+ 547us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 22:15:31.246515 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 22:15:31.247084 (+ 569us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 22:15:31.247085 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 22:15:31.247136 (+ 51us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 22:15:31.247600 (+ 464us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0923 22:15:31.247604 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0923 22:15:31.248835 (+ 1231us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0923 22:15:31.250617 (+ 1782us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0923 22:15:31.250622 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0923 22:15:31.250623 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0923 22:15:31.250799 (+ 176us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0923 22:15:31.250994 (+ 195us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0923 22:15:31.250995 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0923 22:15:31.250995 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0923 22:15:31.251044 (+ 49us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0923 22:15:31.251316 (+ 272us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0923 22:15:31.251321 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0923 22:15:31.251356 (+ 35us) client_negotiation.cc:769] Sending connection context
0923 22:15:31.251414 (+ 58us) client_negotiation.cc:240] Negotiation successful
0923 22:15:31.251491 (+ 77us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":156,"thread_start_us":71,"threads_started":1}
I20260923 22:15:31.251753 17643 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 22:15:31.243914 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34466 (local address 127.1.135.193:46309)
0923 22:15:31.244191 (+ 277us) server_negotiation.cc:206] Beginning negotiation
0923 22:15:31.244195 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0923 22:15:31.244229 (+ 34us) server_negotiation.cc:407] Connection header received
0923 22:15:31.244259 (+ 30us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 22:15:31.244261 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 22:15:31.244274 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 22:15:31.244333 (+ 59us) server_negotiation.cc:226] Negotiated authn=SASL
0923 22:15:31.244801 (+ 468us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 22:15:31.245883 (+ 1082us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 22:15:31.246587 (+ 704us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 22:15:31.247011 (+ 424us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 22:15:31.247079 (+ 68us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 22:15:31.248910 (+ 1831us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0923 22:15:31.248917 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0923 22:15:31.248917 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0923 22:15:31.248924 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0923 22:15:31.250533 (+ 1609us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0923 22:15:31.250872 (+ 339us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0923 22:15:31.250874 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0923 22:15:31.250875 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0923 22:15:31.250931 (+ 56us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0923 22:15:31.251101 (+ 170us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0923 22:15:31.251101 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0923 22:15:31.251101 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0923 22:15:31.251256 (+ 155us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0923 22:15:31.251332 (+ 76us) server_negotiation.cc:1035] Waiting for connection context
0923 22:15:31.251465 (+ 133us) server_negotiation.cc:299] Negotiation successful
0923 22:15:31.251554 (+ 89us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":3,"server-negotiator.queue_time_us":251,"thread_start_us":81,"threads_started":1}
I20260923 22:15:31.252122 17558 tablet_service.cc:1514] Processing DeleteTablet for tablet 95b5c4cf21e84292bd035af291af6641 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-23 22:15:10 UTC) from {username='kudu', principal='kudu/127.1.135.254@KRBTEST.COM'} at 127.0.0.1:34466
I20260923 22:15:31.252633 17644 tablet_replica.cc:330] stopping tablet replica
I20260923 22:15:31.252754 17644 raft_consensus.cc:2229] T 95b5c4cf21e84292bd035af291af6641 P de2e7140d2c742e6983e06beafcffd7a [term 1 FOLLOWER]: Raft consensus shutting down.
I20260923 22:15:31.252836 17644 raft_consensus.cc:2258] T 95b5c4cf21e84292bd035af291af6641 P de2e7140d2c742e6983e06beafcffd7a [term 1 FOLLOWER]: Raft consensus is shut down!
I20260923 22:15:31.253080 17644 ts_tablet_manager.cc:1905] T 95b5c4cf21e84292bd035af291af6641 P de2e7140d2c742e6983e06beafcffd7a: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260923 22:15:31.255342 17644 ts_tablet_manager.cc:1918] T 95b5c4cf21e84292bd035af291af6641 P de2e7140d2c742e6983e06beafcffd7a: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260923 22:15:31.255422 17644 log.cc:1196] T 95b5c4cf21e84292bd035af291af6641 P de2e7140d2c742e6983e06beafcffd7a: Deleting WAL directory at /tmp/dist-test-taskRiQe_a/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790201539513534-1567-0/minicluster-data/ts-0/wal/wals/95b5c4cf21e84292bd035af291af6641
I20260923 22:15:31.255631 17644 ts_tablet_manager.cc:1939] T 95b5c4cf21e84292bd035af291af6641 P de2e7140d2c742e6983e06beafcffd7a: Deleting consensus metadata
I20260923 22:15:31.256134 17433 catalog_manager.cc:4841] TS de2e7140d2c742e6983e06beafcffd7a (127.1.135.193:46309): tablet 95b5c4cf21e84292bd035af291af6641 (table test-table [id=2348099ad6364419b484d26fbd3ac660]) successfully deleted
W20260923 22:15:31.313426 17640 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260923 22:15:31.313603 17640 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260923 22:15:31.313627 17640 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260923 22:15:31.314694 17640 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260923 22:15:31.314728 17640 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260923 22:15:31.314754 17640 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.1.135.194
I20260923 22:15:31.316098 17640 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.1.135.212:34139
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskRiQe_a/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790201539513534-1567-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskRiQe_a/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790201539513534-1567-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskRiQe_a/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790201539513534-1567-0/krb5kdc/kudu_127.1.135.194.keytab
--principal=kudu/127.1.135.194
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.1.135.194:33393
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskRiQe_a/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790201539513534-1567-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.1.135.194
--webserver_port=34741
--tserver_master_addrs=127.1.135.254:45507
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.17640
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.1.135.194
--log_dir=/tmp/dist-test-taskRiQe_a/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790201539513534-1567-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 23 Sep 2026 20:53:37 UTC on ccycloud.quasar-rkviyx.root.comops.site
build id 4668083
I20260923 22:15:31.316257 17640 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260923 22:15:31.316406 17640 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260923 22:15:31.321080 17650 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260923 22:15:31.321125 17652 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260923 22:15:31.321352 17649 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260923 22:15:31.321457 17640 server_base.cc:1034] running on GCE node
I20260923 22:15:31.321656 17640 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260923 22:15:31.321893 17640 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260923 22:15:31.323064 17640 hybrid_clock.cc:648] HybridClock initialized: now 1790201731323041 us; error 40 us; skew 500 ppm
Sep 23 22:15:31 dist-test-slave-s2fj krb5kdc[16869](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790201731, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.135.194@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260923 22:15:31.325449 17640 init.cc:377] Logged in from keytab as kudu/127.1.135.194@KRBTEST.COM (short username kudu)
I20260923 22:15:31.325893 17640 webserver.cc:460] Webserver started at http://127.1.135.194:34741/ using document root <none> and password file <none>
I20260923 22:15:31.326118 17640 fs_manager.cc:359] Metadata directory not provided
I20260923 22:15:31.326201 17640 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260923 22:15:31.327462 17640 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.001s sys 0.000s
I20260923 22:15:31.327967 17659 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260923 22:15:31.328142 17640 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260923 22:15:31.328248 17640 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskRiQe_a/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790201539513534-1567-0/minicluster-data/ts-1/data,/tmp/dist-test-taskRiQe_a/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790201539513534-1567-0/minicluster-data/ts-1/wal
uuid: "1d61ec2a894f4c76afcd904fdb097bfc"
format_stamp: "Formatted at 2026-09-23 22:15:10 on dist-test-slave-s2fj"
server_key: "61fd161f646230d3a8ca49c7ce067153"
server_key_iv: "55c6a41dda98e464a8572f3994d180a6"
server_key_version: "encryptionkey@0"
I20260923 22:15:31.328565 17640 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskRiQe_a/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790201539513534-1567-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskRiQe_a/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790201539513534-1567-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskRiQe_a/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790201539513534-1567-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260923 22:15:31.338371 17640 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260923 22:15:31.339995 17640 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260923 22:15:31.340163 17640 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260923 22:15:31.340366 17640 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260923 22:15:31.340662 17640 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260923 22:15:31.340713 17640 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260923 22:15:31.340754 17640 ts_tablet_manager.cc:610] Registered 0 tablets
I20260923 22:15:31.340781 17640 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260923 22:15:31.346423 17640 rpc_server.cc:307] RPC server started. Bound to: 127.1.135.194:33393
I20260923 22:15:31.346470 17772 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.1.135.194:33393 every 8 connection(s)
I20260923 22:15:31.346757 17640 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskRiQe_a/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790201539513534-1567-0/minicluster-data/ts-1/data/info.pb
Sep 23 22:15:31 dist-test-slave-s2fj krb5kdc[16869](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790201731, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.135.194@KRBTEST.COM for kudu/127.1.135.254@KRBTEST.COM
I20260923 22:15:31.354823 17637 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 22:15:31.347400 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.135.194:38703 (local address 127.1.135.254:45507)
0923 22:15:31.347463 (+ 63us) server_negotiation.cc:206] Beginning negotiation
0923 22:15:31.347465 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0923 22:15:31.347631 (+ 166us) server_negotiation.cc:407] Connection header received
0923 22:15:31.348035 (+ 404us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 22:15:31.348036 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 22:15:31.348046 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 22:15:31.348153 (+ 107us) server_negotiation.cc:226] Negotiated authn=SASL
0923 22:15:31.348937 (+ 784us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 22:15:31.349409 (+ 472us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 22:15:31.350328 (+ 919us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 22:15:31.350518 (+ 190us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 22:15:31.350567 (+ 49us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 22:15:31.352467 (+ 1900us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0923 22:15:31.352469 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0923 22:15:31.352469 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0923 22:15:31.352476 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0923 22:15:31.353799 (+ 1323us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0923 22:15:31.354104 (+ 305us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0923 22:15:31.354105 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0923 22:15:31.354105 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0923 22:15:31.354153 (+ 48us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0923 22:15:31.354343 (+ 190us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0923 22:15:31.354343 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0923 22:15:31.354343 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0923 22:15:31.354431 (+ 88us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0923 22:15:31.354478 (+ 47us) server_negotiation.cc:1035] Waiting for connection context
0923 22:15:31.354699 (+ 221us) server_negotiation.cc:299] Negotiation successful
0923 22:15:31.354764 (+ 65us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":38}
I20260923 22:15:31.355005 17775 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 22:15:31.347379 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.135.254:45507 (local address 127.1.135.194:38703)
0923 22:15:31.347514 (+ 135us) negotiation.cc:107] Waiting for socket to connect
0923 22:15:31.347550 (+ 36us) client_negotiation.cc:174] Beginning negotiation
0923 22:15:31.347963 (+ 413us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0923 22:15:31.348204 (+ 241us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0923 22:15:31.348207 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0923 22:15:31.348702 (+ 495us) client_negotiation.cc:189] Negotiated authn=SASL
0923 22:15:31.348866 (+ 164us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 22:15:31.348870 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 22:15:31.349485 (+ 615us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 22:15:31.349485 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 22:15:31.350261 (+ 776us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 22:15:31.350263 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 22:15:31.350588 (+ 325us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 22:15:31.350588 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 22:15:31.350677 (+ 89us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 22:15:31.351078 (+ 401us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0923 22:15:31.351085 (+ 7us) client_negotiation.cc:656] calling sasl_client_start()
0923 22:15:31.352388 (+ 1303us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0923 22:15:31.353865 (+ 1477us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0923 22:15:31.353869 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0923 22:15:31.353870 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0923 22:15:31.354047 (+ 177us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0923 22:15:31.354210 (+ 163us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0923 22:15:31.354211 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0923 22:15:31.354211 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0923 22:15:31.354293 (+ 82us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0923 22:15:31.354483 (+ 190us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0923 22:15:31.354486 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0923 22:15:31.354579 (+ 93us) client_negotiation.cc:769] Sending connection context
0923 22:15:31.354655 (+ 76us) client_negotiation.cc:240] Negotiation successful
0923 22:15:31.354776 (+ 121us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":97,"thread_start_us":71,"threads_started":1}
I20260923 22:15:31.355489 17773 heartbeater.cc:347] Connected to a master server at 127.1.135.254:45507
I20260923 22:15:31.355590 17773 heartbeater.cc:464] Registering TS with master...
I20260923 22:15:31.355770 17773 heartbeater.cc:511] Master 127.1.135.254:45507 requested a full tablet report, sending...
I20260923 22:15:31.356176 17447 ts_manager.cc:194] Registered new tserver with Master: 1d61ec2a894f4c76afcd904fdb097bfc (127.1.135.194:33393)
I20260923 22:15:31.356498 1567 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskRiQe_a/build/release/bin/kudu as pid 17640
I20260923 22:15:31.356631 17447 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.1.135.194@KRBTEST.COM'} at 127.1.135.194:38703
I20260923 22:15:31.356799 1567 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskRiQe_a/build/release/bin/kudu
/tmp/dist-test-taskRiQe_a/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskRiQe_a/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790201539513534-1567-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskRiQe_a/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790201539513534-1567-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskRiQe_a/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790201539513534-1567-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskRiQe_a/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790201539513534-1567-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.1.135.195:44883
--local_ip_for_outbound_sockets=127.1.135.195
--tserver_master_addrs=127.1.135.254:45507
--webserver_port=37405
--webserver_interface=127.1.135.195
--keytab_file=/tmp/dist-test-taskRiQe_a/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790201539513534-1567-0/krb5kdc/kudu_127.1.135.195.keytab
--principal=kudu/127.1.135.195
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.1.135.212:34139
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskRiQe_a/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790201539513534-1567-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskRiQe_a/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790201539513534-1567-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskRiQe_a/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790201539513534-1567-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260923 22:15:31.434955 17776 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260923 22:15:31.435142 17776 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260923 22:15:31.435165 17776 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260923 22:15:31.436383 17776 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260923 22:15:31.436422 17776 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260923 22:15:31.436448 17776 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.1.135.195
I20260923 22:15:31.437923 17776 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.1.135.212:34139
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskRiQe_a/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790201539513534-1567-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskRiQe_a/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790201539513534-1567-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskRiQe_a/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790201539513534-1567-0/krb5kdc/kudu_127.1.135.195.keytab
--principal=kudu/127.1.135.195
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.1.135.195:44883
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskRiQe_a/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790201539513534-1567-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.1.135.195
--webserver_port=37405
--tserver_master_addrs=127.1.135.254:45507
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.17776
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.1.135.195
--log_dir=/tmp/dist-test-taskRiQe_a/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790201539513534-1567-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 23 Sep 2026 20:53:37 UTC on ccycloud.quasar-rkviyx.root.comops.site
build id 4668083
I20260923 22:15:31.438138 17776 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260923 22:15:31.438347 17776 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260923 22:15:31.443087 17782 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260923 22:15:31.443094 17781 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260923 22:15:31.443094 17784 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260923 22:15:31.443239 17776 server_base.cc:1034] running on GCE node
I20260923 22:15:31.443442 17776 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260923 22:15:31.443667 17776 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260923 22:15:31.444825 17776 hybrid_clock.cc:648] HybridClock initialized: now 1790201731444812 us; error 28 us; skew 500 ppm
Sep 23 22:15:31 dist-test-slave-s2fj krb5kdc[16869](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790201731, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.135.195@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260923 22:15:31.447311 17776 init.cc:377] Logged in from keytab as kudu/127.1.135.195@KRBTEST.COM (short username kudu)
I20260923 22:15:31.447760 17776 webserver.cc:460] Webserver started at http://127.1.135.195:37405/ using document root <none> and password file <none>
I20260923 22:15:31.447974 17776 fs_manager.cc:359] Metadata directory not provided
I20260923 22:15:31.448052 17776 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260923 22:15:31.449323 17776 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.001s sys 0.000s
I20260923 22:15:31.449868 17791 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260923 22:15:31.450021 17776 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260923 22:15:31.450114 17776 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskRiQe_a/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790201539513534-1567-0/minicluster-data/ts-2/data,/tmp/dist-test-taskRiQe_a/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790201539513534-1567-0/minicluster-data/ts-2/wal
uuid: "43f20a4dc81f48e3bdb7a2281b8505c2"
format_stamp: "Formatted at 2026-09-23 22:15:10 on dist-test-slave-s2fj"
server_key: "c319eb09e0c64ba992b15b7f068e6c56"
server_key_iv: "df4df41ccb3bbb369e9bb5805ddba31c"
server_key_version: "encryptionkey@0"
I20260923 22:15:31.450420 17776 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskRiQe_a/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790201539513534-1567-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskRiQe_a/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790201539513534-1567-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskRiQe_a/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790201539513534-1567-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260923 22:15:31.464598 17776 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260923 22:15:31.466284 17776 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260923 22:15:31.466434 17776 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260923 22:15:31.466640 17776 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260923 22:15:31.466957 17776 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260923 22:15:31.467011 17776 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260923 22:15:31.467051 17776 ts_tablet_manager.cc:610] Registered 0 tablets
I20260923 22:15:31.467079 17776 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260923 22:15:31.472730 17776 rpc_server.cc:307] RPC server started. Bound to: 127.1.135.195:44883
I20260923 22:15:31.472765 17904 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.1.135.195:44883 every 8 connection(s)
I20260923 22:15:31.473102 17776 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskRiQe_a/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790201539513534-1567-0/minicluster-data/ts-2/data/info.pb
Sep 23 22:15:31 dist-test-slave-s2fj krb5kdc[16869](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790201731, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.135.195@KRBTEST.COM for kudu/127.1.135.254@KRBTEST.COM
I20260923 22:15:31.481281 17637 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 22:15:31.473730 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.135.195:43787 (local address 127.1.135.254:45507)
0923 22:15:31.473816 (+ 86us) server_negotiation.cc:206] Beginning negotiation
0923 22:15:31.473818 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0923 22:15:31.474020 (+ 202us) server_negotiation.cc:407] Connection header received
0923 22:15:31.474447 (+ 427us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 22:15:31.474448 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 22:15:31.474458 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 22:15:31.474506 (+ 48us) server_negotiation.cc:226] Negotiated authn=SASL
0923 22:15:31.475362 (+ 856us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 22:15:31.475837 (+ 475us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 22:15:31.476832 (+ 995us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 22:15:31.477001 (+ 169us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 22:15:31.477048 (+ 47us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 22:15:31.478967 (+ 1919us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0923 22:15:31.478968 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0923 22:15:31.478969 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0923 22:15:31.478974 (+ 5us) server_negotiation.cc:922] calling sasl_server_start()
0923 22:15:31.480183 (+ 1209us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0923 22:15:31.480519 (+ 336us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0923 22:15:31.480520 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0923 22:15:31.480520 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0923 22:15:31.480567 (+ 47us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0923 22:15:31.480776 (+ 209us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0923 22:15:31.480776 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0923 22:15:31.480777 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0923 22:15:31.480869 (+ 92us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0923 22:15:31.480918 (+ 49us) server_negotiation.cc:1035] Waiting for connection context
0923 22:15:31.481133 (+ 215us) server_negotiation.cc:299] Negotiation successful
0923 22:15:31.481187 (+ 54us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":51}
I20260923 22:15:31.481380 17907 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 22:15:31.473718 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.135.254:45507 (local address 127.1.135.195:43787)
0923 22:15:31.473883 (+ 165us) negotiation.cc:107] Waiting for socket to connect
0923 22:15:31.473914 (+ 31us) client_negotiation.cc:174] Beginning negotiation
0923 22:15:31.474356 (+ 442us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0923 22:15:31.474556 (+ 200us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0923 22:15:31.474559 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0923 22:15:31.475107 (+ 548us) client_negotiation.cc:189] Negotiated authn=SASL
0923 22:15:31.475287 (+ 180us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 22:15:31.475291 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 22:15:31.475916 (+ 625us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 22:15:31.475916 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 22:15:31.476759 (+ 843us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 22:15:31.476761 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 22:15:31.477068 (+ 307us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 22:15:31.477069 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 22:15:31.477157 (+ 88us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 22:15:31.477544 (+ 387us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0923 22:15:31.477551 (+ 7us) client_negotiation.cc:656] calling sasl_client_start()
0923 22:15:31.478883 (+ 1332us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0923 22:15:31.480250 (+ 1367us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0923 22:15:31.480255 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0923 22:15:31.480257 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0923 22:15:31.480458 (+ 201us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0923 22:15:31.480624 (+ 166us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0923 22:15:31.480624 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0923 22:15:31.480624 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0923 22:15:31.480723 (+ 99us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0923 22:15:31.480922 (+ 199us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0923 22:15:31.480938 (+ 16us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0923 22:15:31.481034 (+ 96us) client_negotiation.cc:769] Sending connection context
0923 22:15:31.481107 (+ 73us) client_negotiation.cc:240] Negotiation successful
0923 22:15:31.481209 (+ 102us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":133,"thread_start_us":103,"threads_started":1}
I20260923 22:15:31.481434 1567 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskRiQe_a/build/release/bin/kudu as pid 17776
I20260923 22:15:31.481945 17905 heartbeater.cc:347] Connected to a master server at 127.1.135.254:45507
I20260923 22:15:31.482043 17905 heartbeater.cc:464] Registering TS with master...
I20260923 22:15:31.482219 17905 heartbeater.cc:511] Master 127.1.135.254:45507 requested a full tablet report, sending...
I20260923 22:15:31.482656 17447 ts_manager.cc:194] Registered new tserver with Master: 43f20a4dc81f48e3bdb7a2281b8505c2 (127.1.135.195:44883)
I20260923 22:15:31.483026 17447 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.1.135.195@KRBTEST.COM'} at 127.1.135.195:43787
I20260923 22:15:31.487968 17637 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 22:15:31.481816 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57852 (local address 127.1.135.254:45507)
0923 22:15:31.481899 (+ 83us) server_negotiation.cc:206] Beginning negotiation
0923 22:15:31.481901 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0923 22:15:31.481993 (+ 92us) server_negotiation.cc:407] Connection header received
0923 22:15:31.482055 (+ 62us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 22:15:31.482056 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 22:15:31.482066 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 22:15:31.482118 (+ 52us) server_negotiation.cc:226] Negotiated authn=SASL
0923 22:15:31.482938 (+ 820us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 22:15:31.483501 (+ 563us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 22:15:31.484355 (+ 854us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 22:15:31.484537 (+ 182us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 22:15:31.484617 (+ 80us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 22:15:31.485794 (+ 1177us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0923 22:15:31.485796 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0923 22:15:31.485796 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0923 22:15:31.485802 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0923 22:15:31.486878 (+ 1076us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0923 22:15:31.487221 (+ 343us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0923 22:15:31.487222 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0923 22:15:31.487223 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0923 22:15:31.487277 (+ 54us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0923 22:15:31.487490 (+ 213us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0923 22:15:31.487491 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0923 22:15:31.487491 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0923 22:15:31.487591 (+ 100us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0923 22:15:31.487778 (+ 187us) server_negotiation.cc:1035] Waiting for connection context
0923 22:15:31.487814 (+ 36us) server_negotiation.cc:299] Negotiation successful
0923 22:15:31.487879 (+ 65us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":63}
I20260923 22:15:31.488636 1567 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260923 22:15:31.491170 17637 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 22:15:31.488958 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57854 (local address 127.1.135.254:45507)
0923 22:15:31.489026 (+ 68us) server_negotiation.cc:206] Beginning negotiation
0923 22:15:31.489028 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0923 22:15:31.489100 (+ 72us) server_negotiation.cc:407] Connection header received
0923 22:15:31.489178 (+ 78us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 22:15:31.489179 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 22:15:31.489220 (+ 41us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 22:15:31.489292 (+ 72us) server_negotiation.cc:226] Negotiated authn=TOKEN
0923 22:15:31.489407 (+ 115us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 22:15:31.489818 (+ 411us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 22:15:31.490509 (+ 691us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 22:15:31.490685 (+ 176us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 22:15:31.490765 (+ 80us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 22:15:31.490933 (+ 168us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0923 22:15:31.490974 (+ 41us) server_negotiation.cc:394] Sending RPC error: FATAL_INVALID_AUTHENTICATION_TOKEN: Not authorized: token expired
0923 22:15:31.491111 (+ 137us) negotiation.cc:327] Negotiation complete: Not authorized: Server connection negotiation failed: server connection from 127.0.0.1:57854: token expired
Metrics: {"server-negotiator.queue_time_us":46}
W20260923 22:15:31.491228 17637 negotiation.cc:344] Unauthorized connection attempt: Server connection negotiation failed: server connection from 127.0.0.1:57854: token expired
W20260923 22:15:31.491302 17378 master_proxy_rpc.cc:203] Re-attempting CreateTable request to leader Master (127.1.135.254:45507)
I20260923 22:15:31.496696 17637 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 22:15:31.491523 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57856 (local address 127.1.135.254:45507)
0923 22:15:31.491553 (+ 30us) server_negotiation.cc:206] Beginning negotiation
0923 22:15:31.491555 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0923 22:15:31.491608 (+ 53us) server_negotiation.cc:407] Connection header received
0923 22:15:31.491665 (+ 57us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 22:15:31.491666 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 22:15:31.491674 (+ 8us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 22:15:31.491725 (+ 51us) server_negotiation.cc:226] Negotiated authn=SASL
0923 22:15:31.492213 (+ 488us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 22:15:31.492640 (+ 427us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 22:15:31.493313 (+ 673us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 22:15:31.493496 (+ 183us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 22:15:31.493578 (+ 82us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 22:15:31.494546 (+ 968us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0923 22:15:31.494548 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0923 22:15:31.494548 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0923 22:15:31.494555 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0923 22:15:31.495613 (+ 1058us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0923 22:15:31.495941 (+ 328us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0923 22:15:31.495941 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0923 22:15:31.495942 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0923 22:15:31.495997 (+ 55us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0923 22:15:31.496241 (+ 244us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0923 22:15:31.496242 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0923 22:15:31.496242 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0923 22:15:31.496344 (+ 102us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0923 22:15:31.496444 (+ 100us) server_negotiation.cc:1035] Waiting for connection context
0923 22:15:31.496545 (+ 101us) server_negotiation.cc:299] Negotiation successful
0923 22:15:31.496610 (+ 65us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":14}
I20260923 22:15:31.497694 17447 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:57856:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260923 22:15:31.499119 17447 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260923 22:15:31.501195 17558 tablet_service.cc:1467] Processing CreateTablet for tablet 40e0f745b16b42e59aca8379b3f40153 (DEFAULT_TABLE table=test-table [id=4e43cd4445a84304a110c1254f6f1bc9]), partition=RANGE (key) PARTITION UNBOUNDED
I20260923 22:15:31.501396 17558 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 40e0f745b16b42e59aca8379b3f40153. 1 dirs total, 0 dirs full, 0 dirs failed
I20260923 22:15:31.502942 17527 tablet_bootstrap.cc:492] T 40e0f745b16b42e59aca8379b3f40153 P de2e7140d2c742e6983e06beafcffd7a: Bootstrap starting.
I20260923 22:15:31.503262 17527 tablet_bootstrap.cc:654] T 40e0f745b16b42e59aca8379b3f40153 P de2e7140d2c742e6983e06beafcffd7a: Neither blocks nor log segments found. Creating new log.
I20260923 22:15:31.503942 17527 tablet_bootstrap.cc:492] T 40e0f745b16b42e59aca8379b3f40153 P de2e7140d2c742e6983e06beafcffd7a: No bootstrap required, opened a new log
I20260923 22:15:31.504019 17527 ts_tablet_manager.cc:1397] T 40e0f745b16b42e59aca8379b3f40153 P de2e7140d2c742e6983e06beafcffd7a: Time spent bootstrapping tablet: real 0.001s user 0.000s sys 0.000s
I20260923 22:15:31.504403 17527 raft_consensus.cc:348] T 40e0f745b16b42e59aca8379b3f40153 P de2e7140d2c742e6983e06beafcffd7a [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "43f20a4dc81f48e3bdb7a2281b8505c2" member_type: VOTER last_known_addr { host: "127.1.135.195" port: 44883 } } peers { permanent_uuid: "1d61ec2a894f4c76afcd904fdb097bfc" member_type: VOTER last_known_addr { host: "127.1.135.194" port: 33393 } } peers { permanent_uuid: "de2e7140d2c742e6983e06beafcffd7a" member_type: VOTER last_known_addr { host: "127.1.135.193" port: 46309 } }
I20260923 22:15:31.504484 17527 raft_consensus.cc:374] T 40e0f745b16b42e59aca8379b3f40153 P de2e7140d2c742e6983e06beafcffd7a [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260923 22:15:31.504522 17527 raft_consensus.cc:729] T 40e0f745b16b42e59aca8379b3f40153 P de2e7140d2c742e6983e06beafcffd7a [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: de2e7140d2c742e6983e06beafcffd7a, State: Initialized, Role: FOLLOWER
I20260923 22:15:31.504577 17527 consensus_queue.cc:260] T 40e0f745b16b42e59aca8379b3f40153 P de2e7140d2c742e6983e06beafcffd7a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "43f20a4dc81f48e3bdb7a2281b8505c2" member_type: VOTER last_known_addr { host: "127.1.135.195" port: 44883 } } peers { permanent_uuid: "1d61ec2a894f4c76afcd904fdb097bfc" member_type: VOTER last_known_addr { host: "127.1.135.194" port: 33393 } } peers { permanent_uuid: "de2e7140d2c742e6983e06beafcffd7a" member_type: VOTER last_known_addr { host: "127.1.135.193" port: 46309 } }
I20260923 22:15:31.504655 17527 ts_tablet_manager.cc:1428] T 40e0f745b16b42e59aca8379b3f40153 P de2e7140d2c742e6983e06beafcffd7a: Time spent starting tablet: real 0.001s user 0.000s sys 0.001s
I20260923 22:15:31.504824 17642 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 22:15:31.500820 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.135.195:44883 (local address 127.0.0.1:55852)
0923 22:15:31.500880 (+ 60us) negotiation.cc:107] Waiting for socket to connect
0923 22:15:31.500890 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0923 22:15:31.500923 (+ 33us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0923 22:15:31.501373 (+ 450us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0923 22:15:31.501374 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0923 22:15:31.501375 (+ 1us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0923 22:15:31.501429 (+ 54us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 22:15:31.501431 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 22:15:31.502573 (+ 1142us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 22:15:31.502574 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 22:15:31.503681 (+ 1107us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 22:15:31.503685 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 22:15:31.504522 (+ 837us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 22:15:31.504523 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 22:15:31.504606 (+ 83us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 22:15:31.504625 (+ 19us) client_negotiation.cc:769] Sending connection context
0923 22:15:31.504702 (+ 77us) client_negotiation.cc:240] Negotiation successful
0923 22:15:31.504745 (+ 43us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":45}
I20260923 22:15:31.504870 17911 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 22:15:31.501056 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55852 (local address 127.1.135.195:44883)
0923 22:15:31.501190 (+ 134us) server_negotiation.cc:206] Beginning negotiation
0923 22:15:31.501195 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0923 22:15:31.501204 (+ 9us) server_negotiation.cc:407] Connection header received
0923 22:15:31.501242 (+ 38us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 22:15:31.501244 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 22:15:31.501258 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 22:15:31.501517 (+ 259us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0923 22:15:31.501595 (+ 78us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 22:15:31.502487 (+ 892us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 22:15:31.503772 (+ 1285us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 22:15:31.504425 (+ 653us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 22:15:31.504519 (+ 94us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 22:15:31.504611 (+ 92us) server_negotiation.cc:1035] Waiting for connection context
0923 22:15:31.504722 (+ 111us) server_negotiation.cc:299] Negotiation successful
0923 22:15:31.504756 (+ 34us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":102,"thread_start_us":74,"threads_started":1}
I20260923 22:15:31.505230 17910 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 22:15:31.500883 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.135.194:33393 (local address 127.0.0.1:57736)
0923 22:15:31.500968 (+ 85us) negotiation.cc:107] Waiting for socket to connect
0923 22:15:31.500976 (+ 8us) client_negotiation.cc:174] Beginning negotiation
0923 22:15:31.501012 (+ 36us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0923 22:15:31.501709 (+ 697us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0923 22:15:31.501710 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0923 22:15:31.501712 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0923 22:15:31.501749 (+ 37us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 22:15:31.501750 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 22:15:31.502981 (+ 1231us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 22:15:31.502982 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 22:15:31.504083 (+ 1101us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 22:15:31.504085 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 22:15:31.505020 (+ 935us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 22:15:31.505021 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 22:15:31.505065 (+ 44us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 22:15:31.505067 (+ 2us) client_negotiation.cc:769] Sending connection context
0923 22:15:31.505128 (+ 61us) client_negotiation.cc:240] Negotiation successful
0923 22:15:31.505165 (+ 37us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":75,"thread_start_us":48,"threads_started":1}
I20260923 22:15:31.505280 17912 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 22:15:31.501306 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57736 (local address 127.1.135.194:33393)
0923 22:15:31.501534 (+ 228us) server_negotiation.cc:206] Beginning negotiation
0923 22:15:31.501538 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0923 22:15:31.501549 (+ 11us) server_negotiation.cc:407] Connection header received
0923 22:15:31.501587 (+ 38us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 22:15:31.501589 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 22:15:31.501603 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 22:15:31.501687 (+ 84us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0923 22:15:31.501840 (+ 153us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 22:15:31.502903 (+ 1063us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 22:15:31.504212 (+ 1309us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 22:15:31.504948 (+ 736us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 22:15:31.505012 (+ 64us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 22:15:31.505065 (+ 53us) server_negotiation.cc:1035] Waiting for connection context
0923 22:15:31.505158 (+ 93us) server_negotiation.cc:299] Negotiation successful
0923 22:15:31.505199 (+ 41us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":190,"thread_start_us":99,"threads_started":1}
I20260923 22:15:31.505607 17839 tablet_service.cc:1467] Processing CreateTablet for tablet 40e0f745b16b42e59aca8379b3f40153 (DEFAULT_TABLE table=test-table [id=4e43cd4445a84304a110c1254f6f1bc9]), partition=RANGE (key) PARTITION UNBOUNDED
I20260923 22:15:31.505847 17839 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 40e0f745b16b42e59aca8379b3f40153. 1 dirs total, 0 dirs full, 0 dirs failed
I20260923 22:15:31.508525 17914 tablet_bootstrap.cc:492] T 40e0f745b16b42e59aca8379b3f40153 P 43f20a4dc81f48e3bdb7a2281b8505c2: Bootstrap starting.
I20260923 22:15:31.508965 17707 tablet_service.cc:1467] Processing CreateTablet for tablet 40e0f745b16b42e59aca8379b3f40153 (DEFAULT_TABLE table=test-table [id=4e43cd4445a84304a110c1254f6f1bc9]), partition=RANGE (key) PARTITION UNBOUNDED
I20260923 22:15:31.509131 17914 tablet_bootstrap.cc:654] T 40e0f745b16b42e59aca8379b3f40153 P 43f20a4dc81f48e3bdb7a2281b8505c2: Neither blocks nor log segments found. Creating new log.
I20260923 22:15:31.509244 17707 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 40e0f745b16b42e59aca8379b3f40153. 1 dirs total, 0 dirs full, 0 dirs failed
I20260923 22:15:31.509667 17914 log.cc:824] T 40e0f745b16b42e59aca8379b3f40153 P 43f20a4dc81f48e3bdb7a2281b8505c2: Log is configured to *not* fsync() on all Append() calls
I20260923 22:15:31.510453 17914 tablet_bootstrap.cc:492] T 40e0f745b16b42e59aca8379b3f40153 P 43f20a4dc81f48e3bdb7a2281b8505c2: No bootstrap required, opened a new log
I20260923 22:15:31.510542 17914 ts_tablet_manager.cc:1397] T 40e0f745b16b42e59aca8379b3f40153 P 43f20a4dc81f48e3bdb7a2281b8505c2: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20260923 22:15:31.511842 17914 raft_consensus.cc:348] T 40e0f745b16b42e59aca8379b3f40153 P 43f20a4dc81f48e3bdb7a2281b8505c2 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "43f20a4dc81f48e3bdb7a2281b8505c2" member_type: VOTER last_known_addr { host: "127.1.135.195" port: 44883 } } peers { permanent_uuid: "1d61ec2a894f4c76afcd904fdb097bfc" member_type: VOTER last_known_addr { host: "127.1.135.194" port: 33393 } } peers { permanent_uuid: "de2e7140d2c742e6983e06beafcffd7a" member_type: VOTER last_known_addr { host: "127.1.135.193" port: 46309 } }
I20260923 22:15:31.511976 17914 raft_consensus.cc:374] T 40e0f745b16b42e59aca8379b3f40153 P 43f20a4dc81f48e3bdb7a2281b8505c2 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260923 22:15:31.511997 17914 raft_consensus.cc:729] T 40e0f745b16b42e59aca8379b3f40153 P 43f20a4dc81f48e3bdb7a2281b8505c2 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 43f20a4dc81f48e3bdb7a2281b8505c2, State: Initialized, Role: FOLLOWER
I20260923 22:15:31.512149 17914 consensus_queue.cc:260] T 40e0f745b16b42e59aca8379b3f40153 P 43f20a4dc81f48e3bdb7a2281b8505c2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "43f20a4dc81f48e3bdb7a2281b8505c2" member_type: VOTER last_known_addr { host: "127.1.135.195" port: 44883 } } peers { permanent_uuid: "1d61ec2a894f4c76afcd904fdb097bfc" member_type: VOTER last_known_addr { host: "127.1.135.194" port: 33393 } } peers { permanent_uuid: "de2e7140d2c742e6983e06beafcffd7a" member_type: VOTER last_known_addr { host: "127.1.135.193" port: 46309 } }
I20260923 22:15:31.512224 17916 tablet_bootstrap.cc:492] T 40e0f745b16b42e59aca8379b3f40153 P 1d61ec2a894f4c76afcd904fdb097bfc: Bootstrap starting.
I20260923 22:15:31.512421 17914 ts_tablet_manager.cc:1428] T 40e0f745b16b42e59aca8379b3f40153 P 43f20a4dc81f48e3bdb7a2281b8505c2: Time spent starting tablet: real 0.002s user 0.000s sys 0.002s
I20260923 22:15:31.512426 17905 heartbeater.cc:503] Master 127.1.135.254:45507 was elected leader, sending a full tablet report...
I20260923 22:15:31.512799 17916 tablet_bootstrap.cc:654] T 40e0f745b16b42e59aca8379b3f40153 P 1d61ec2a894f4c76afcd904fdb097bfc: Neither blocks nor log segments found. Creating new log.
I20260923 22:15:31.513267 17916 log.cc:824] T 40e0f745b16b42e59aca8379b3f40153 P 1d61ec2a894f4c76afcd904fdb097bfc: Log is configured to *not* fsync() on all Append() calls
I20260923 22:15:31.513880 17916 tablet_bootstrap.cc:492] T 40e0f745b16b42e59aca8379b3f40153 P 1d61ec2a894f4c76afcd904fdb097bfc: No bootstrap required, opened a new log
I20260923 22:15:31.513958 17916 ts_tablet_manager.cc:1397] T 40e0f745b16b42e59aca8379b3f40153 P 1d61ec2a894f4c76afcd904fdb097bfc: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20260923 22:15:31.515204 17916 raft_consensus.cc:348] T 40e0f745b16b42e59aca8379b3f40153 P 1d61ec2a894f4c76afcd904fdb097bfc [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "43f20a4dc81f48e3bdb7a2281b8505c2" member_type: VOTER last_known_addr { host: "127.1.135.195" port: 44883 } } peers { permanent_uuid: "1d61ec2a894f4c76afcd904fdb097bfc" member_type: VOTER last_known_addr { host: "127.1.135.194" port: 33393 } } peers { permanent_uuid: "de2e7140d2c742e6983e06beafcffd7a" member_type: VOTER last_known_addr { host: "127.1.135.193" port: 46309 } }
I20260923 22:15:31.515352 17916 raft_consensus.cc:374] T 40e0f745b16b42e59aca8379b3f40153 P 1d61ec2a894f4c76afcd904fdb097bfc [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260923 22:15:31.515398 17916 raft_consensus.cc:729] T 40e0f745b16b42e59aca8379b3f40153 P 1d61ec2a894f4c76afcd904fdb097bfc [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 1d61ec2a894f4c76afcd904fdb097bfc, State: Initialized, Role: FOLLOWER
I20260923 22:15:31.515514 17916 consensus_queue.cc:260] T 40e0f745b16b42e59aca8379b3f40153 P 1d61ec2a894f4c76afcd904fdb097bfc [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "43f20a4dc81f48e3bdb7a2281b8505c2" member_type: VOTER last_known_addr { host: "127.1.135.195" port: 44883 } } peers { permanent_uuid: "1d61ec2a894f4c76afcd904fdb097bfc" member_type: VOTER last_known_addr { host: "127.1.135.194" port: 33393 } } peers { permanent_uuid: "de2e7140d2c742e6983e06beafcffd7a" member_type: VOTER last_known_addr { host: "127.1.135.193" port: 46309 } }
I20260923 22:15:31.515777 17916 ts_tablet_manager.cc:1428] T 40e0f745b16b42e59aca8379b3f40153 P 1d61ec2a894f4c76afcd904fdb097bfc: Time spent starting tablet: real 0.002s user 0.002s sys 0.000s
I20260923 22:15:31.515813 17773 heartbeater.cc:503] Master 127.1.135.254:45507 was elected leader, sending a full tablet report...
W20260923 22:15:31.597259 17774 tablet.cc:2369] T 40e0f745b16b42e59aca8379b3f40153 P 1d61ec2a894f4c76afcd904fdb097bfc: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260923 22:15:31.706794 17917 raft_consensus.cc:482] T 40e0f745b16b42e59aca8379b3f40153 P 43f20a4dc81f48e3bdb7a2281b8505c2 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260923 22:15:31.706910 17917 raft_consensus.cc:504] T 40e0f745b16b42e59aca8379b3f40153 P 43f20a4dc81f48e3bdb7a2281b8505c2 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "43f20a4dc81f48e3bdb7a2281b8505c2" member_type: VOTER last_known_addr { host: "127.1.135.195" port: 44883 } } peers { permanent_uuid: "1d61ec2a894f4c76afcd904fdb097bfc" member_type: VOTER last_known_addr { host: "127.1.135.194" port: 33393 } } peers { permanent_uuid: "de2e7140d2c742e6983e06beafcffd7a" member_type: VOTER last_known_addr { host: "127.1.135.193" port: 46309 } }
I20260923 22:15:31.707299 17917 leader_election.cc:288] T 40e0f745b16b42e59aca8379b3f40153 P 43f20a4dc81f48e3bdb7a2281b8505c2 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 1d61ec2a894f4c76afcd904fdb097bfc (127.1.135.194:33393), de2e7140d2c742e6983e06beafcffd7a (127.1.135.193:46309)
I20260923 22:15:31.710129 17912 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 22:15:31.707559 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.135.195:45913 (local address 127.1.135.194:33393)
0923 22:15:31.707617 (+ 58us) server_negotiation.cc:206] Beginning negotiation
0923 22:15:31.707619 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0923 22:15:31.707629 (+ 10us) server_negotiation.cc:407] Connection header received
0923 22:15:31.707745 (+ 116us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 22:15:31.707746 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 22:15:31.707753 (+ 7us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 22:15:31.707810 (+ 57us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0923 22:15:31.707962 (+ 152us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 22:15:31.708503 (+ 541us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 22:15:31.709410 (+ 907us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 22:15:31.709783 (+ 373us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 22:15:31.709867 (+ 84us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 22:15:31.709875 (+ 8us) server_negotiation.cc:1035] Waiting for connection context
0923 22:15:31.710028 (+ 153us) server_negotiation.cc:299] Negotiation successful
0923 22:15:31.710061 (+ 33us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":33}
I20260923 22:15:31.710127 17907 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 22:15:31.707491 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.135.194:33393 (local address 127.1.135.195:45913)
0923 22:15:31.707546 (+ 55us) negotiation.cc:107] Waiting for socket to connect
0923 22:15:31.707558 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0923 22:15:31.707631 (+ 73us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0923 22:15:31.707826 (+ 195us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0923 22:15:31.707827 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0923 22:15:31.707829 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0923 22:15:31.707900 (+ 71us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 22:15:31.707902 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 22:15:31.708584 (+ 682us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 22:15:31.708585 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 22:15:31.709311 (+ 726us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 22:15:31.709313 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 22:15:31.709861 (+ 548us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 22:15:31.709862 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 22:15:31.709937 (+ 75us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 22:15:31.709940 (+ 3us) client_negotiation.cc:769] Sending connection context
0923 22:15:31.710010 (+ 70us) client_negotiation.cc:240] Negotiation successful
0923 22:15:31.710046 (+ 36us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":34}
I20260923 22:15:31.710320 17643 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 22:15:31.707661 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.135.195:57325 (local address 127.1.135.193:46309)
0923 22:15:31.707704 (+ 43us) server_negotiation.cc:206] Beginning negotiation
0923 22:15:31.707706 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0923 22:15:31.707845 (+ 139us) server_negotiation.cc:407] Connection header received
0923 22:15:31.707904 (+ 59us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 22:15:31.707905 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 22:15:31.707930 (+ 25us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 22:15:31.708003 (+ 73us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0923 22:15:31.708245 (+ 242us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 22:15:31.708710 (+ 465us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 22:15:31.709583 (+ 873us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 22:15:31.709962 (+ 379us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 22:15:31.710047 (+ 85us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 22:15:31.710129 (+ 82us) server_negotiation.cc:1035] Waiting for connection context
0923 22:15:31.710175 (+ 46us) server_negotiation.cc:299] Negotiation successful
0923 22:15:31.710212 (+ 37us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":21}
I20260923 22:15:31.710449 17727 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "40e0f745b16b42e59aca8379b3f40153" candidate_uuid: "43f20a4dc81f48e3bdb7a2281b8505c2" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1d61ec2a894f4c76afcd904fdb097bfc" is_pre_election: true
I20260923 22:15:31.710512 17920 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 22:15:31.707599 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.135.193:46309 (local address 127.1.135.195:57325)
0923 22:15:31.707763 (+ 164us) negotiation.cc:107] Waiting for socket to connect
0923 22:15:31.707771 (+ 8us) client_negotiation.cc:174] Beginning negotiation
0923 22:15:31.707850 (+ 79us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0923 22:15:31.708008 (+ 158us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0923 22:15:31.708009 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0923 22:15:31.708010 (+ 1us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0923 22:15:31.708146 (+ 136us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 22:15:31.708148 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 22:15:31.708793 (+ 645us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 22:15:31.708794 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 22:15:31.709483 (+ 689us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 22:15:31.709485 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 22:15:31.710045 (+ 560us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 22:15:31.710046 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 22:15:31.710107 (+ 61us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 22:15:31.710123 (+ 16us) client_negotiation.cc:769] Sending connection context
0923 22:15:31.710167 (+ 44us) client_negotiation.cc:240] Negotiation successful
0923 22:15:31.710438 (+ 271us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":140,"thread_start_us":108,"threads_started":1}
I20260923 22:15:31.710596 17727 raft_consensus.cc:2454] T 40e0f745b16b42e59aca8379b3f40153 P 1d61ec2a894f4c76afcd904fdb097bfc [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 43f20a4dc81f48e3bdb7a2281b8505c2 in term 0.
I20260923 22:15:31.710796 17794 leader_election.cc:302] T 40e0f745b16b42e59aca8379b3f40153 P 43f20a4dc81f48e3bdb7a2281b8505c2 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1d61ec2a894f4c76afcd904fdb097bfc, 43f20a4dc81f48e3bdb7a2281b8505c2; no voters:
I20260923 22:15:31.710793 17589 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "40e0f745b16b42e59aca8379b3f40153" candidate_uuid: "43f20a4dc81f48e3bdb7a2281b8505c2" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "de2e7140d2c742e6983e06beafcffd7a" is_pre_election: true
I20260923 22:15:31.710919 17589 raft_consensus.cc:2454] T 40e0f745b16b42e59aca8379b3f40153 P de2e7140d2c742e6983e06beafcffd7a [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 43f20a4dc81f48e3bdb7a2281b8505c2 in term 0.
I20260923 22:15:31.710925 17917 raft_consensus.cc:2781] T 40e0f745b16b42e59aca8379b3f40153 P 43f20a4dc81f48e3bdb7a2281b8505c2 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260923 22:15:31.710994 17917 raft_consensus.cc:482] T 40e0f745b16b42e59aca8379b3f40153 P 43f20a4dc81f48e3bdb7a2281b8505c2 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260923 22:15:31.711024 17917 raft_consensus.cc:3037] T 40e0f745b16b42e59aca8379b3f40153 P 43f20a4dc81f48e3bdb7a2281b8505c2 [term 0 FOLLOWER]: Advancing to term 1
I20260923 22:15:31.711696 17917 raft_consensus.cc:504] T 40e0f745b16b42e59aca8379b3f40153 P 43f20a4dc81f48e3bdb7a2281b8505c2 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "43f20a4dc81f48e3bdb7a2281b8505c2" member_type: VOTER last_known_addr { host: "127.1.135.195" port: 44883 } } peers { permanent_uuid: "1d61ec2a894f4c76afcd904fdb097bfc" member_type: VOTER last_known_addr { host: "127.1.135.194" port: 33393 } } peers { permanent_uuid: "de2e7140d2c742e6983e06beafcffd7a" member_type: VOTER last_known_addr { host: "127.1.135.193" port: 46309 } }
I20260923 22:15:31.711863 17917 leader_election.cc:288] T 40e0f745b16b42e59aca8379b3f40153 P 43f20a4dc81f48e3bdb7a2281b8505c2 [CANDIDATE]: Term 1 election: Requested vote from peers 1d61ec2a894f4c76afcd904fdb097bfc (127.1.135.194:33393), de2e7140d2c742e6983e06beafcffd7a (127.1.135.193:46309)
I20260923 22:15:31.712046 17589 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "40e0f745b16b42e59aca8379b3f40153" candidate_uuid: "43f20a4dc81f48e3bdb7a2281b8505c2" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "de2e7140d2c742e6983e06beafcffd7a"
I20260923 22:15:31.712033 17727 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "40e0f745b16b42e59aca8379b3f40153" candidate_uuid: "43f20a4dc81f48e3bdb7a2281b8505c2" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1d61ec2a894f4c76afcd904fdb097bfc"
I20260923 22:15:31.712138 17589 raft_consensus.cc:3037] T 40e0f745b16b42e59aca8379b3f40153 P de2e7140d2c742e6983e06beafcffd7a [term 0 FOLLOWER]: Advancing to term 1
I20260923 22:15:31.712148 17727 raft_consensus.cc:3037] T 40e0f745b16b42e59aca8379b3f40153 P 1d61ec2a894f4c76afcd904fdb097bfc [term 0 FOLLOWER]: Advancing to term 1
I20260923 22:15:31.712823 17727 raft_consensus.cc:2454] T 40e0f745b16b42e59aca8379b3f40153 P 1d61ec2a894f4c76afcd904fdb097bfc [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 43f20a4dc81f48e3bdb7a2281b8505c2 in term 1.
I20260923 22:15:31.712882 17589 raft_consensus.cc:2454] T 40e0f745b16b42e59aca8379b3f40153 P de2e7140d2c742e6983e06beafcffd7a [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 43f20a4dc81f48e3bdb7a2281b8505c2 in term 1.
I20260923 22:15:31.713027 17794 leader_election.cc:302] T 40e0f745b16b42e59aca8379b3f40153 P 43f20a4dc81f48e3bdb7a2281b8505c2 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1d61ec2a894f4c76afcd904fdb097bfc, 43f20a4dc81f48e3bdb7a2281b8505c2; no voters:
I20260923 22:15:31.713120 17917 raft_consensus.cc:2781] T 40e0f745b16b42e59aca8379b3f40153 P 43f20a4dc81f48e3bdb7a2281b8505c2 [term 1 FOLLOWER]: Leader election won for term 1
I20260923 22:15:31.713239 17917 raft_consensus.cc:686] T 40e0f745b16b42e59aca8379b3f40153 P 43f20a4dc81f48e3bdb7a2281b8505c2 [term 1 LEADER]: Becoming Leader. State: Replica: 43f20a4dc81f48e3bdb7a2281b8505c2, State: Running, Role: LEADER
I20260923 22:15:31.713335 17917 consensus_queue.cc:237] T 40e0f745b16b42e59aca8379b3f40153 P 43f20a4dc81f48e3bdb7a2281b8505c2 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "43f20a4dc81f48e3bdb7a2281b8505c2" member_type: VOTER last_known_addr { host: "127.1.135.195" port: 44883 } } peers { permanent_uuid: "1d61ec2a894f4c76afcd904fdb097bfc" member_type: VOTER last_known_addr { host: "127.1.135.194" port: 33393 } } peers { permanent_uuid: "de2e7140d2c742e6983e06beafcffd7a" member_type: VOTER last_known_addr { host: "127.1.135.193" port: 46309 } }
I20260923 22:15:31.714124 17447 catalog_manager.cc:5495] T 40e0f745b16b42e59aca8379b3f40153 P 43f20a4dc81f48e3bdb7a2281b8505c2 reported cstate change: term changed from 0 to 1, leader changed from <none> to 43f20a4dc81f48e3bdb7a2281b8505c2 (127.1.135.195). New cstate: current_term: 1 leader_uuid: "43f20a4dc81f48e3bdb7a2281b8505c2" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "43f20a4dc81f48e3bdb7a2281b8505c2" member_type: VOTER last_known_addr { host: "127.1.135.195" port: 44883 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "1d61ec2a894f4c76afcd904fdb097bfc" member_type: VOTER last_known_addr { host: "127.1.135.194" port: 33393 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "de2e7140d2c742e6983e06beafcffd7a" member_type: VOTER last_known_addr { host: "127.1.135.193" port: 46309 } health_report { overall_health: UNKNOWN } } }
W20260923 22:15:31.723592 17906 tablet.cc:2369] T 40e0f745b16b42e59aca8379b3f40153 P 43f20a4dc81f48e3bdb7a2281b8505c2: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260923 22:15:31.729068 17636 tablet.cc:2369] T 40e0f745b16b42e59aca8379b3f40153 P de2e7140d2c742e6983e06beafcffd7a: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260923 22:15:31.731254 17911 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 22:15:31.729114 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55856 (local address 127.1.135.195:44883)
0923 22:15:31.729178 (+ 64us) server_negotiation.cc:206] Beginning negotiation
0923 22:15:31.729180 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0923 22:15:31.729191 (+ 11us) server_negotiation.cc:407] Connection header received
0923 22:15:31.729220 (+ 29us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 22:15:31.729220 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 22:15:31.729243 (+ 23us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 22:15:31.729302 (+ 59us) server_negotiation.cc:226] Negotiated authn=TOKEN
0923 22:15:31.729427 (+ 125us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 22:15:31.729848 (+ 421us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 22:15:31.730509 (+ 661us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 22:15:31.730706 (+ 197us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 22:15:31.730758 (+ 52us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 22:15:31.730920 (+ 162us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0923 22:15:31.731002 (+ 82us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0923 22:15:31.731058 (+ 56us) server_negotiation.cc:1035] Waiting for connection context
0923 22:15:31.731125 (+ 67us) server_negotiation.cc:299] Negotiation successful
0923 22:15:31.731144 (+ 19us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":43}
I20260923 22:15:31.732893 17589 raft_consensus.cc:1261] T 40e0f745b16b42e59aca8379b3f40153 P de2e7140d2c742e6983e06beafcffd7a [term 1 FOLLOWER]: Refusing update from remote peer 43f20a4dc81f48e3bdb7a2281b8505c2: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260923 22:15:31.732920 17727 raft_consensus.cc:1261] T 40e0f745b16b42e59aca8379b3f40153 P 1d61ec2a894f4c76afcd904fdb097bfc [term 1 FOLLOWER]: Refusing update from remote peer 43f20a4dc81f48e3bdb7a2281b8505c2: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260923 22:15:31.733232 17917 consensus_queue.cc:1035] T 40e0f745b16b42e59aca8379b3f40153 P 43f20a4dc81f48e3bdb7a2281b8505c2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "1d61ec2a894f4c76afcd904fdb097bfc" member_type: VOTER last_known_addr { host: "127.1.135.194" port: 33393 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260923 22:15:31.733356 17921 consensus_queue.cc:1035] T 40e0f745b16b42e59aca8379b3f40153 P 43f20a4dc81f48e3bdb7a2281b8505c2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "de2e7140d2c742e6983e06beafcffd7a" member_type: VOTER last_known_addr { host: "127.1.135.193" port: 46309 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260923 22:15:31.736580 17925 mvcc.cc:205] Tried to move back new op lower bound from 7332666293175562240 to 7332666293098463232. Current Snapshot: MvccSnapshot[applied={T|T < 7332666293175562240 or (T in {7332666293175562240})}]
I20260923 22:15:31.737800 17447 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:57856:
table { table_name: "test-table" } modify_external_catalogs: true
I20260923 22:15:31.737883 17447 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:57856:
table { table_name: "test-table" } modify_external_catalogs: true
I20260923 22:15:31.738582 17447 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 2d2671e82fe14a6fadf679b384125ea3: Sending DeleteTablet for 3 replicas of tablet 40e0f745b16b42e59aca8379b3f40153
I20260923 22:15:31.738866 17839 tablet_service.cc:1514] Processing DeleteTablet for tablet 40e0f745b16b42e59aca8379b3f40153 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-23 22:15:31 UTC) from {username='kudu', principal='kudu/127.1.135.254@KRBTEST.COM'} at 127.0.0.1:55852
I20260923 22:15:31.738899 17707 tablet_service.cc:1514] Processing DeleteTablet for tablet 40e0f745b16b42e59aca8379b3f40153 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-23 22:15:31 UTC) from {username='kudu', principal='kudu/127.1.135.254@KRBTEST.COM'} at 127.0.0.1:57736
I20260923 22:15:31.738929 1567 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskRiQe_a/build/release/bin/kudu with pid 17485
I20260923 22:15:31.738945 17558 tablet_service.cc:1514] Processing DeleteTablet for tablet 40e0f745b16b42e59aca8379b3f40153 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-23 22:15:31 UTC) from {username='kudu', principal='kudu/127.1.135.254@KRBTEST.COM'} at 127.0.0.1:34466
I20260923 22:15:31.739029 17644 tablet_replica.cc:330] T 40e0f745b16b42e59aca8379b3f40153 P de2e7140d2c742e6983e06beafcffd7a: stopping tablet replica
I20260923 22:15:31.739073 17644 raft_consensus.cc:2229] T 40e0f745b16b42e59aca8379b3f40153 P de2e7140d2c742e6983e06beafcffd7a [term 1 FOLLOWER]: Raft consensus shutting down.
I20260923 22:15:31.739110 17644 raft_consensus.cc:2258] T 40e0f745b16b42e59aca8379b3f40153 P de2e7140d2c742e6983e06beafcffd7a [term 1 FOLLOWER]: Raft consensus is shut down!
I20260923 22:15:31.740912 17935 tablet_replica.cc:330] T 40e0f745b16b42e59aca8379b3f40153 P 43f20a4dc81f48e3bdb7a2281b8505c2: stopping tablet replica
I20260923 22:15:31.741017 17935 raft_consensus.cc:2229] T 40e0f745b16b42e59aca8379b3f40153 P 43f20a4dc81f48e3bdb7a2281b8505c2 [term 1 LEADER]: Raft consensus shutting down.
I20260923 22:15:31.741110 17935 raft_consensus.cc:2258] T 40e0f745b16b42e59aca8379b3f40153 P 43f20a4dc81f48e3bdb7a2281b8505c2 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260923 22:15:31.741285 17934 tablet_replica.cc:330] T 40e0f745b16b42e59aca8379b3f40153 P 1d61ec2a894f4c76afcd904fdb097bfc: stopping tablet replica
I20260923 22:15:31.741380 17934 raft_consensus.cc:2229] T 40e0f745b16b42e59aca8379b3f40153 P 1d61ec2a894f4c76afcd904fdb097bfc [term 1 FOLLOWER]: Raft consensus shutting down.
I20260923 22:15:31.741480 17934 raft_consensus.cc:2258] T 40e0f745b16b42e59aca8379b3f40153 P 1d61ec2a894f4c76afcd904fdb097bfc [term 1 FOLLOWER]: Raft consensus is shut down!
I20260923 22:15:31.741585 17935 ts_tablet_manager.cc:1905] T 40e0f745b16b42e59aca8379b3f40153 P 43f20a4dc81f48e3bdb7a2281b8505c2: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260923 22:15:31.741889 17934 ts_tablet_manager.cc:1905] T 40e0f745b16b42e59aca8379b3f40153 P 1d61ec2a894f4c76afcd904fdb097bfc: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260923 22:15:31.743777 17934 ts_tablet_manager.cc:1918] T 40e0f745b16b42e59aca8379b3f40153 P 1d61ec2a894f4c76afcd904fdb097bfc: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260923 22:15:31.743850 17934 log.cc:1196] T 40e0f745b16b42e59aca8379b3f40153 P 1d61ec2a894f4c76afcd904fdb097bfc: Deleting WAL directory at /tmp/dist-test-taskRiQe_a/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790201539513534-1567-0/minicluster-data/ts-1/wal/wals/40e0f745b16b42e59aca8379b3f40153
I20260923 22:15:31.743923 17935 ts_tablet_manager.cc:1918] T 40e0f745b16b42e59aca8379b3f40153 P 43f20a4dc81f48e3bdb7a2281b8505c2: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260923 22:15:31.744043 17935 log.cc:1196] T 40e0f745b16b42e59aca8379b3f40153 P 43f20a4dc81f48e3bdb7a2281b8505c2: Deleting WAL directory at /tmp/dist-test-taskRiQe_a/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790201539513534-1567-0/minicluster-data/ts-2/wal/wals/40e0f745b16b42e59aca8379b3f40153
I20260923 22:15:31.744150 17934 ts_tablet_manager.cc:1939] T 40e0f745b16b42e59aca8379b3f40153 P 1d61ec2a894f4c76afcd904fdb097bfc: Deleting consensus metadata
I20260923 22:15:31.744448 17935 ts_tablet_manager.cc:1939] T 40e0f745b16b42e59aca8379b3f40153 P 43f20a4dc81f48e3bdb7a2281b8505c2: Deleting consensus metadata
I20260923 22:15:31.744570 17433 catalog_manager.cc:4841] TS 1d61ec2a894f4c76afcd904fdb097bfc (127.1.135.194:33393): tablet 40e0f745b16b42e59aca8379b3f40153 (table test-table [id=4e43cd4445a84304a110c1254f6f1bc9]) successfully deleted
I20260923 22:15:31.744928 17434 catalog_manager.cc:4841] TS 43f20a4dc81f48e3bdb7a2281b8505c2 (127.1.135.195:44883): tablet 40e0f745b16b42e59aca8379b3f40153 (table test-table [id=4e43cd4445a84304a110c1254f6f1bc9]) successfully deleted
W20260923 22:15:31.745625 17433 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.1.135.193:46309 (error 108)
I20260923 22:15:31.746028 17910 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 22:15:31.745927 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.135.193:46309 (local address 127.0.0.1:34474)
0923 22:15:31.745969 (+ 42us) negotiation.cc:107] Waiting for socket to connect
0923 22:15:31.746000 (+ 31us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.135.193:46309: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":26}
I20260923 22:15:31.746049 1567 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskRiQe_a/build/release/bin/kudu with pid 17640
W20260923 22:15:31.746151 17433 catalog_manager.cc:4584] TS de2e7140d2c742e6983e06beafcffd7a (127.1.135.193:46309): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet 40e0f745b16b42e59aca8379b3f40153: Network error: Client connection negotiation failed: client connection to 127.1.135.193:46309: connect: Connection refused (error 111)
I20260923 22:15:31.751739 1567 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskRiQe_a/build/release/bin/kudu with pid 17776
I20260923 22:15:31.757050 1567 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskRiQe_a/build/release/bin/kudu with pid 17414
I20260923 22:15:51.762351 1567 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskRiQe_a/build/release/bin/kudu
/tmp/dist-test-taskRiQe_a/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskRiQe_a/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790201539513534-1567-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskRiQe_a/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790201539513534-1567-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskRiQe_a/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790201539513534-1567-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskRiQe_a/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790201539513534-1567-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.1.135.254:45507
--webserver_interface=127.1.135.254
--webserver_port=34177
--keytab_file=/tmp/dist-test-taskRiQe_a/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790201539513534-1567-0/krb5kdc/kudu_127.1.135.254.keytab
--principal=kudu/127.1.135.254
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.1.135.212:34139
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.1.135.254:45507
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--authn_token_validity_seconds=20
--authz_token_validity_seconds=20
--tsk_rotation_seconds=20 with env {KRB5CCNAME=/tmp/dist-test-taskRiQe_a/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790201539513534-1567-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskRiQe_a/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790201539513534-1567-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskRiQe_a/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790201539513534-1567-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260923 22:15:51.841178 17937 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260923 22:15:51.841379 17937 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260923 22:15:51.841413 17937 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260923 22:15:51.842540 17937 flags.cc:425] Enabled experimental flag: --authn_token_validity_seconds=20
W20260923 22:15:51.842598 17937 flags.cc:425] Enabled experimental flag: --authz_token_validity_seconds=20
W20260923 22:15:51.842618 17937 flags.cc:425] Enabled experimental flag: --tsk_rotation_seconds=20
W20260923 22:15:51.842636 17937 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260923 22:15:51.842665 17937 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260923 22:15:51.842692 17937 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260923 22:15:51.842716 17937 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260923 22:15:51.842746 17937 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260923 22:15:51.844218 17937 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.1.135.212:34139
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskRiQe_a/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790201539513534-1567-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskRiQe_a/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790201539513534-1567-0/minicluster-data/master-0/wal
--authn_token_validity_seconds=20
--authz_token_validity_seconds=20
--tsk_rotation_seconds=20
--ipki_ca_key_size=768
--master_addresses=127.1.135.254:45507
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskRiQe_a/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790201539513534-1567-0/krb5kdc/kudu_127.1.135.254.keytab
--principal=kudu/127.1.135.254
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.1.135.254:45507
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskRiQe_a/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790201539513534-1567-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.1.135.254
--webserver_port=34177
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.17937
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskRiQe_a/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790201539513534-1567-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 23 Sep 2026 20:53:37 UTC on ccycloud.quasar-rkviyx.root.comops.site
build id 4668083
I20260923 22:15:51.844477 17937 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260923 22:15:51.844684 17937 file_cache.cc:493] Constructed file cache file cache with capacity 419430
I20260923 22:15:51.849252 17937 server_base.cc:1034] running on GCE node
W20260923 22:15:51.849231 17945 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260923 22:15:51.849226 17943 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260923 22:15:51.849226 17942 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260923 22:15:51.849627 17937 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260923 22:15:51.849898 17937 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260923 22:15:51.851058 17937 hybrid_clock.cc:648] HybridClock initialized: now 1790201751851048 us; error 35 us; skew 500 ppm
Sep 23 22:15:51 dist-test-slave-s2fj krb5kdc[16869](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790201751, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.135.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260923 22:15:51.853336 17937 init.cc:377] Logged in from keytab as kudu/127.1.135.254@KRBTEST.COM (short username kudu)
I20260923 22:15:51.853731 17937 webserver.cc:460] Webserver started at http://127.1.135.254:34177/ using document root <none> and password file <none>
I20260923 22:15:51.853940 17937 fs_manager.cc:359] Metadata directory not provided
I20260923 22:15:51.854000 17937 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260923 22:15:51.855258 17937 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.001s sys 0.000s
I20260923 22:15:51.855926 17952 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260923 22:15:51.856130 17937 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260923 22:15:51.856225 17937 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskRiQe_a/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790201539513534-1567-0/minicluster-data/master-0/data,/tmp/dist-test-taskRiQe_a/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790201539513534-1567-0/minicluster-data/master-0/wal
uuid: "2d2671e82fe14a6fadf679b384125ea3"
format_stamp: "Formatted at 2026-09-23 22:15:10 on dist-test-slave-s2fj"
server_key: "b4b3416d8d0a070e64cead2cd4d90893"
server_key_iv: "6a4a527f9c3e1de279d16239eceaa956"
server_key_version: "encryptionkey@0"
I20260923 22:15:51.856566 17937 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskRiQe_a/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790201539513534-1567-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskRiQe_a/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790201539513534-1567-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskRiQe_a/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790201539513534-1567-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260923 22:15:51.872488 17937 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260923 22:15:51.874367 17937 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260923 22:15:51.874538 17937 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260923 22:15:51.878402 17937 rpc_server.cc:307] RPC server started. Bound to: 127.1.135.254:45507
I20260923 22:15:51.878439 18004 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.1.135.254:45507 every 8 connection(s)
I20260923 22:15:51.878816 17937 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskRiQe_a/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790201539513534-1567-0/minicluster-data/master-0/data/info.pb
I20260923 22:15:51.880210 18005 sys_catalog.cc:263] Verifying existing consensus state
I20260923 22:15:51.880839 18005 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 2d2671e82fe14a6fadf679b384125ea3: Bootstrap starting.
I20260923 22:15:51.887082 1567 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskRiQe_a/build/release/bin/kudu as pid 17937
I20260923 22:15:51.887373 1567 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskRiQe_a/build/release/bin/kudu
/tmp/dist-test-taskRiQe_a/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskRiQe_a/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790201539513534-1567-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskRiQe_a/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790201539513534-1567-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskRiQe_a/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790201539513534-1567-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskRiQe_a/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790201539513534-1567-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.1.135.193:46309
--local_ip_for_outbound_sockets=127.1.135.193
--tserver_master_addrs=127.1.135.254:45507
--webserver_port=35971
--webserver_interface=127.1.135.193
--keytab_file=/tmp/dist-test-taskRiQe_a/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790201539513534-1567-0/krb5kdc/kudu_127.1.135.193.keytab
--principal=kudu/127.1.135.193
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.1.135.212:34139
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskRiQe_a/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790201539513534-1567-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskRiQe_a/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790201539513534-1567-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskRiQe_a/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790201539513534-1567-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260923 22:15:51.888942 18005 log.cc:824] T 00000000000000000000000000000000 P 2d2671e82fe14a6fadf679b384125ea3: Log is configured to *not* fsync() on all Append() calls
I20260923 22:15:51.891096 18005 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 2d2671e82fe14a6fadf679b384125ea3: Bootstrap replayed 1/1 log segments. Stats: ops{read=14 overwritten=0 applied=14 ignored=0} inserts{seen=8 ignored=0} mutations{seen=8 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260923 22:15:51.891367 18005 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 2d2671e82fe14a6fadf679b384125ea3: Bootstrap complete.
I20260923 22:15:51.893527 18005 raft_consensus.cc:348] T 00000000000000000000000000000000 P 2d2671e82fe14a6fadf679b384125ea3 [term 2 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2d2671e82fe14a6fadf679b384125ea3" member_type: VOTER last_known_addr { host: "127.1.135.254" port: 45507 } }
I20260923 22:15:51.893765 18005 raft_consensus.cc:729] T 00000000000000000000000000000000 P 2d2671e82fe14a6fadf679b384125ea3 [term 2 FOLLOWER]: Becoming Follower/Learner. State: Replica: 2d2671e82fe14a6fadf679b384125ea3, State: Initialized, Role: FOLLOWER
I20260923 22:15:51.893949 18005 consensus_queue.cc:260] T 00000000000000000000000000000000 P 2d2671e82fe14a6fadf679b384125ea3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 14, Last appended: 2.14, Last appended by leader: 14, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2d2671e82fe14a6fadf679b384125ea3" member_type: VOTER last_known_addr { host: "127.1.135.254" port: 45507 } }
I20260923 22:15:51.894042 18005 raft_consensus.cc:388] T 00000000000000000000000000000000 P 2d2671e82fe14a6fadf679b384125ea3 [term 2 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260923 22:15:51.894094 18005 raft_consensus.cc:482] T 00000000000000000000000000000000 P 2d2671e82fe14a6fadf679b384125ea3 [term 2 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260923 22:15:51.894141 18005 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 2d2671e82fe14a6fadf679b384125ea3 [term 2 FOLLOWER]: Advancing to term 3
I20260923 22:15:51.895068 18005 raft_consensus.cc:504] T 00000000000000000000000000000000 P 2d2671e82fe14a6fadf679b384125ea3 [term 3 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2d2671e82fe14a6fadf679b384125ea3" member_type: VOTER last_known_addr { host: "127.1.135.254" port: 45507 } }
I20260923 22:15:51.895182 18005 leader_election.cc:302] T 00000000000000000000000000000000 P 2d2671e82fe14a6fadf679b384125ea3 [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 2d2671e82fe14a6fadf679b384125ea3; no voters:
I20260923 22:15:51.895355 18005 leader_election.cc:288] T 00000000000000000000000000000000 P 2d2671e82fe14a6fadf679b384125ea3 [CANDIDATE]: Term 3 election: Requested vote from peers
I20260923 22:15:51.895396 18009 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 2d2671e82fe14a6fadf679b384125ea3 [term 3 FOLLOWER]: Leader election won for term 3
I20260923 22:15:51.895519 18009 raft_consensus.cc:686] T 00000000000000000000000000000000 P 2d2671e82fe14a6fadf679b384125ea3 [term 3 LEADER]: Becoming Leader. State: Replica: 2d2671e82fe14a6fadf679b384125ea3, State: Running, Role: LEADER
I20260923 22:15:51.895612 18005 sys_catalog.cc:564] T 00000000000000000000000000000000 P 2d2671e82fe14a6fadf679b384125ea3 [sys.catalog]: configured and running, proceeding with master startup.
I20260923 22:15:51.895601 18009 consensus_queue.cc:237] T 00000000000000000000000000000000 P 2d2671e82fe14a6fadf679b384125ea3 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 14, Committed index: 14, Last appended: 2.14, Last appended by leader: 14, Current term: 3, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2d2671e82fe14a6fadf679b384125ea3" member_type: VOTER last_known_addr { host: "127.1.135.254" port: 45507 } }
I20260923 22:15:51.895848 18010 sys_catalog.cc:455] T 00000000000000000000000000000000 P 2d2671e82fe14a6fadf679b384125ea3 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 3 leader_uuid: "2d2671e82fe14a6fadf679b384125ea3" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2d2671e82fe14a6fadf679b384125ea3" member_type: VOTER last_known_addr { host: "127.1.135.254" port: 45507 } } }
I20260923 22:15:51.895948 18010 sys_catalog.cc:458] T 00000000000000000000000000000000 P 2d2671e82fe14a6fadf679b384125ea3 [sys.catalog]: This master's current role is: LEADER
I20260923 22:15:51.896180 18011 sys_catalog.cc:455] T 00000000000000000000000000000000 P 2d2671e82fe14a6fadf679b384125ea3 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 2d2671e82fe14a6fadf679b384125ea3. Latest consensus state: current_term: 3 leader_uuid: "2d2671e82fe14a6fadf679b384125ea3" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2d2671e82fe14a6fadf679b384125ea3" member_type: VOTER last_known_addr { host: "127.1.135.254" port: 45507 } } }
I20260923 22:15:51.896263 18011 sys_catalog.cc:458] T 00000000000000000000000000000000 P 2d2671e82fe14a6fadf679b384125ea3 [sys.catalog]: This master's current role is: LEADER
I20260923 22:15:51.896381 18014 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260923 22:15:51.897187 18014 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260923 22:15:51.897284 18014 catalog_manager.cc:1259] Loaded cluster ID: 88dd55e093394f6881df1895285fec40
I20260923 22:15:51.897313 18014 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260923 22:15:51.898614 18014 catalog_manager.cc:1504] Loading token signing keys...
I20260923 22:15:51.898777 18014 catalog_manager.cc:5879] T 00000000000000000000000000000000 P 2d2671e82fe14a6fadf679b384125ea3: Loaded TSK: 0
I20260923 22:15:51.898818 18014 catalog_manager.cc:5879] T 00000000000000000000000000000000 P 2d2671e82fe14a6fadf679b384125ea3: Loaded TSK: 1
F20260923 22:15:51.899047 18014 token_signing_key.cc:66] Check failed: _s.ok() Bad status: Runtime error: unable to load data from memory: error:0D0680A8:asn1 encoding routines:ASN1_CHECK_TLEN:wrong tag:tasn_dec.c:1220 error:0D06C03A:asn1 encoding routines:ASN1_D2I_EX_PRIMITIVE:nested asn1 error:tasn_dec.c:788 error:0D08303A:asn1 encoding routines:ASN1_TEMPLATE_NOEXP_D2I:nested asn1 error:tasn_dec.c:720:Field=version, Type=RSA error:04093004:rsa routines:OLD_RSA_PRIV_DECODE:RSA lib:rsa_ameth.c:121 error:0D0680A8:asn1 encoding routines:ASN1_CHECK_TLEN:wrong tag:tasn_dec.c:1220 error:0D06C03A:asn1 encoding routines:ASN1_D2I_EX_PRIMITIVE:nested asn1 error:tasn_dec.c:788 error:0D08303A:asn1 encoding routines:ASN1_TEMPLATE_NOEXP_D2I:nested asn1 error:tasn_dec.c:720:Field=version, Type=PKCS8_PRIV_KEY_INFO
*** Check failure stack trace: ***
@ 0x7f00a38379ad google::LogMessage::Fail() at ??:0
@ 0x7f00a383a88a google::LogMessage::SendToLog() at ??:0
@ 0x7f00a38374de google::LogMessage::Flush() at ??:0
@ 0x7f00a3838a19 google::LogMessageFatal::~LogMessageFatal() at ??:0
@ 0x7f00a5fa4104 kudu::security::TokenSigningPrivateKey::TokenSigningPrivateKey() at ??:0
@ 0x7f00a5fa1d41 kudu::security::TokenSigner::ImportKeys() at ??:0
@ 0x7f00ab7951cd kudu::master::CatalogManager::LoadTskEntries() at ??:0
@ 0x7f00ab79548e kudu::master::CatalogManager::InitTokenSigner() at ??:0
@ 0x7f00ab795594 _ZNSt17_Function_handlerIFN4kudu6StatusEvEZNS0_6master14CatalogManager24PrepareForLeadershipTaskEvEUlvE3_E9_M_invokeERKSt9_Any_data at ??:0
@ 0x7f00ab75bc48 _ZZN4kudu6master14CatalogManager24PrepareForLeadershipTaskEvENKUlSt8functionIFNS_6StatusEvEERKNS_9consensus13RaftConsensusElPKcE_clES5_S9_lSB_.isra.591 at ??:0
@ 0x7f00ab765ad0 kudu::master::CatalogManager::PrepareForLeadershipTask() at ??:0
@ 0x7f00a57050d3 kudu::ThreadPool::DispatchThread() at ??:0
@ 0x7f00a56fa954 kudu::Thread::SuperviseThread() at ??:0
@ 0x7f00a655e6db start_thread at ??:0
@ 0x7f00a233561f clone at ??:0
W20260923 22:15:51.991397 18007 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260923 22:15:51.991592 18007 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260923 22:15:51.991616 18007 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260923 22:15:51.992794 18007 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260923 22:15:51.992831 18007 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260923 22:15:51.992856 18007 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.1.135.193
I20260923 22:15:51.994240 18007 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.1.135.212:34139
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskRiQe_a/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790201539513534-1567-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskRiQe_a/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790201539513534-1567-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskRiQe_a/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790201539513534-1567-0/krb5kdc/kudu_127.1.135.193.keytab
--principal=kudu/127.1.135.193
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.1.135.193:46309
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskRiQe_a/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790201539513534-1567-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.1.135.193
--webserver_port=35971
--tserver_master_addrs=127.1.135.254:45507
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.18007
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.1.135.193
--log_dir=/tmp/dist-test-taskRiQe_a/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790201539513534-1567-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 23 Sep 2026 20:53:37 UTC on ccycloud.quasar-rkviyx.root.comops.site
build id 4668083
I20260923 22:15:51.994458 18007 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260923 22:15:51.994678 18007 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260923 22:15:51.999670 18046 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260923 22:15:51.999758 18047 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260923 22:15:51.999670 18049 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260923 22:15:51.999892 18007 server_base.cc:1034] running on GCE node
I20260923 22:15:52.000195 18007 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260923 22:15:52.000450 18007 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260923 22:15:52.001616 18007 hybrid_clock.cc:648] HybridClock initialized: now 1790201752001602 us; error 32 us; skew 500 ppm
Sep 23 22:15:51 dist-test-slave-s2fj krb5kdc[16869](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790201751, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.135.193@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260923 22:15:52.004276 18007 init.cc:377] Logged in from keytab as kudu/127.1.135.193@KRBTEST.COM (short username kudu)
I20260923 22:15:52.004768 18007 webserver.cc:460] Webserver started at http://127.1.135.193:35971/ using document root <none> and password file <none>
I20260923 22:15:52.005054 18007 fs_manager.cc:359] Metadata directory not provided
I20260923 22:15:52.005128 18007 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260923 22:15:52.006857 18007 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.002s sys 0.000s
I20260923 22:15:52.007619 18056 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260923 22:15:52.007802 18007 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260923 22:15:52.007908 18007 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskRiQe_a/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790201539513534-1567-0/minicluster-data/ts-0/data,/tmp/dist-test-taskRiQe_a/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790201539513534-1567-0/minicluster-data/ts-0/wal
uuid: "de2e7140d2c742e6983e06beafcffd7a"
format_stamp: "Formatted at 2026-09-23 22:15:10 on dist-test-slave-s2fj"
server_key: "c01fe50084c85daf4a825bec04f9a216"
server_key_iv: "b82882840d6cb85ee11076a203f25d2f"
server_key_version: "encryptionkey@0"
I20260923 22:15:52.008353 18007 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskRiQe_a/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790201539513534-1567-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskRiQe_a/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790201539513534-1567-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskRiQe_a/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790201539513534-1567-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260923 22:15:52.018103 18007 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260923 22:15:52.020367 18007 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260923 22:15:52.020529 18007 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260923 22:15:52.020772 18007 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260923 22:15:52.021248 18063 ts_tablet_manager.cc:536] Loading tablet metadata (0/1 complete)
I20260923 22:15:52.022672 18007 ts_tablet_manager.cc:579] Loaded tablet metadata (1 total tablets, 1 live tablets)
I20260923 22:15:52.022735 18007 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.002s user 0.000s sys 0.000s
I20260923 22:15:52.022768 18007 ts_tablet_manager.cc:594] Registering tablets (0/1 complete)
I20260923 22:15:52.023576 18007 ts_tablet_manager.cc:610] Registered 1 tablets
I20260923 22:15:52.023623 18007 ts_tablet_manager.cc:589] Time spent register tablets: real 0.001s user 0.000s sys 0.000s
I20260923 22:15:52.023680 18063 tablet_bootstrap.cc:492] T 40e0f745b16b42e59aca8379b3f40153 P de2e7140d2c742e6983e06beafcffd7a: Bootstrap starting.
I20260923 22:15:52.030582 18007 rpc_server.cc:307] RPC server started. Bound to: 127.1.135.193:46309
I20260923 22:15:52.030654 18170 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.1.135.193:46309 every 8 connection(s)
I20260923 22:15:52.031003 18007 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskRiQe_a/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790201539513534-1567-0/minicluster-data/ts-0/data/info.pb
I20260923 22:15:52.032568 1567 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskRiQe_a/build/release/bin/kudu as pid 18007
I20260923 22:15:52.032917 1567 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskRiQe_a/build/release/bin/kudu
/tmp/dist-test-taskRiQe_a/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskRiQe_a/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790201539513534-1567-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskRiQe_a/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790201539513534-1567-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskRiQe_a/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790201539513534-1567-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskRiQe_a/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790201539513534-1567-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.1.135.194:33393
--local_ip_for_outbound_sockets=127.1.135.194
--tserver_master_addrs=127.1.135.254:45507
--webserver_port=34741
--webserver_interface=127.1.135.194
--keytab_file=/tmp/dist-test-taskRiQe_a/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790201539513534-1567-0/krb5kdc/kudu_127.1.135.194.keytab
--principal=kudu/127.1.135.194
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.1.135.212:34139
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskRiQe_a/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790201539513534-1567-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskRiQe_a/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790201539513534-1567-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskRiQe_a/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790201539513534-1567-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260923 22:15:52.033617 18063 log.cc:824] T 40e0f745b16b42e59aca8379b3f40153 P de2e7140d2c742e6983e06beafcffd7a: Log is configured to *not* fsync() on all Append() calls
I20260923 22:15:52.035861 18063 tablet_bootstrap.cc:492] T 40e0f745b16b42e59aca8379b3f40153 P de2e7140d2c742e6983e06beafcffd7a: Bootstrap replayed 1/1 log segments. Stats: ops{read=2 overwritten=0 applied=2 ignored=0} inserts{seen=1 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260923 22:15:52.036182 18063 tablet_bootstrap.cc:492] T 40e0f745b16b42e59aca8379b3f40153 P de2e7140d2c742e6983e06beafcffd7a: Bootstrap complete.
I20260923 22:15:52.037375 18063 ts_tablet_manager.cc:1397] T 40e0f745b16b42e59aca8379b3f40153 P de2e7140d2c742e6983e06beafcffd7a: Time spent bootstrapping tablet: real 0.014s user 0.006s sys 0.006s
I20260923 22:15:52.039115 18063 raft_consensus.cc:348] T 40e0f745b16b42e59aca8379b3f40153 P de2e7140d2c742e6983e06beafcffd7a [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "43f20a4dc81f48e3bdb7a2281b8505c2" member_type: VOTER last_known_addr { host: "127.1.135.195" port: 44883 } } peers { permanent_uuid: "1d61ec2a894f4c76afcd904fdb097bfc" member_type: VOTER last_known_addr { host: "127.1.135.194" port: 33393 } } peers { permanent_uuid: "de2e7140d2c742e6983e06beafcffd7a" member_type: VOTER last_known_addr { host: "127.1.135.193" port: 46309 } }
I20260923 22:15:52.039544 18063 raft_consensus.cc:729] T 40e0f745b16b42e59aca8379b3f40153 P de2e7140d2c742e6983e06beafcffd7a [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: de2e7140d2c742e6983e06beafcffd7a, State: Initialized, Role: FOLLOWER
I20260923 22:15:52.039721 18063 consensus_queue.cc:260] T 40e0f745b16b42e59aca8379b3f40153 P de2e7140d2c742e6983e06beafcffd7a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 2, Last appended: 1.2, Last appended by leader: 2, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "43f20a4dc81f48e3bdb7a2281b8505c2" member_type: VOTER last_known_addr { host: "127.1.135.195" port: 44883 } } peers { permanent_uuid: "1d61ec2a894f4c76afcd904fdb097bfc" member_type: VOTER last_known_addr { host: "127.1.135.194" port: 33393 } } peers { permanent_uuid: "de2e7140d2c742e6983e06beafcffd7a" member_type: VOTER last_known_addr { host: "127.1.135.193" port: 46309 } }
I20260923 22:15:52.040024 18063 ts_tablet_manager.cc:1428] T 40e0f745b16b42e59aca8379b3f40153 P de2e7140d2c742e6983e06beafcffd7a: Time spent starting tablet: real 0.003s user 0.001s sys 0.003s
W20260923 22:15:52.094245 18173 negotiation.cc:337] Failed RPC negotiation. Trace:
0923 22:15:52.031712 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.135.254:45507 (local address 127.1.135.193:46563)
0923 22:15:52.031931 (+ 219us) negotiation.cc:107] Waiting for socket to connect
0923 22:15:52.031978 (+ 47us) client_negotiation.cc:174] Beginning negotiation
0923 22:15:52.032598 (+ 620us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0923 22:15:52.094040 (+ 61442us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.135.254:45507: BlockingRecv error: recv error from unknown peer: Transport endpoint is not connected (error 107)
Metrics: {"client-negotiator.queue_time_us":166,"thread_start_us":114,"threads_started":1}
W20260923 22:15:52.094566 18171 heartbeater.cc:644] Failed to heartbeat to 127.1.135.254:45507 (0 consecutive failures): Network error: Failed to ping master at 127.1.135.254:45507: Client connection negotiation failed: client connection to 127.1.135.254:45507: BlockingRecv error: recv error from unknown peer: Transport endpoint is not connected (error 107)
I20260923 22:15:52.094919 18173 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 22:15:52.094791 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.135.254:45507 (local address 127.1.135.193:50573)
0923 22:15:52.094844 (+ 53us) negotiation.cc:107] Waiting for socket to connect
0923 22:15:52.094893 (+ 49us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.135.254:45507: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":31}
I20260923 22:15:52.095296 18173 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 22:15:52.095193 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.135.254:45507 (local address 127.1.135.193:60021)
0923 22:15:52.095236 (+ 43us) negotiation.cc:107] Waiting for socket to connect
0923 22:15:52.095260 (+ 24us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.135.254:45507: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":29}
W20260923 22:15:52.095445 18171 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
W20260923 22:15:52.114926 18175 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260923 22:15:52.115132 18175 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260923 22:15:52.115170 18175 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260923 22:15:52.116423 18175 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260923 22:15:52.116480 18175 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260923 22:15:52.116526 18175 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.1.135.194
I20260923 22:15:52.117949 18175 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.1.135.212:34139
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskRiQe_a/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790201539513534-1567-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskRiQe_a/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790201539513534-1567-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskRiQe_a/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790201539513534-1567-0/krb5kdc/kudu_127.1.135.194.keytab
--principal=kudu/127.1.135.194
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.1.135.194:33393
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskRiQe_a/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790201539513534-1567-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.1.135.194
--webserver_port=34741
--tserver_master_addrs=127.1.135.254:45507
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.18175
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.1.135.194
--log_dir=/tmp/dist-test-taskRiQe_a/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790201539513534-1567-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 23 Sep 2026 20:53:37 UTC on ccycloud.quasar-rkviyx.root.comops.site
build id 4668083
I20260923 22:15:52.118196 18175 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260923 22:15:52.118420 18175 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260923 22:15:52.123199 18184 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260923 22:15:52.123195 18182 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260923 22:15:52.123234 18181 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260923 22:15:52.123407 18175 server_base.cc:1034] running on GCE node
I20260923 22:15:52.123595 18175 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260923 22:15:52.123775 18175 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260923 22:15:52.124905 18175 hybrid_clock.cc:648] HybridClock initialized: now 1790201752124893 us; error 27 us; skew 500 ppm
Sep 23 22:15:52 dist-test-slave-s2fj krb5kdc[16869](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790201752, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.135.194@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260923 22:15:52.127300 18175 init.cc:377] Logged in from keytab as kudu/127.1.135.194@KRBTEST.COM (short username kudu)
I20260923 22:15:52.127749 18175 webserver.cc:460] Webserver started at http://127.1.135.194:34741/ using document root <none> and password file <none>
I20260923 22:15:52.127959 18175 fs_manager.cc:359] Metadata directory not provided
I20260923 22:15:52.128023 18175 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260923 22:15:52.129295 18175 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.001s sys 0.000s
I20260923 22:15:52.129839 18191 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260923 22:15:52.129994 18175 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260923 22:15:52.130092 18175 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskRiQe_a/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790201539513534-1567-0/minicluster-data/ts-1/data,/tmp/dist-test-taskRiQe_a/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790201539513534-1567-0/minicluster-data/ts-1/wal
uuid: "1d61ec2a894f4c76afcd904fdb097bfc"
format_stamp: "Formatted at 2026-09-23 22:15:10 on dist-test-slave-s2fj"
server_key: "61fd161f646230d3a8ca49c7ce067153"
server_key_iv: "55c6a41dda98e464a8572f3994d180a6"
server_key_version: "encryptionkey@0"
I20260923 22:15:52.130405 18175 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskRiQe_a/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790201539513534-1567-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskRiQe_a/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790201539513534-1567-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskRiQe_a/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790201539513534-1567-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260923 22:15:52.147825 18175 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260923 22:15:52.149616 18175 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260923 22:15:52.149767 18175 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260923 22:15:52.149983 18175 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260923 22:15:52.150262 18175 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260923 22:15:52.150310 18175 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260923 22:15:52.150349 18175 ts_tablet_manager.cc:610] Registered 0 tablets
I20260923 22:15:52.150377 18175 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260923 22:15:52.156024 18175 rpc_server.cc:307] RPC server started. Bound to: 127.1.135.194:33393
I20260923 22:15:52.156100 18304 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.1.135.194:33393 every 8 connection(s)
I20260923 22:15:52.156409 18175 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskRiQe_a/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790201539513534-1567-0/minicluster-data/ts-1/data/info.pb
I20260923 22:15:52.157333 18307 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 22:15:52.156964 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.135.254:45507 (local address 127.1.135.194:47325)
0923 22:15:52.157132 (+ 168us) negotiation.cc:107] Waiting for socket to connect
0923 22:15:52.157199 (+ 67us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.135.254:45507: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":133,"thread_start_us":98,"threads_started":1}
W20260923 22:15:52.157723 18305 heartbeater.cc:644] Failed to heartbeat to 127.1.135.254:45507 (0 consecutive failures): Network error: Failed to ping master at 127.1.135.254:45507: Client connection negotiation failed: client connection to 127.1.135.254:45507: connect: Connection refused (error 111)
I20260923 22:15:52.157984 18307 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 22:15:52.157893 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.135.254:45507 (local address 127.1.135.194:51125)
0923 22:15:52.157942 (+ 49us) negotiation.cc:107] Waiting for socket to connect
0923 22:15:52.157967 (+ 25us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.135.254:45507: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":33}
I20260923 22:15:52.158265 18307 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 22:15:52.158197 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.135.254:45507 (local address 127.1.135.194:59551)
0923 22:15:52.158234 (+ 37us) negotiation.cc:107] Waiting for socket to connect
0923 22:15:52.158253 (+ 19us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.135.254:45507: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":27}
W20260923 22:15:52.158377 18305 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
I20260923 22:15:52.158860 1567 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskRiQe_a/build/release/bin/kudu as pid 18175
I20260923 22:15:52.159142 1567 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskRiQe_a/build/release/bin/kudu
/tmp/dist-test-taskRiQe_a/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskRiQe_a/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790201539513534-1567-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskRiQe_a/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790201539513534-1567-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskRiQe_a/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790201539513534-1567-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskRiQe_a/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790201539513534-1567-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.1.135.195:44883
--local_ip_for_outbound_sockets=127.1.135.195
--tserver_master_addrs=127.1.135.254:45507
--webserver_port=37405
--webserver_interface=127.1.135.195
--keytab_file=/tmp/dist-test-taskRiQe_a/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790201539513534-1567-0/krb5kdc/kudu_127.1.135.195.keytab
--principal=kudu/127.1.135.195
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.1.135.212:34139
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskRiQe_a/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790201539513534-1567-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskRiQe_a/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790201539513534-1567-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskRiQe_a/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790201539513534-1567-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260923 22:15:52.237519 18308 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260923 22:15:52.237712 18308 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260923 22:15:52.237735 18308 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260923 22:15:52.238862 18308 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260923 22:15:52.238903 18308 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260923 22:15:52.238953 18308 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.1.135.195
I20260923 22:15:52.240347 18308 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.1.135.212:34139
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskRiQe_a/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790201539513534-1567-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskRiQe_a/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790201539513534-1567-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskRiQe_a/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790201539513534-1567-0/krb5kdc/kudu_127.1.135.195.keytab
--principal=kudu/127.1.135.195
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.1.135.195:44883
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskRiQe_a/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790201539513534-1567-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.1.135.195
--webserver_port=37405
--tserver_master_addrs=127.1.135.254:45507
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.18308
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.1.135.195
--log_dir=/tmp/dist-test-taskRiQe_a/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790201539513534-1567-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 23 Sep 2026 20:53:37 UTC on ccycloud.quasar-rkviyx.root.comops.site
build id 4668083
I20260923 22:15:52.240594 18308 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260923 22:15:52.240806 18308 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260923 22:15:52.245514 18313 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260923 22:15:52.245568 18316 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260923 22:15:52.245559 18314 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260923 22:15:52.245729 18308 server_base.cc:1034] running on GCE node
I20260923 22:15:52.245976 18308 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260923 22:15:52.246238 18308 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260923 22:15:52.247378 18308 hybrid_clock.cc:648] HybridClock initialized: now 1790201752247363 us; error 39 us; skew 500 ppm
Sep 23 22:15:52 dist-test-slave-s2fj krb5kdc[16869](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790201752, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.135.195@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260923 22:15:52.249679 18308 init.cc:377] Logged in from keytab as kudu/127.1.135.195@KRBTEST.COM (short username kudu)
I20260923 22:15:52.250082 18308 webserver.cc:460] Webserver started at http://127.1.135.195:37405/ using document root <none> and password file <none>
I20260923 22:15:52.250298 18308 fs_manager.cc:359] Metadata directory not provided
I20260923 22:15:52.250362 18308 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260923 22:15:52.251583 18308 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.001s sys 0.000s
I20260923 22:15:52.252173 18323 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260923 22:15:52.252328 18308 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.000s
I20260923 22:15:52.252424 18308 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskRiQe_a/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790201539513534-1567-0/minicluster-data/ts-2/data,/tmp/dist-test-taskRiQe_a/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790201539513534-1567-0/minicluster-data/ts-2/wal
uuid: "43f20a4dc81f48e3bdb7a2281b8505c2"
format_stamp: "Formatted at 2026-09-23 22:15:10 on dist-test-slave-s2fj"
server_key: "c319eb09e0c64ba992b15b7f068e6c56"
server_key_iv: "df4df41ccb3bbb369e9bb5805ddba31c"
server_key_version: "encryptionkey@0"
I20260923 22:15:52.252723 18308 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskRiQe_a/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790201539513534-1567-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskRiQe_a/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790201539513534-1567-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskRiQe_a/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790201539513534-1567-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260923 22:15:52.272245 18308 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260923 22:15:52.273970 18308 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260923 22:15:52.274118 18308 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260923 22:15:52.274326 18308 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260923 22:15:52.274696 18308 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260923 22:15:52.274745 18308 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260923 22:15:52.274785 18308 ts_tablet_manager.cc:610] Registered 0 tablets
I20260923 22:15:52.274814 18308 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260923 22:15:52.280279 18308 rpc_server.cc:307] RPC server started. Bound to: 127.1.135.195:44883
I20260923 22:15:52.280345 18436 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.1.135.195:44883 every 8 connection(s)
I20260923 22:15:52.280664 18308 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskRiQe_a/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790201539513534-1567-0/minicluster-data/ts-2/data/info.pb
I20260923 22:15:52.281558 18439 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 22:15:52.281252 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.135.254:45507 (local address 127.1.135.195:41987)
0923 22:15:52.281371 (+ 119us) negotiation.cc:107] Waiting for socket to connect
0923 22:15:52.281430 (+ 59us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.135.254:45507: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":87,"thread_start_us":66,"threads_started":1}
W20260923 22:15:52.281924 18437 heartbeater.cc:644] Failed to heartbeat to 127.1.135.254:45507 (0 consecutive failures): Network error: Failed to ping master at 127.1.135.254:45507: Client connection negotiation failed: client connection to 127.1.135.254:45507: connect: Connection refused (error 111)
I20260923 22:15:52.282181 18439 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 22:15:52.282115 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.135.254:45507 (local address 127.1.135.195:45149)
0923 22:15:52.282143 (+ 28us) negotiation.cc:107] Waiting for socket to connect
0923 22:15:52.282166 (+ 23us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.135.254:45507: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":16}
I20260923 22:15:52.282418 18439 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 22:15:52.282364 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.135.254:45507 (local address 127.1.135.195:44805)
0923 22:15:52.282383 (+ 19us) negotiation.cc:107] Waiting for socket to connect
0923 22:15:52.282407 (+ 24us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.135.254:45507: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":11}
W20260923 22:15:52.282552 18437 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
I20260923 22:15:52.283591 1567 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskRiQe_a/build/release/bin/kudu as pid 18308
I20260923 22:15:53.095996 18441 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 22:15:53.095774 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.135.254:45507 (local address 127.1.135.193:60099)
0923 22:15:53.095884 (+ 110us) negotiation.cc:107] Waiting for socket to connect
0923 22:15:53.095933 (+ 49us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.135.254:45507: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":94,"thread_start_us":58,"threads_started":1}
I20260923 22:15:53.158869 18442 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 22:15:53.158697 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.135.254:45507 (local address 127.1.135.194:51075)
0923 22:15:53.158793 (+ 96us) negotiation.cc:107] Waiting for socket to connect
0923 22:15:53.158822 (+ 29us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.135.254:45507: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":82,"thread_start_us":50,"threads_started":1}
I20260923 22:15:53.283021 18443 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 22:15:53.282833 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.135.254:45507 (local address 127.1.135.195:54869)
0923 22:15:53.282933 (+ 100us) negotiation.cc:107] Waiting for socket to connect
0923 22:15:53.282965 (+ 32us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.135.254:45507: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":86,"thread_start_us":52,"threads_started":1}
I20260923 22:15:53.391908 18444 raft_consensus.cc:482] T 40e0f745b16b42e59aca8379b3f40153 P de2e7140d2c742e6983e06beafcffd7a [term 1 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260923 22:15:53.392033 18444 raft_consensus.cc:504] T 40e0f745b16b42e59aca8379b3f40153 P de2e7140d2c742e6983e06beafcffd7a [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "43f20a4dc81f48e3bdb7a2281b8505c2" member_type: VOTER last_known_addr { host: "127.1.135.195" port: 44883 } } peers { permanent_uuid: "1d61ec2a894f4c76afcd904fdb097bfc" member_type: VOTER last_known_addr { host: "127.1.135.194" port: 33393 } } peers { permanent_uuid: "de2e7140d2c742e6983e06beafcffd7a" member_type: VOTER last_known_addr { host: "127.1.135.193" port: 46309 } }
I20260923 22:15:53.392581 18444 leader_election.cc:288] T 40e0f745b16b42e59aca8379b3f40153 P de2e7140d2c742e6983e06beafcffd7a [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 43f20a4dc81f48e3bdb7a2281b8505c2 (127.1.135.195:44883), 1d61ec2a894f4c76afcd904fdb097bfc (127.1.135.194:33393)
Sep 23 22:15:53 dist-test-slave-s2fj krb5kdc[16869](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790201751, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.135.193@KRBTEST.COM for kudu/127.1.135.194@KRBTEST.COM
Sep 23 22:15:53 dist-test-slave-s2fj krb5kdc[16869](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790201751, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.135.193@KRBTEST.COM for kudu/127.1.135.195@KRBTEST.COM
I20260923 22:15:53.407779 18441 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 22:15:53.392721 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.135.195:44883 (local address 127.1.135.193:57445)
0923 22:15:53.392774 (+ 53us) negotiation.cc:107] Waiting for socket to connect
0923 22:15:53.392786 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0923 22:15:53.392841 (+ 55us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0923 22:15:53.393852 (+ 1011us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0923 22:15:53.393855 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0923 22:15:53.394153 (+ 298us) client_negotiation.cc:189] Negotiated authn=SASL
0923 22:15:53.394347 (+ 194us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 22:15:53.394352 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 22:15:53.395425 (+ 1073us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 22:15:53.395426 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 22:15:53.396517 (+ 1091us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 22:15:53.396520 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 22:15:53.397277 (+ 757us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 22:15:53.397278 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 22:15:53.397321 (+ 43us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 22:15:53.402131 (+ 4810us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0923 22:15:53.402139 (+ 8us) client_negotiation.cc:656] calling sasl_client_start()
0923 22:15:53.404340 (+ 2201us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0923 22:15:53.406323 (+ 1983us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0923 22:15:53.406324 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0923 22:15:53.406324 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0923 22:15:53.406576 (+ 252us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0923 22:15:53.406837 (+ 261us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0923 22:15:53.406838 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0923 22:15:53.406838 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0923 22:15:53.406916 (+ 78us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0923 22:15:53.407328 (+ 412us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0923 22:15:53.407329 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0923 22:15:53.407381 (+ 52us) client_negotiation.cc:769] Sending connection context
0923 22:15:53.407485 (+ 104us) client_negotiation.cc:240] Negotiation successful
0923 22:15:53.407641 (+ 156us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":35,"mutex_wait_us":2}
I20260923 22:15:53.407836 18446 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 22:15:53.392834 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.135.193:57445 (local address 127.1.135.195:44883)
0923 22:15:53.392945 (+ 111us) server_negotiation.cc:206] Beginning negotiation
0923 22:15:53.392958 (+ 13us) server_negotiation.cc:399] Waiting for connection header
0923 22:15:53.393015 (+ 57us) server_negotiation.cc:407] Connection header received
0923 22:15:53.393598 (+ 583us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 22:15:53.393601 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 22:15:53.393659 (+ 58us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 22:15:53.393811 (+ 152us) server_negotiation.cc:226] Negotiated authn=SASL
0923 22:15:53.394486 (+ 675us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 22:15:53.395312 (+ 826us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 22:15:53.396669 (+ 1357us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 22:15:53.397183 (+ 514us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 22:15:53.397296 (+ 113us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 22:15:53.404472 (+ 7176us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0923 22:15:53.404476 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0923 22:15:53.404476 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0923 22:15:53.404486 (+ 10us) server_negotiation.cc:922] calling sasl_server_start()
0923 22:15:53.406238 (+ 1752us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0923 22:15:53.406657 (+ 419us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0923 22:15:53.406660 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0923 22:15:53.406660 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0923 22:15:53.406761 (+ 101us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0923 22:15:53.406983 (+ 222us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0923 22:15:53.406984 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0923 22:15:53.406984 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0923 22:15:53.407253 (+ 269us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0923 22:15:53.407339 (+ 86us) server_negotiation.cc:1035] Waiting for connection context
0923 22:15:53.407567 (+ 228us) server_negotiation.cc:299] Negotiation successful
0923 22:15:53.407715 (+ 148us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":79,"thread_start_us":55,"threads_started":1}
I20260923 22:15:53.407928 18447 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 22:15:53.392859 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.135.193:42521 (local address 127.1.135.194:33393)
0923 22:15:53.393009 (+ 150us) server_negotiation.cc:206] Beginning negotiation
0923 22:15:53.393016 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0923 22:15:53.393054 (+ 38us) server_negotiation.cc:407] Connection header received
0923 22:15:53.393597 (+ 543us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 22:15:53.393600 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 22:15:53.393659 (+ 59us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 22:15:53.393811 (+ 152us) server_negotiation.cc:226] Negotiated authn=SASL
0923 22:15:53.394472 (+ 661us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 22:15:53.395584 (+ 1112us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 22:15:53.396519 (+ 935us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 22:15:53.396969 (+ 450us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 22:15:53.397049 (+ 80us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 22:15:53.404002 (+ 6953us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0923 22:15:53.404006 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0923 22:15:53.404007 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0923 22:15:53.404016 (+ 9us) server_negotiation.cc:922] calling sasl_server_start()
0923 22:15:53.405894 (+ 1878us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0923 22:15:53.406398 (+ 504us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0923 22:15:53.406401 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0923 22:15:53.406402 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0923 22:15:53.406517 (+ 115us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0923 22:15:53.406838 (+ 321us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0923 22:15:53.406838 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0923 22:15:53.406839 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0923 22:15:53.407132 (+ 293us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0923 22:15:53.407192 (+ 60us) server_negotiation.cc:1035] Waiting for connection context
0923 22:15:53.407665 (+ 473us) server_negotiation.cc:299] Negotiation successful
0923 22:15:53.407795 (+ 130us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":93,"thread_start_us":65,"threads_started":1}
I20260923 22:15:53.407833 18445 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 22:15:53.392748 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.135.194:33393 (local address 127.1.135.193:42521)
0923 22:15:53.392864 (+ 116us) negotiation.cc:107] Waiting for socket to connect
0923 22:15:53.392874 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0923 22:15:53.392930 (+ 56us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0923 22:15:53.393935 (+ 1005us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0923 22:15:53.393936 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0923 22:15:53.394182 (+ 246us) client_negotiation.cc:189] Negotiated authn=SASL
0923 22:15:53.394347 (+ 165us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 22:15:53.394352 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 22:15:53.395686 (+ 1334us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 22:15:53.395687 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 22:15:53.396340 (+ 653us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 22:15:53.396343 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 22:15:53.397079 (+ 736us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 22:15:53.397081 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 22:15:53.397209 (+ 128us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 22:15:53.402157 (+ 4948us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0923 22:15:53.402162 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0923 22:15:53.403889 (+ 1727us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0923 22:15:53.405984 (+ 2095us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0923 22:15:53.405992 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0923 22:15:53.405994 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0923 22:15:53.406314 (+ 320us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0923 22:15:53.406589 (+ 275us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0923 22:15:53.406590 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0923 22:15:53.406590 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0923 22:15:53.406748 (+ 158us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0923 22:15:53.407204 (+ 456us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0923 22:15:53.407211 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0923 22:15:53.407368 (+ 157us) client_negotiation.cc:769] Sending connection context
0923 22:15:53.407506 (+ 138us) client_negotiation.cc:240] Negotiation successful
0923 22:15:53.407699 (+ 193us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":103,"mutex_wait_us":83,"thread_start_us":52,"threads_started":1}
W20260923 22:15:53.408931 18060 leader_election.cc:341] T 40e0f745b16b42e59aca8379b3f40153 P de2e7140d2c742e6983e06beafcffd7a [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 43f20a4dc81f48e3bdb7a2281b8505c2 (127.1.135.195:44883): Not found: Tablet not found: 40e0f745b16b42e59aca8379b3f40153
W20260923 22:15:53.409027 18059 leader_election.cc:341] T 40e0f745b16b42e59aca8379b3f40153 P de2e7140d2c742e6983e06beafcffd7a [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 1d61ec2a894f4c76afcd904fdb097bfc (127.1.135.194:33393): Not found: Tablet not found: 40e0f745b16b42e59aca8379b3f40153
I20260923 22:15:53.409071 18059 leader_election.cc:302] T 40e0f745b16b42e59aca8379b3f40153 P de2e7140d2c742e6983e06beafcffd7a [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: de2e7140d2c742e6983e06beafcffd7a; no voters: 1d61ec2a894f4c76afcd904fdb097bfc, 43f20a4dc81f48e3bdb7a2281b8505c2
I20260923 22:15:53.409180 18444 raft_consensus.cc:2726] T 40e0f745b16b42e59aca8379b3f40153 P de2e7140d2c742e6983e06beafcffd7a [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260923 22:15:54.096664 18448 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 22:15:54.096467 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.135.254:45507 (local address 127.1.135.193:56861)
0923 22:15:54.096587 (+ 120us) negotiation.cc:107] Waiting for socket to connect
0923 22:15:54.096621 (+ 34us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.135.254:45507: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":104,"thread_start_us":64,"threads_started":1}
I20260923 22:15:54.159420 18449 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 22:15:54.159222 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.135.254:45507 (local address 127.1.135.194:54809)
0923 22:15:54.159349 (+ 127us) negotiation.cc:107] Waiting for socket to connect
0923 22:15:54.159384 (+ 35us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.135.254:45507: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":111,"thread_start_us":55,"threads_started":1}
I20260923 22:15:54.283619 18450 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 22:15:54.283411 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.135.254:45507 (local address 127.1.135.195:40471)
0923 22:15:54.283536 (+ 125us) negotiation.cc:107] Waiting for socket to connect
0923 22:15:54.283585 (+ 49us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.135.254:45507: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":110,"thread_start_us":81,"threads_started":1}
I20260923 22:15:55.097299 18451 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 22:15:55.097086 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.135.254:45507 (local address 127.1.135.193:45659)
0923 22:15:55.097220 (+ 134us) negotiation.cc:107] Waiting for socket to connect
0923 22:15:55.097252 (+ 32us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.135.254:45507: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":119,"thread_start_us":78,"threads_started":1}
I20260923 22:15:55.160035 18452 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 22:15:55.159839 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.135.254:45507 (local address 127.1.135.194:45561)
0923 22:15:55.159964 (+ 125us) negotiation.cc:107] Waiting for socket to connect
0923 22:15:55.160002 (+ 38us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.135.254:45507: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":111,"thread_start_us":72,"threads_started":1}
I20260923 22:15:55.186012 18453 raft_consensus.cc:482] T 40e0f745b16b42e59aca8379b3f40153 P de2e7140d2c742e6983e06beafcffd7a [term 1 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260923 22:15:55.186097 18453 raft_consensus.cc:504] T 40e0f745b16b42e59aca8379b3f40153 P de2e7140d2c742e6983e06beafcffd7a [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "43f20a4dc81f48e3bdb7a2281b8505c2" member_type: VOTER last_known_addr { host: "127.1.135.195" port: 44883 } } peers { permanent_uuid: "1d61ec2a894f4c76afcd904fdb097bfc" member_type: VOTER last_known_addr { host: "127.1.135.194" port: 33393 } } peers { permanent_uuid: "de2e7140d2c742e6983e06beafcffd7a" member_type: VOTER last_known_addr { host: "127.1.135.193" port: 46309 } }
I20260923 22:15:55.186250 18453 leader_election.cc:288] T 40e0f745b16b42e59aca8379b3f40153 P de2e7140d2c742e6983e06beafcffd7a [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 43f20a4dc81f48e3bdb7a2281b8505c2 (127.1.135.195:44883), 1d61ec2a894f4c76afcd904fdb097bfc (127.1.135.194:33393)
W20260923 22:15:55.186609 18059 leader_election.cc:341] T 40e0f745b16b42e59aca8379b3f40153 P de2e7140d2c742e6983e06beafcffd7a [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 1d61ec2a894f4c76afcd904fdb097bfc (127.1.135.194:33393): Not found: Tablet not found: 40e0f745b16b42e59aca8379b3f40153
W20260923 22:15:55.186707 18060 leader_election.cc:341] T 40e0f745b16b42e59aca8379b3f40153 P de2e7140d2c742e6983e06beafcffd7a [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 43f20a4dc81f48e3bdb7a2281b8505c2 (127.1.135.195:44883): Not found: Tablet not found: 40e0f745b16b42e59aca8379b3f40153
I20260923 22:15:55.186790 18060 leader_election.cc:302] T 40e0f745b16b42e59aca8379b3f40153 P de2e7140d2c742e6983e06beafcffd7a [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: de2e7140d2c742e6983e06beafcffd7a; no voters: 1d61ec2a894f4c76afcd904fdb097bfc, 43f20a4dc81f48e3bdb7a2281b8505c2
I20260923 22:15:55.186877 18453 raft_consensus.cc:2726] T 40e0f745b16b42e59aca8379b3f40153 P de2e7140d2c742e6983e06beafcffd7a [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260923 22:15:55.284193 18454 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 22:15:55.283975 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.135.254:45507 (local address 127.1.135.195:48313)
0923 22:15:55.284115 (+ 140us) negotiation.cc:107] Waiting for socket to connect
0923 22:15:55.284154 (+ 39us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.135.254:45507: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":122,"thread_start_us":58,"threads_started":1}
I20260923 22:15:56.098214 18455 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 22:15:56.097908 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.135.254:45507 (local address 127.1.135.193:45107)
0923 22:15:56.098069 (+ 161us) negotiation.cc:107] Waiting for socket to connect
0923 22:15:56.098149 (+ 80us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.135.254:45507: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":135,"thread_start_us":103,"threads_started":1}
I20260923 22:15:56.160687 18456 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 22:15:56.160490 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.135.254:45507 (local address 127.1.135.194:48661)
0923 22:15:56.160607 (+ 117us) negotiation.cc:107] Waiting for socket to connect
0923 22:15:56.160645 (+ 38us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.135.254:45507: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":100,"thread_start_us":58,"threads_started":1}
I20260923 22:15:56.284811 18457 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 22:15:56.284639 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.135.254:45507 (local address 127.1.135.195:40315)
0923 22:15:56.284742 (+ 103us) negotiation.cc:107] Waiting for socket to connect
0923 22:15:56.284775 (+ 33us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.135.254:45507: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":88,"thread_start_us":53,"threads_started":1}
I20260923 22:15:57.098881 18458 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 22:15:57.098690 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.135.254:45507 (local address 127.1.135.193:44453)
0923 22:15:57.098804 (+ 114us) negotiation.cc:107] Waiting for socket to connect
0923 22:15:57.098836 (+ 32us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.135.254:45507: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":100,"thread_start_us":62,"threads_started":1}
I20260923 22:15:57.161315 18459 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 22:15:57.161119 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.135.254:45507 (local address 127.1.135.194:60505)
0923 22:15:57.161242 (+ 123us) negotiation.cc:107] Waiting for socket to connect
0923 22:15:57.161278 (+ 36us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.135.254:45507: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":107,"thread_start_us":68,"threads_started":1}
I20260923 22:15:57.285426 18460 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 22:15:57.285226 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.135.254:45507 (local address 127.1.135.195:43609)
0923 22:15:57.285356 (+ 130us) negotiation.cc:107] Waiting for socket to connect
0923 22:15:57.285390 (+ 34us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.135.254:45507: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":114,"thread_start_us":68,"threads_started":1}
I20260923 22:15:58.099493 18461 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 22:15:58.099299 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.135.254:45507 (local address 127.1.135.193:56265)
0923 22:15:58.099411 (+ 112us) negotiation.cc:107] Waiting for socket to connect
0923 22:15:58.099443 (+ 32us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.135.254:45507: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":98,"thread_start_us":61,"threads_started":1}
I20260923 22:15:58.162027 18462 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 22:15:58.161731 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.135.254:45507 (local address 127.1.135.194:51345)
0923 22:15:58.161958 (+ 227us) negotiation.cc:107] Waiting for socket to connect
0923 22:15:58.161992 (+ 34us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.135.254:45507: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":198,"thread_start_us":55,"threads_started":1}
I20260923 22:15:58.187539 18463 raft_consensus.cc:482] T 40e0f745b16b42e59aca8379b3f40153 P de2e7140d2c742e6983e06beafcffd7a [term 1 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260923 22:15:58.187634 18463 raft_consensus.cc:504] T 40e0f745b16b42e59aca8379b3f40153 P de2e7140d2c742e6983e06beafcffd7a [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "43f20a4dc81f48e3bdb7a2281b8505c2" member_type: VOTER last_known_addr { host: "127.1.135.195" port: 44883 } } peers { permanent_uuid: "1d61ec2a894f4c76afcd904fdb097bfc" member_type: VOTER last_known_addr { host: "127.1.135.194" port: 33393 } } peers { permanent_uuid: "de2e7140d2c742e6983e06beafcffd7a" member_type: VOTER last_known_addr { host: "127.1.135.193" port: 46309 } }
I20260923 22:15:58.187845 18463 leader_election.cc:288] T 40e0f745b16b42e59aca8379b3f40153 P de2e7140d2c742e6983e06beafcffd7a [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 43f20a4dc81f48e3bdb7a2281b8505c2 (127.1.135.195:44883), 1d61ec2a894f4c76afcd904fdb097bfc (127.1.135.194:33393)
W20260923 22:15:58.188202 18060 leader_election.cc:341] T 40e0f745b16b42e59aca8379b3f40153 P de2e7140d2c742e6983e06beafcffd7a [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 43f20a4dc81f48e3bdb7a2281b8505c2 (127.1.135.195:44883): Not found: Tablet not found: 40e0f745b16b42e59aca8379b3f40153
W20260923 22:15:58.188305 18059 leader_election.cc:341] T 40e0f745b16b42e59aca8379b3f40153 P de2e7140d2c742e6983e06beafcffd7a [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 1d61ec2a894f4c76afcd904fdb097bfc (127.1.135.194:33393): Not found: Tablet not found: 40e0f745b16b42e59aca8379b3f40153
I20260923 22:15:58.188382 18059 leader_election.cc:302] T 40e0f745b16b42e59aca8379b3f40153 P de2e7140d2c742e6983e06beafcffd7a [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: de2e7140d2c742e6983e06beafcffd7a; no voters: 1d61ec2a894f4c76afcd904fdb097bfc, 43f20a4dc81f48e3bdb7a2281b8505c2
I20260923 22:15:58.188453 18463 raft_consensus.cc:2726] T 40e0f745b16b42e59aca8379b3f40153 P de2e7140d2c742e6983e06beafcffd7a [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260923 22:15:58.286034 18464 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 22:15:58.285859 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.135.254:45507 (local address 127.1.135.195:41483)
0923 22:15:58.285967 (+ 108us) negotiation.cc:107] Waiting for socket to connect
0923 22:15:58.285999 (+ 32us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.135.254:45507: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":96,"thread_start_us":58,"threads_started":1}
I20260923 22:15:59.100172 18465 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 22:15:59.099891 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.135.254:45507 (local address 127.1.135.193:38267)
0923 22:15:59.100026 (+ 135us) negotiation.cc:107] Waiting for socket to connect
0923 22:15:59.100103 (+ 77us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.135.254:45507: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":119,"thread_start_us":74,"threads_started":1}
I20260923 22:15:59.162684 18466 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 22:15:59.162513 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.135.254:45507 (local address 127.1.135.194:45495)
0923 22:15:59.162621 (+ 108us) negotiation.cc:107] Waiting for socket to connect
0923 22:15:59.162652 (+ 31us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.135.254:45507: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":94,"thread_start_us":56,"threads_started":1}
I20260923 22:15:59.286607 18467 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 22:15:59.286425 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.135.254:45507 (local address 127.1.135.195:44565)
0923 22:15:59.286541 (+ 116us) negotiation.cc:107] Waiting for socket to connect
0923 22:15:59.286572 (+ 31us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.135.254:45507: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":102,"thread_start_us":62,"threads_started":1}
I20260923 22:16:00.100847 18468 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 22:16:00.100626 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.135.254:45507 (local address 127.1.135.193:36315)
0923 22:16:00.100765 (+ 139us) negotiation.cc:107] Waiting for socket to connect
0923 22:16:00.100801 (+ 36us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.135.254:45507: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":120,"thread_start_us":60,"threads_started":1}
I20260923 22:16:00.163232 18469 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 22:16:00.163062 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.135.254:45507 (local address 127.1.135.194:58977)
0923 22:16:00.163170 (+ 108us) negotiation.cc:107] Waiting for socket to connect
0923 22:16:00.163200 (+ 30us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.135.254:45507: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":95,"thread_start_us":57,"threads_started":1}
I20260923 22:16:00.287223 18470 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 22:16:00.287018 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.135.254:45507 (local address 127.1.135.195:37017)
0923 22:16:00.287153 (+ 135us) negotiation.cc:107] Waiting for socket to connect
0923 22:16:00.287185 (+ 32us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.135.254:45507: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":122,"thread_start_us":74,"threads_started":1}
I20260923 22:16:01.101537 18471 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 22:16:01.101314 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.135.254:45507 (local address 127.1.135.193:38909)
0923 22:16:01.101448 (+ 134us) negotiation.cc:107] Waiting for socket to connect
0923 22:16:01.101475 (+ 27us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.135.254:45507: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":119,"thread_start_us":63,"threads_started":1}
I20260923 22:16:01.163833 18472 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 22:16:01.163635 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.135.254:45507 (local address 127.1.135.194:50281)
0923 22:16:01.163764 (+ 129us) negotiation.cc:107] Waiting for socket to connect
0923 22:16:01.163799 (+ 35us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.135.254:45507: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":113,"thread_start_us":88,"threads_started":1}
I20260923 22:16:01.288028 18473 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 22:16:01.287826 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.135.254:45507 (local address 127.1.135.195:33207)
0923 22:16:01.287955 (+ 129us) negotiation.cc:107] Waiting for socket to connect
0923 22:16:01.287992 (+ 37us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.135.254:45507: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":114,"thread_start_us":59,"threads_started":1}
I20260923 22:16:01.550832 18474 raft_consensus.cc:482] T 40e0f745b16b42e59aca8379b3f40153 P de2e7140d2c742e6983e06beafcffd7a [term 1 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260923 22:16:01.550972 18474 raft_consensus.cc:504] T 40e0f745b16b42e59aca8379b3f40153 P de2e7140d2c742e6983e06beafcffd7a [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "43f20a4dc81f48e3bdb7a2281b8505c2" member_type: VOTER last_known_addr { host: "127.1.135.195" port: 44883 } } peers { permanent_uuid: "1d61ec2a894f4c76afcd904fdb097bfc" member_type: VOTER last_known_addr { host: "127.1.135.194" port: 33393 } } peers { permanent_uuid: "de2e7140d2c742e6983e06beafcffd7a" member_type: VOTER last_known_addr { host: "127.1.135.193" port: 46309 } }
I20260923 22:16:01.551165 18474 leader_election.cc:288] T 40e0f745b16b42e59aca8379b3f40153 P de2e7140d2c742e6983e06beafcffd7a [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 43f20a4dc81f48e3bdb7a2281b8505c2 (127.1.135.195:44883), 1d61ec2a894f4c76afcd904fdb097bfc (127.1.135.194:33393)
W20260923 22:16:01.551538 18059 leader_election.cc:341] T 40e0f745b16b42e59aca8379b3f40153 P de2e7140d2c742e6983e06beafcffd7a [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 1d61ec2a894f4c76afcd904fdb097bfc (127.1.135.194:33393): Not found: Tablet not found: 40e0f745b16b42e59aca8379b3f40153
W20260923 22:16:01.551620 18060 leader_election.cc:341] T 40e0f745b16b42e59aca8379b3f40153 P de2e7140d2c742e6983e06beafcffd7a [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 43f20a4dc81f48e3bdb7a2281b8505c2 (127.1.135.195:44883): Not found: Tablet not found: 40e0f745b16b42e59aca8379b3f40153
I20260923 22:16:01.551683 18060 leader_election.cc:302] T 40e0f745b16b42e59aca8379b3f40153 P de2e7140d2c742e6983e06beafcffd7a [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: de2e7140d2c742e6983e06beafcffd7a; no voters: 1d61ec2a894f4c76afcd904fdb097bfc, 43f20a4dc81f48e3bdb7a2281b8505c2
I20260923 22:16:01.551766 18474 raft_consensus.cc:2726] T 40e0f745b16b42e59aca8379b3f40153 P de2e7140d2c742e6983e06beafcffd7a [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260923 22:16:02.102131 18475 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 22:16:02.101938 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.135.254:45507 (local address 127.1.135.193:44865)
0923 22:16:02.102058 (+ 120us) negotiation.cc:107] Waiting for socket to connect
0923 22:16:02.102088 (+ 30us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.135.254:45507: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":106,"thread_start_us":68,"threads_started":1}
I20260923 22:16:02.164508 18476 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 22:16:02.164308 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.135.254:45507 (local address 127.1.135.194:51765)
0923 22:16:02.164432 (+ 124us) negotiation.cc:107] Waiting for socket to connect
0923 22:16:02.164461 (+ 29us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.135.254:45507: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":111,"thread_start_us":71,"threads_started":1}
I20260923 22:16:02.288684 18477 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 22:16:02.288507 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.135.254:45507 (local address 127.1.135.195:48963)
0923 22:16:02.288607 (+ 100us) negotiation.cc:107] Waiting for socket to connect
0923 22:16:02.288641 (+ 34us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.135.254:45507: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":86,"thread_start_us":61,"threads_started":1}
I20260923 22:16:03.102739 18478 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 22:16:03.102525 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.135.254:45507 (local address 127.1.135.193:52121)
0923 22:16:03.102661 (+ 136us) negotiation.cc:107] Waiting for socket to connect
0923 22:16:03.102694 (+ 33us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.135.254:45507: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":121,"thread_start_us":70,"threads_started":1}
I20260923 22:16:03.165063 18479 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 22:16:03.164887 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.135.254:45507 (local address 127.1.135.194:56289)
0923 22:16:03.164997 (+ 110us) negotiation.cc:107] Waiting for socket to connect
0923 22:16:03.165026 (+ 29us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.135.254:45507: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":96,"thread_start_us":59,"threads_started":1}
I20260923 22:16:03.289275 18480 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 22:16:03.289048 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.135.254:45507 (local address 127.1.135.195:44645)
0923 22:16:03.289200 (+ 152us) negotiation.cc:107] Waiting for socket to connect
0923 22:16:03.289242 (+ 42us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.135.254:45507: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":137,"thread_start_us":80,"threads_started":1}
I20260923 22:16:04.103503 18481 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 22:16:04.103254 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.135.254:45507 (local address 127.1.135.193:41829)
0923 22:16:04.103408 (+ 154us) negotiation.cc:107] Waiting for socket to connect
0923 22:16:04.103456 (+ 48us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.135.254:45507: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":131,"thread_start_us":61,"threads_started":1}
I20260923 22:16:04.165776 18482 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 22:16:04.165583 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.135.254:45507 (local address 127.1.135.194:32783)
0923 22:16:04.165699 (+ 116us) negotiation.cc:107] Waiting for socket to connect
0923 22:16:04.165740 (+ 41us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.135.254:45507: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":97,"thread_start_us":56,"threads_started":1}
I20260923 22:16:04.290064 18483 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 22:16:04.289841 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.135.254:45507 (local address 127.1.135.195:34651)
0923 22:16:04.289986 (+ 145us) negotiation.cc:107] Waiting for socket to connect
0923 22:16:04.290027 (+ 41us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.135.254:45507: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":127,"thread_start_us":74,"threads_started":1}
I20260923 22:16:05.104290 18484 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 22:16:05.104050 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.135.254:45507 (local address 127.1.135.193:42597)
0923 22:16:05.104212 (+ 162us) negotiation.cc:107] Waiting for socket to connect
0923 22:16:05.104245 (+ 33us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.135.254:45507: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":145,"thread_start_us":102,"threads_started":1}
I20260923 22:16:05.166603 18485 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 22:16:05.166275 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.135.254:45507 (local address 127.1.135.194:47365)
0923 22:16:05.166520 (+ 245us) negotiation.cc:107] Waiting for socket to connect
0923 22:16:05.166552 (+ 32us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.135.254:45507: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":228,"thread_start_us":54,"threads_started":1}
I20260923 22:16:05.290772 18486 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 22:16:05.290532 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.135.254:45507 (local address 127.1.135.195:45397)
0923 22:16:05.290678 (+ 146us) negotiation.cc:107] Waiting for socket to connect
0923 22:16:05.290715 (+ 37us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.135.254:45507: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":119,"thread_start_us":70,"threads_started":1}
I20260923 22:16:06.104945 18487 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 22:16:06.104757 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.135.254:45507 (local address 127.1.135.193:39879)
0923 22:16:06.104866 (+ 109us) negotiation.cc:107] Waiting for socket to connect
0923 22:16:06.104900 (+ 34us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.135.254:45507: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":95,"thread_start_us":57,"threads_started":1}
I20260923 22:16:06.167212 18488 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 22:16:06.167011 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.135.254:45507 (local address 127.1.135.194:33007)
0923 22:16:06.167148 (+ 137us) negotiation.cc:107] Waiting for socket to connect
0923 22:16:06.167174 (+ 26us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.135.254:45507: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":116,"thread_start_us":86,"threads_started":1}
I20260923 22:16:06.291343 18489 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 22:16:06.291148 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.135.254:45507 (local address 127.1.135.195:50377)
0923 22:16:06.291253 (+ 105us) negotiation.cc:107] Waiting for socket to connect
0923 22:16:06.291281 (+ 28us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.135.254:45507: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":93,"thread_start_us":53,"threads_started":1}
I20260923 22:16:07.105792 18490 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 22:16:07.105545 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.135.254:45507 (local address 127.1.135.193:47115)
0923 22:16:07.105696 (+ 151us) negotiation.cc:107] Waiting for socket to connect
0923 22:16:07.105745 (+ 49us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.135.254:45507: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":131,"thread_start_us":72,"threads_started":1}
I20260923 22:16:07.168100 18491 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 22:16:07.167830 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.135.254:45507 (local address 127.1.135.194:54255)
0923 22:16:07.167984 (+ 154us) negotiation.cc:107] Waiting for socket to connect
0923 22:16:07.168027 (+ 43us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.135.254:45507: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":135,"thread_start_us":82,"threads_started":1}
I20260923 22:16:07.286754 1567 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskRiQe_a/build/release/bin/kudu with pid 18007
I20260923 22:16:07.291486 1567 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskRiQe_a/build/release/bin/kudu with pid 18175
I20260923 22:16:07.292796 18492 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 22:16:07.292583 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.135.254:45507 (local address 127.1.135.195:59073)
0923 22:16:07.292709 (+ 126us) negotiation.cc:107] Waiting for socket to connect
0923 22:16:07.292744 (+ 35us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.135.254:45507: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":110,"thread_start_us":61,"threads_started":1}
I20260923 22:16:07.295718 1567 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskRiQe_a/build/release/bin/kudu with pid 18308
2026-09-23T22:16:07Z chronyd exiting
[ OK ] SecurityITest.TestEnableAndDisableEncryptedTSK (58360 ms)
[----------] 30 tests from SecurityITest (227790 ms total)
[----------] 6 tests from EncryptionPolicyTest
[ RUN ] EncryptionPolicyTest.TestEncryptionPolicy/0
Loading random data
Initializing database '/tmp/dist-test-taskRiQe_a/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790201539513534-1567-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Sep 23 22:16:07 dist-test-slave-s2fj krb5kdc[18495](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Sep 23 22:16:07 dist-test-slave-s2fj krb5kdc[18495](info): set up 2 sockets
Sep 23 22:16:07 dist-test-slave-s2fj krb5kdc[18495](info): commencing operation
krb5kdc: starting...
W20260923 22:16:08.355564 1567 mini_kdc.cc:121] Time spent starting KDC: real 1.039s user 0.000s sys 0.005s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskRiQe_a/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790201539513534-1567-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskRiQe_a/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790201539513534-1567-0/krb5kdc/test-user.keytab.
Sep 23 22:16:08 dist-test-slave-s2fj krb5kdc[18495](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790201768, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-09-23T22:16:08Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-09-23T22:16:08Z Disabled control of system clock
WARNING: no policy specified for kudu/127.1.135.254@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.1.135.254@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.1.135.254 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskRiQe_a/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790201539513534-1567-0/krb5kdc/kudu_127.1.135.254.keytab.
Entry for principal kudu/127.1.135.254 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskRiQe_a/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790201539513534-1567-0/krb5kdc/kudu_127.1.135.254.keytab.
I20260923 22:16:08.466967 1567 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskRiQe_a/build/release/bin/kudu
/tmp/dist-test-taskRiQe_a/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskRiQe_a/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790201539513534-1567-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskRiQe_a/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790201539513534-1567-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskRiQe_a/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790201539513534-1567-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskRiQe_a/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790201539513534-1567-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.1.135.254:38385
--webserver_interface=127.1.135.254
--webserver_port=0
--keytab_file=/tmp/dist-test-taskRiQe_a/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790201539513534-1567-0/krb5kdc/kudu_127.1.135.254.keytab
--principal=kudu/127.1.135.254
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.1.135.212:43605
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.1.135.254:38385
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskRiQe_a/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790201539513534-1567-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskRiQe_a/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790201539513534-1567-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskRiQe_a/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790201539513534-1567-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260923 22:16:08.545557 18508 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260923 22:16:08.545742 18508 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260923 22:16:08.545764 18508 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260923 22:16:08.546880 18508 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260923 22:16:08.546919 18508 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260923 22:16:08.546933 18508 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260923 22:16:08.546972 18508 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260923 22:16:08.546998 18508 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260923 22:16:08.548431 18508 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.1.135.212:43605
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskRiQe_a/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790201539513534-1567-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskRiQe_a/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790201539513534-1567-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.1.135.254:38385
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskRiQe_a/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790201539513534-1567-0/krb5kdc/kudu_127.1.135.254.keytab
--principal=kudu/127.1.135.254
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.1.135.254:38385
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskRiQe_a/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790201539513534-1567-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.1.135.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.18508
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskRiQe_a/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790201539513534-1567-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 23 Sep 2026 20:53:37 UTC on ccycloud.quasar-rkviyx.root.comops.site
build id 4668083
I20260923 22:16:08.548631 18508 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260923 22:16:08.548857 18508 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260923 22:16:08.553620 18514 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260923 22:16:08.553647 18513 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260923 22:16:08.553647 18516 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260923 22:16:08.553694 18508 server_base.cc:1034] running on GCE node
I20260923 22:16:08.554105 18508 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260923 22:16:08.554384 18508 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260923 22:16:08.555546 18508 hybrid_clock.cc:648] HybridClock initialized: now 1790201768555532 us; error 44 us; skew 500 ppm
Sep 23 22:16:08 dist-test-slave-s2fj krb5kdc[18495](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790201768, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.135.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260923 22:16:08.557924 18508 init.cc:377] Logged in from keytab as kudu/127.1.135.254@KRBTEST.COM (short username kudu)
I20260923 22:16:08.558312 18508 webserver.cc:460] Webserver started at http://127.1.135.254:43847/ using document root <none> and password file <none>
I20260923 22:16:08.558534 18508 fs_manager.cc:359] Metadata directory not provided
I20260923 22:16:08.558596 18508 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260923 22:16:08.558733 18508 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260923 22:16:08.559720 18508 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskRiQe_a/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790201539513534-1567-0/minicluster-data/master-0/data/instance:
uuid: "20d73f4b1d8a4851ba5e5aa4c58c4b82"
format_stamp: "Formatted at 2026-09-23 22:16:08 on dist-test-slave-s2fj"
server_key: "39ca045e073a70c7b04560f4d99e1c92"
server_key_iv: "bc0394758ea22174adecd8125e59303f"
server_key_version: "encryptionkey@0"
I20260923 22:16:08.560104 18508 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskRiQe_a/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790201539513534-1567-0/minicluster-data/master-0/wal/instance:
uuid: "20d73f4b1d8a4851ba5e5aa4c58c4b82"
format_stamp: "Formatted at 2026-09-23 22:16:08 on dist-test-slave-s2fj"
server_key: "39ca045e073a70c7b04560f4d99e1c92"
server_key_iv: "bc0394758ea22174adecd8125e59303f"
server_key_version: "encryptionkey@0"
I20260923 22:16:08.561491 18508 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.001s
I20260923 22:16:08.562197 18523 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260923 22:16:08.562391 18508 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.001s
I20260923 22:16:08.562490 18508 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskRiQe_a/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790201539513534-1567-0/minicluster-data/master-0/data,/tmp/dist-test-taskRiQe_a/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790201539513534-1567-0/minicluster-data/master-0/wal
uuid: "20d73f4b1d8a4851ba5e5aa4c58c4b82"
format_stamp: "Formatted at 2026-09-23 22:16:08 on dist-test-slave-s2fj"
server_key: "39ca045e073a70c7b04560f4d99e1c92"
server_key_iv: "bc0394758ea22174adecd8125e59303f"
server_key_version: "encryptionkey@0"
I20260923 22:16:08.562577 18508 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskRiQe_a/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790201539513534-1567-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskRiQe_a/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790201539513534-1567-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskRiQe_a/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790201539513534-1567-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260923 22:16:08.582484 18508 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260923 22:16:08.584699 18508 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260923 22:16:08.584862 18508 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260923 22:16:08.588707 18508 rpc_server.cc:307] RPC server started. Bound to: 127.1.135.254:38385
I20260923 22:16:08.588761 18575 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.1.135.254:38385 every 8 connection(s)
I20260923 22:16:08.589119 18508 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskRiQe_a/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790201539513534-1567-0/minicluster-data/master-0/data/info.pb
I20260923 22:16:08.589905 18576 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260923 22:16:08.591180 1567 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskRiQe_a/build/release/bin/kudu as pid 18508
I20260923 22:16:08.591295 1567 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskRiQe_a/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790201539513534-1567-0/minicluster-data/master-0/wal/instance
I20260923 22:16:08.591502 1567 external_mini_cluster.cc:1428] Setting key 13e02e742d105aed9a6f4adef3b436b8
I20260923 22:16:08.592587 18576 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 20d73f4b1d8a4851ba5e5aa4c58c4b82: Bootstrap starting.
I20260923 22:16:08.593127 18576 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 20d73f4b1d8a4851ba5e5aa4c58c4b82: Neither blocks nor log segments found. Creating new log.
I20260923 22:16:08.593416 18576 log.cc:824] T 00000000000000000000000000000000 P 20d73f4b1d8a4851ba5e5aa4c58c4b82: Log is configured to *not* fsync() on all Append() calls
I20260923 22:16:08.594095 18576 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 20d73f4b1d8a4851ba5e5aa4c58c4b82: No bootstrap required, opened a new log
I20260923 22:16:08.595728 18576 raft_consensus.cc:348] T 00000000000000000000000000000000 P 20d73f4b1d8a4851ba5e5aa4c58c4b82 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "20d73f4b1d8a4851ba5e5aa4c58c4b82" member_type: VOTER last_known_addr { host: "127.1.135.254" port: 38385 } }
I20260923 22:16:08.595897 18576 raft_consensus.cc:374] T 00000000000000000000000000000000 P 20d73f4b1d8a4851ba5e5aa4c58c4b82 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260923 22:16:08.595963 18576 raft_consensus.cc:729] T 00000000000000000000000000000000 P 20d73f4b1d8a4851ba5e5aa4c58c4b82 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 20d73f4b1d8a4851ba5e5aa4c58c4b82, State: Initialized, Role: FOLLOWER
I20260923 22:16:08.596120 18576 consensus_queue.cc:260] T 00000000000000000000000000000000 P 20d73f4b1d8a4851ba5e5aa4c58c4b82 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "20d73f4b1d8a4851ba5e5aa4c58c4b82" member_type: VOTER last_known_addr { host: "127.1.135.254" port: 38385 } }
I20260923 22:16:08.596225 18576 raft_consensus.cc:388] T 00000000000000000000000000000000 P 20d73f4b1d8a4851ba5e5aa4c58c4b82 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260923 22:16:08.596269 18576 raft_consensus.cc:482] T 00000000000000000000000000000000 P 20d73f4b1d8a4851ba5e5aa4c58c4b82 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260923 22:16:08.596320 18576 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 20d73f4b1d8a4851ba5e5aa4c58c4b82 [term 0 FOLLOWER]: Advancing to term 1
I20260923 22:16:08.596804 18576 raft_consensus.cc:504] T 00000000000000000000000000000000 P 20d73f4b1d8a4851ba5e5aa4c58c4b82 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "20d73f4b1d8a4851ba5e5aa4c58c4b82" member_type: VOTER last_known_addr { host: "127.1.135.254" port: 38385 } }
I20260923 22:16:08.596890 18576 leader_election.cc:302] T 00000000000000000000000000000000 P 20d73f4b1d8a4851ba5e5aa4c58c4b82 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 20d73f4b1d8a4851ba5e5aa4c58c4b82; no voters:
I20260923 22:16:08.597090 18576 leader_election.cc:288] T 00000000000000000000000000000000 P 20d73f4b1d8a4851ba5e5aa4c58c4b82 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260923 22:16:08.597115 18581 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 20d73f4b1d8a4851ba5e5aa4c58c4b82 [term 1 FOLLOWER]: Leader election won for term 1
I20260923 22:16:08.597239 18581 raft_consensus.cc:686] T 00000000000000000000000000000000 P 20d73f4b1d8a4851ba5e5aa4c58c4b82 [term 1 LEADER]: Becoming Leader. State: Replica: 20d73f4b1d8a4851ba5e5aa4c58c4b82, State: Running, Role: LEADER
I20260923 22:16:08.597338 18576 sys_catalog.cc:564] T 00000000000000000000000000000000 P 20d73f4b1d8a4851ba5e5aa4c58c4b82 [sys.catalog]: configured and running, proceeding with master startup.
I20260923 22:16:08.597314 18581 consensus_queue.cc:237] T 00000000000000000000000000000000 P 20d73f4b1d8a4851ba5e5aa4c58c4b82 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "20d73f4b1d8a4851ba5e5aa4c58c4b82" member_type: VOTER last_known_addr { host: "127.1.135.254" port: 38385 } }
Sep 23 22:16:08 dist-test-slave-s2fj krb5kdc[18495](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790201768, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.1.135.254@KRBTEST.COM
I20260923 22:16:08.597867 18582 sys_catalog.cc:455] T 00000000000000000000000000000000 P 20d73f4b1d8a4851ba5e5aa4c58c4b82 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 20d73f4b1d8a4851ba5e5aa4c58c4b82. Latest consensus state: current_term: 1 leader_uuid: "20d73f4b1d8a4851ba5e5aa4c58c4b82" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "20d73f4b1d8a4851ba5e5aa4c58c4b82" member_type: VOTER last_known_addr { host: "127.1.135.254" port: 38385 } } }
I20260923 22:16:08.598289 18582 sys_catalog.cc:458] T 00000000000000000000000000000000 P 20d73f4b1d8a4851ba5e5aa4c58c4b82 [sys.catalog]: This master's current role is: LEADER
I20260923 22:16:08.598479 18583 sys_catalog.cc:455] T 00000000000000000000000000000000 P 20d73f4b1d8a4851ba5e5aa4c58c4b82 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "20d73f4b1d8a4851ba5e5aa4c58c4b82" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "20d73f4b1d8a4851ba5e5aa4c58c4b82" member_type: VOTER last_known_addr { host: "127.1.135.254" port: 38385 } } }
I20260923 22:16:08.598529 18592 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260923 22:16:08.599114 18592 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260923 22:16:08.599745 18583 sys_catalog.cc:458] T 00000000000000000000000000000000 P 20d73f4b1d8a4851ba5e5aa4c58c4b82 [sys.catalog]: This master's current role is: LEADER
I20260923 22:16:08.600663 18592 catalog_manager.cc:1347] Generated new cluster ID: a9813de8793d454eb6b94325afdfb17d
I20260923 22:16:08.600708 18592 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260923 22:16:08.602110 18579 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 22:16:08.591992 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37994 (local address 127.1.135.254:38385)
0923 22:16:08.592171 (+ 179us) server_negotiation.cc:206] Beginning negotiation
0923 22:16:08.592179 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0923 22:16:08.592210 (+ 31us) server_negotiation.cc:407] Connection header received
0923 22:16:08.592573 (+ 363us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 22:16:08.592579 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 22:16:08.592624 (+ 45us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 22:16:08.592778 (+ 154us) server_negotiation.cc:226] Negotiated authn=SASL
0923 22:16:08.593452 (+ 674us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 22:16:08.594435 (+ 983us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 22:16:08.595181 (+ 746us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 22:16:08.595581 (+ 400us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 22:16:08.595700 (+ 119us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 22:16:08.598368 (+ 2668us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0923 22:16:08.598373 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0923 22:16:08.598373 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0923 22:16:08.598384 (+ 11us) server_negotiation.cc:922] calling sasl_server_start()
0923 22:16:08.600500 (+ 2116us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0923 22:16:08.600915 (+ 415us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0923 22:16:08.600917 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0923 22:16:08.600918 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0923 22:16:08.601031 (+ 113us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0923 22:16:08.601239 (+ 208us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0923 22:16:08.601240 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0923 22:16:08.601240 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0923 22:16:08.601542 (+ 302us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0923 22:16:08.601694 (+ 152us) server_negotiation.cc:1035] Waiting for connection context
0923 22:16:08.601776 (+ 82us) server_negotiation.cc:299] Negotiation successful
0923 22:16:08.601912 (+ 136us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":123,"thread_start_us":89,"threads_started":1}
I20260923 22:16:08.608444 18592 catalog_manager.cc:1370] Generated new certificate authority record
I20260923 22:16:08.608939 18592 catalog_manager.cc:1504] Loading token signing keys...
I20260923 22:16:08.624869 18592 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 20d73f4b1d8a4851ba5e5aa4c58c4b82: Generated new TSK 0
I20260923 22:16:08.625051 18592 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.1.135.193@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.1.135.193@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.1.135.193 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskRiQe_a/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790201539513534-1567-0/krb5kdc/kudu_127.1.135.193.keytab.
Entry for principal kudu/127.1.135.193 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskRiQe_a/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790201539513534-1567-0/krb5kdc/kudu_127.1.135.193.keytab.
I20260923 22:16:08.658617 1567 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskRiQe_a/build/release/bin/kudu
/tmp/dist-test-taskRiQe_a/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskRiQe_a/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790201539513534-1567-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskRiQe_a/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790201539513534-1567-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskRiQe_a/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790201539513534-1567-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskRiQe_a/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790201539513534-1567-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.1.135.193:0
--local_ip_for_outbound_sockets=127.1.135.193
--webserver_interface=127.1.135.193
--webserver_port=0
--tserver_master_addrs=127.1.135.254:38385
--keytab_file=/tmp/dist-test-taskRiQe_a/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790201539513534-1567-0/krb5kdc/kudu_127.1.135.193.keytab
--principal=kudu/127.1.135.193
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.1.135.212:43605
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskRiQe_a/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790201539513534-1567-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskRiQe_a/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790201539513534-1567-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskRiQe_a/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790201539513534-1567-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260923 22:16:08.736673 18602 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260923 22:16:08.736907 18602 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260923 22:16:08.736946 18602 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260923 22:16:08.738090 18602 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260923 22:16:08.738181 18602 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260923 22:16:08.738226 18602 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.1.135.193
I20260923 22:16:08.739660 18602 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.1.135.212:43605
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskRiQe_a/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790201539513534-1567-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskRiQe_a/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790201539513534-1567-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskRiQe_a/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790201539513534-1567-0/krb5kdc/kudu_127.1.135.193.keytab
--principal=kudu/127.1.135.193
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.1.135.193:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskRiQe_a/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790201539513534-1567-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.1.135.193
--webserver_port=0
--tserver_master_addrs=127.1.135.254:38385
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.18602
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.1.135.193
--log_dir=/tmp/dist-test-taskRiQe_a/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790201539513534-1567-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 23 Sep 2026 20:53:37 UTC on ccycloud.quasar-rkviyx.root.comops.site
build id 4668083
I20260923 22:16:08.739921 18602 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260923 22:16:08.740173 18602 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260923 22:16:08.744904 18607 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260923 22:16:08.744927 18608 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260923 22:16:08.745083 18602 server_base.cc:1034] running on GCE node
W20260923 22:16:08.745095 18610 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260923 22:16:08.745333 18602 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260923 22:16:08.745606 18602 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260923 22:16:08.746770 18602 hybrid_clock.cc:648] HybridClock initialized: now 1790201768746769 us; error 35 us; skew 500 ppm
Sep 23 22:16:08 dist-test-slave-s2fj krb5kdc[18495](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790201768, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.135.193@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260923 22:16:08.749092 18602 init.cc:377] Logged in from keytab as kudu/127.1.135.193@KRBTEST.COM (short username kudu)
I20260923 22:16:08.749487 18602 webserver.cc:460] Webserver started at http://127.1.135.193:43225/ using document root <none> and password file <none>
I20260923 22:16:08.749704 18602 fs_manager.cc:359] Metadata directory not provided
I20260923 22:16:08.749771 18602 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260923 22:16:08.749917 18602 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260923 22:16:08.750864 18602 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskRiQe_a/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790201539513534-1567-0/minicluster-data/ts-0/data/instance:
uuid: "1c89669cd4df4d2a9d7f4c80f199a124"
format_stamp: "Formatted at 2026-09-23 22:16:08 on dist-test-slave-s2fj"
server_key: "b0987e2cdfd283010824ac49394256d3"
server_key_iv: "e0a5ff6205d1a5d394051a7a4e989af1"
server_key_version: "encryptionkey@0"
I20260923 22:16:08.751228 18602 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskRiQe_a/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790201539513534-1567-0/minicluster-data/ts-0/wal/instance:
uuid: "1c89669cd4df4d2a9d7f4c80f199a124"
format_stamp: "Formatted at 2026-09-23 22:16:08 on dist-test-slave-s2fj"
server_key: "b0987e2cdfd283010824ac49394256d3"
server_key_iv: "e0a5ff6205d1a5d394051a7a4e989af1"
server_key_version: "encryptionkey@0"
I20260923 22:16:08.752633 18602 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.001s sys 0.002s
I20260923 22:16:08.753332 18617 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260923 22:16:08.753515 18602 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260923 22:16:08.753609 18602 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskRiQe_a/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790201539513534-1567-0/minicluster-data/ts-0/data,/tmp/dist-test-taskRiQe_a/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790201539513534-1567-0/minicluster-data/ts-0/wal
uuid: "1c89669cd4df4d2a9d7f4c80f199a124"
format_stamp: "Formatted at 2026-09-23 22:16:08 on dist-test-slave-s2fj"
server_key: "b0987e2cdfd283010824ac49394256d3"
server_key_iv: "e0a5ff6205d1a5d394051a7a4e989af1"
server_key_version: "encryptionkey@0"
I20260923 22:16:08.753685 18602 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskRiQe_a/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790201539513534-1567-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskRiQe_a/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790201539513534-1567-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskRiQe_a/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790201539513534-1567-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260923 22:16:08.776691 18602 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260923 22:16:08.778439 18602 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260923 22:16:08.778595 18602 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260923 22:16:08.778801 18602 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260923 22:16:08.779124 18602 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260923 22:16:08.779183 18602 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260923 22:16:08.779222 18602 ts_tablet_manager.cc:610] Registered 0 tablets
I20260923 22:16:08.779251 18602 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260923 22:16:08.785231 18602 rpc_server.cc:307] RPC server started. Bound to: 127.1.135.193:41015
I20260923 22:16:08.785274 18730 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.1.135.193:41015 every 8 connection(s)
I20260923 22:16:08.785567 18602 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskRiQe_a/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790201539513534-1567-0/minicluster-data/ts-0/data/info.pb
Sep 23 22:16:08 dist-test-slave-s2fj krb5kdc[18495](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790201768, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.135.193@KRBTEST.COM for kudu/127.1.135.254@KRBTEST.COM
I20260923 22:16:08.793704 1567 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskRiQe_a/build/release/bin/kudu as pid 18602
I20260923 22:16:08.793813 1567 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskRiQe_a/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790201539513534-1567-0/minicluster-data/ts-0/wal/instance
I20260923 22:16:08.794032 18579 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 22:16:08.786267 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.135.193:49921 (local address 127.1.135.254:38385)
0923 22:16:08.786325 (+ 58us) server_negotiation.cc:206] Beginning negotiation
0923 22:16:08.786327 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0923 22:16:08.786527 (+ 200us) server_negotiation.cc:407] Connection header received
0923 22:16:08.786972 (+ 445us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 22:16:08.786972 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 22:16:08.786982 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 22:16:08.787029 (+ 47us) server_negotiation.cc:226] Negotiated authn=SASL
0923 22:16:08.787867 (+ 838us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 22:16:08.788417 (+ 550us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 22:16:08.789329 (+ 912us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 22:16:08.789524 (+ 195us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 22:16:08.789573 (+ 49us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 22:16:08.791644 (+ 2071us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0923 22:16:08.791645 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0923 22:16:08.791646 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0923 22:16:08.791652 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0923 22:16:08.792846 (+ 1194us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0923 22:16:08.793191 (+ 345us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0923 22:16:08.793191 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0923 22:16:08.793192 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0923 22:16:08.793239 (+ 47us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0923 22:16:08.793408 (+ 169us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0923 22:16:08.793409 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0923 22:16:08.793409 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0923 22:16:08.793521 (+ 112us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0923 22:16:08.793592 (+ 71us) server_negotiation.cc:1035] Waiting for connection context
0923 22:16:08.793843 (+ 251us) server_negotiation.cc:299] Negotiation successful
0923 22:16:08.793931 (+ 88us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":30}
I20260923 22:16:08.794051 1567 external_mini_cluster.cc:1428] Setting key 9ab25406f5f8a92b220e866313687cf9
I20260923 22:16:08.794170 18733 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 22:16:08.786254 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.135.254:38385 (local address 127.1.135.193:49921)
0923 22:16:08.786421 (+ 167us) negotiation.cc:107] Waiting for socket to connect
0923 22:16:08.786452 (+ 31us) client_negotiation.cc:174] Beginning negotiation
0923 22:16:08.786895 (+ 443us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0923 22:16:08.787086 (+ 191us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0923 22:16:08.787089 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0923 22:16:08.787618 (+ 529us) client_negotiation.cc:189] Negotiated authn=SASL
0923 22:16:08.787793 (+ 175us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 22:16:08.787799 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 22:16:08.788488 (+ 689us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 22:16:08.788489 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 22:16:08.789256 (+ 767us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 22:16:08.789258 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 22:16:08.789594 (+ 336us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 22:16:08.789595 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 22:16:08.789694 (+ 99us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 22:16:08.790200 (+ 506us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0923 22:16:08.790214 (+ 14us) client_negotiation.cc:656] calling sasl_client_start()
0923 22:16:08.791558 (+ 1344us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0923 22:16:08.792915 (+ 1357us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0923 22:16:08.792921 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0923 22:16:08.792923 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0923 22:16:08.793126 (+ 203us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0923 22:16:08.793280 (+ 154us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0923 22:16:08.793281 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0923 22:16:08.793281 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0923 22:16:08.793358 (+ 77us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0923 22:16:08.793587 (+ 229us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0923 22:16:08.793592 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0923 22:16:08.793694 (+ 102us) client_negotiation.cc:769] Sending connection context
0923 22:16:08.793802 (+ 108us) client_negotiation.cc:240] Negotiation successful
0923 22:16:08.793971 (+ 169us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":132,"thread_start_us":90,"threads_started":1}
I20260923 22:16:08.794735 18731 heartbeater.cc:347] Connected to a master server at 127.1.135.254:38385
I20260923 22:16:08.794839 18731 heartbeater.cc:464] Registering TS with master...
I20260923 22:16:08.795006 18731 heartbeater.cc:511] Master 127.1.135.254:38385 requested a full tablet report, sending...
I20260923 22:16:08.795508 18540 ts_manager.cc:194] Registered new tserver with Master: 1c89669cd4df4d2a9d7f4c80f199a124 (127.1.135.193:41015)
I20260923 22:16:08.796191 18540 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.1.135.193@KRBTEST.COM'} at 127.1.135.193:49921
WARNING: no policy specified for kudu/127.1.135.194@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.1.135.194@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.1.135.194 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskRiQe_a/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790201539513534-1567-0/krb5kdc/kudu_127.1.135.194.keytab.
Entry for principal kudu/127.1.135.194 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskRiQe_a/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790201539513534-1567-0/krb5kdc/kudu_127.1.135.194.keytab.
I20260923 22:16:08.818073 1567 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskRiQe_a/build/release/bin/kudu
/tmp/dist-test-taskRiQe_a/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskRiQe_a/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790201539513534-1567-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskRiQe_a/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790201539513534-1567-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskRiQe_a/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790201539513534-1567-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskRiQe_a/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790201539513534-1567-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.1.135.194:0
--local_ip_for_outbound_sockets=127.1.135.194
--webserver_interface=127.1.135.194
--webserver_port=0
--tserver_master_addrs=127.1.135.254:38385
--keytab_file=/tmp/dist-test-taskRiQe_a/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790201539513534-1567-0/krb5kdc/kudu_127.1.135.194.keytab
--principal=kudu/127.1.135.194
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.1.135.212:43605
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskRiQe_a/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790201539513534-1567-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskRiQe_a/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790201539513534-1567-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskRiQe_a/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790201539513534-1567-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260923 22:16:08.895442 18736 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260923 22:16:08.895635 18736 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260923 22:16:08.895658 18736 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260923 22:16:08.896996 18736 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260923 22:16:08.897037 18736 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260923 22:16:08.897070 18736 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.1.135.194
I20260923 22:16:08.898693 18736 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.1.135.212:43605
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskRiQe_a/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790201539513534-1567-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskRiQe_a/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790201539513534-1567-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskRiQe_a/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790201539513534-1567-0/krb5kdc/kudu_127.1.135.194.keytab
--principal=kudu/127.1.135.194
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.1.135.194:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskRiQe_a/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790201539513534-1567-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.1.135.194
--webserver_port=0
--tserver_master_addrs=127.1.135.254:38385
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.18736
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.1.135.194
--log_dir=/tmp/dist-test-taskRiQe_a/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790201539513534-1567-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 23 Sep 2026 20:53:37 UTC on ccycloud.quasar-rkviyx.root.comops.site
build id 4668083
I20260923 22:16:08.898923 18736 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260923 22:16:08.899142 18736 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260923 22:16:08.903769 18744 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260923 22:16:08.903795 18742 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260923 22:16:08.903868 18741 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260923 22:16:08.904119 18736 server_base.cc:1034] running on GCE node
I20260923 22:16:08.904358 18736 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260923 22:16:08.904611 18736 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260923 22:16:08.905763 18736 hybrid_clock.cc:648] HybridClock initialized: now 1790201768905745 us; error 30 us; skew 500 ppm
Sep 23 22:16:08 dist-test-slave-s2fj krb5kdc[18495](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790201768, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.135.194@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260923 22:16:08.908324 18736 init.cc:377] Logged in from keytab as kudu/127.1.135.194@KRBTEST.COM (short username kudu)
I20260923 22:16:08.908730 18736 webserver.cc:460] Webserver started at http://127.1.135.194:35431/ using document root <none> and password file <none>
I20260923 22:16:08.908962 18736 fs_manager.cc:359] Metadata directory not provided
I20260923 22:16:08.909029 18736 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260923 22:16:08.909176 18736 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260923 22:16:08.910163 18736 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskRiQe_a/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790201539513534-1567-0/minicluster-data/ts-1/data/instance:
uuid: "4cecc19a08b0495282c8939a7e0b5da6"
format_stamp: "Formatted at 2026-09-23 22:16:08 on dist-test-slave-s2fj"
server_key: "911280231e92c9d77a55f1e3a412df85"
server_key_iv: "2cfc2f028b7cd001cdd8091d19e7718a"
server_key_version: "encryptionkey@0"
I20260923 22:16:08.910511 18736 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskRiQe_a/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790201539513534-1567-0/minicluster-data/ts-1/wal/instance:
uuid: "4cecc19a08b0495282c8939a7e0b5da6"
format_stamp: "Formatted at 2026-09-23 22:16:08 on dist-test-slave-s2fj"
server_key: "911280231e92c9d77a55f1e3a412df85"
server_key_iv: "2cfc2f028b7cd001cdd8091d19e7718a"
server_key_version: "encryptionkey@0"
I20260923 22:16:08.911751 18736 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.002s
I20260923 22:16:08.912518 18751 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260923 22:16:08.912667 18736 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.000s
I20260923 22:16:08.912765 18736 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskRiQe_a/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790201539513534-1567-0/minicluster-data/ts-1/data,/tmp/dist-test-taskRiQe_a/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790201539513534-1567-0/minicluster-data/ts-1/wal
uuid: "4cecc19a08b0495282c8939a7e0b5da6"
format_stamp: "Formatted at 2026-09-23 22:16:08 on dist-test-slave-s2fj"
server_key: "911280231e92c9d77a55f1e3a412df85"
server_key_iv: "2cfc2f028b7cd001cdd8091d19e7718a"
server_key_version: "encryptionkey@0"
I20260923 22:16:08.912840 18736 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskRiQe_a/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790201539513534-1567-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskRiQe_a/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790201539513534-1567-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskRiQe_a/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790201539513534-1567-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260923 22:16:08.920106 18736 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260923 22:16:08.922436 18736 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260923 22:16:08.922582 18736 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260923 22:16:08.922780 18736 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260923 22:16:08.923063 18736 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260923 22:16:08.923115 18736 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260923 22:16:08.923154 18736 ts_tablet_manager.cc:610] Registered 0 tablets
I20260923 22:16:08.923182 18736 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260923 22:16:08.928663 18736 rpc_server.cc:307] RPC server started. Bound to: 127.1.135.194:45839
I20260923 22:16:08.928728 18864 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.1.135.194:45839 every 8 connection(s)
I20260923 22:16:08.929034 18736 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskRiQe_a/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790201539513534-1567-0/minicluster-data/ts-1/data/info.pb
I20260923 22:16:08.932544 1567 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskRiQe_a/build/release/bin/kudu as pid 18736
I20260923 22:16:08.932646 1567 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskRiQe_a/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790201539513534-1567-0/minicluster-data/ts-1/wal/instance
I20260923 22:16:08.932843 1567 external_mini_cluster.cc:1428] Setting key bb38aa0934b8e3fd507fdbc98e38f5af
Sep 23 22:16:08 dist-test-slave-s2fj krb5kdc[18495](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790201768, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.135.194@KRBTEST.COM for kudu/127.1.135.254@KRBTEST.COM
I20260923 22:16:08.938315 18579 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 22:16:08.929703 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.135.194:49039 (local address 127.1.135.254:38385)
0923 22:16:08.929768 (+ 65us) server_negotiation.cc:206] Beginning negotiation
0923 22:16:08.929771 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0923 22:16:08.929955 (+ 184us) server_negotiation.cc:407] Connection header received
0923 22:16:08.930399 (+ 444us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 22:16:08.930400 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 22:16:08.930409 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 22:16:08.930459 (+ 50us) server_negotiation.cc:226] Negotiated authn=SASL
0923 22:16:08.931247 (+ 788us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 22:16:08.931717 (+ 470us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 22:16:08.933034 (+ 1317us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 22:16:08.933275 (+ 241us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 22:16:08.933356 (+ 81us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 22:16:08.935459 (+ 2103us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0923 22:16:08.935461 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0923 22:16:08.935461 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0923 22:16:08.935469 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0923 22:16:08.936934 (+ 1465us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0923 22:16:08.937371 (+ 437us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0923 22:16:08.937372 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0923 22:16:08.937373 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0923 22:16:08.937427 (+ 54us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0923 22:16:08.937684 (+ 257us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0923 22:16:08.937685 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0923 22:16:08.937685 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0923 22:16:08.937800 (+ 115us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0923 22:16:08.937888 (+ 88us) server_negotiation.cc:1035] Waiting for connection context
0923 22:16:08.938148 (+ 260us) server_negotiation.cc:299] Negotiation successful
0923 22:16:08.938226 (+ 78us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":38}
I20260923 22:16:08.938517 18867 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 22:16:08.929682 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.135.254:38385 (local address 127.1.135.194:49039)
0923 22:16:08.929848 (+ 166us) negotiation.cc:107] Waiting for socket to connect
0923 22:16:08.929879 (+ 31us) client_negotiation.cc:174] Beginning negotiation
0923 22:16:08.930320 (+ 441us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0923 22:16:08.930507 (+ 187us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0923 22:16:08.930510 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0923 22:16:08.931022 (+ 512us) client_negotiation.cc:189] Negotiated authn=SASL
0923 22:16:08.931180 (+ 158us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 22:16:08.931184 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 22:16:08.931786 (+ 602us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 22:16:08.931786 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 22:16:08.932931 (+ 1145us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 22:16:08.932934 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 22:16:08.933367 (+ 433us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 22:16:08.933368 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 22:16:08.933501 (+ 133us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 22:16:08.934017 (+ 516us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0923 22:16:08.934024 (+ 7us) client_negotiation.cc:656] calling sasl_client_start()
0923 22:16:08.935339 (+ 1315us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0923 22:16:08.937034 (+ 1695us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0923 22:16:08.937040 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0923 22:16:08.937042 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0923 22:16:08.937276 (+ 234us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0923 22:16:08.937511 (+ 235us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0923 22:16:08.937512 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0923 22:16:08.937512 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0923 22:16:08.937617 (+ 105us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0923 22:16:08.937882 (+ 265us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0923 22:16:08.937888 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0923 22:16:08.938020 (+ 132us) client_negotiation.cc:769] Sending connection context
0923 22:16:08.938125 (+ 105us) client_negotiation.cc:240] Negotiation successful
0923 22:16:08.938273 (+ 148us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":133,"thread_start_us":86,"threads_started":1}
I20260923 22:16:08.938999 18865 heartbeater.cc:347] Connected to a master server at 127.1.135.254:38385
I20260923 22:16:08.939095 18865 heartbeater.cc:464] Registering TS with master...
I20260923 22:16:08.939258 18865 heartbeater.cc:511] Master 127.1.135.254:38385 requested a full tablet report, sending...
I20260923 22:16:08.939643 18540 ts_manager.cc:194] Registered new tserver with Master: 4cecc19a08b0495282c8939a7e0b5da6 (127.1.135.194:45839)
I20260923 22:16:08.940200 18540 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.1.135.194@KRBTEST.COM'} at 127.1.135.194:49039
WARNING: no policy specified for kudu/127.1.135.195@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.1.135.195@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.1.135.195 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskRiQe_a/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790201539513534-1567-0/krb5kdc/kudu_127.1.135.195.keytab.
Entry for principal kudu/127.1.135.195 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskRiQe_a/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790201539513534-1567-0/krb5kdc/kudu_127.1.135.195.keytab.
I20260923 22:16:08.957237 1567 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskRiQe_a/build/release/bin/kudu
/tmp/dist-test-taskRiQe_a/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskRiQe_a/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790201539513534-1567-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskRiQe_a/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790201539513534-1567-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskRiQe_a/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790201539513534-1567-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskRiQe_a/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790201539513534-1567-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.1.135.195:0
--local_ip_for_outbound_sockets=127.1.135.195
--webserver_interface=127.1.135.195
--webserver_port=0
--tserver_master_addrs=127.1.135.254:38385
--keytab_file=/tmp/dist-test-taskRiQe_a/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790201539513534-1567-0/krb5kdc/kudu_127.1.135.195.keytab
--principal=kudu/127.1.135.195
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.1.135.212:43605
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskRiQe_a/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790201539513534-1567-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskRiQe_a/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790201539513534-1567-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskRiQe_a/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790201539513534-1567-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260923 22:16:09.037151 18870 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260923 22:16:09.037333 18870 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260923 22:16:09.037361 18870 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260923 22:16:09.038403 18870 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260923 22:16:09.038434 18870 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260923 22:16:09.038460 18870 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.1.135.195
I20260923 22:16:09.039783 18870 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.1.135.212:43605
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskRiQe_a/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790201539513534-1567-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskRiQe_a/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790201539513534-1567-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskRiQe_a/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790201539513534-1567-0/krb5kdc/kudu_127.1.135.195.keytab
--principal=kudu/127.1.135.195
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.1.135.195:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskRiQe_a/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790201539513534-1567-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.1.135.195
--webserver_port=0
--tserver_master_addrs=127.1.135.254:38385
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.18870
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.1.135.195
--log_dir=/tmp/dist-test-taskRiQe_a/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790201539513534-1567-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 23 Sep 2026 20:53:37 UTC on ccycloud.quasar-rkviyx.root.comops.site
build id 4668083
I20260923 22:16:09.039945 18870 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260923 22:16:09.040180 18870 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260923 22:16:09.044935 18876 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260923 22:16:09.044942 18875 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260923 22:16:09.044986 18878 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260923 22:16:09.044972 18870 server_base.cc:1034] running on GCE node
I20260923 22:16:09.045238 18870 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260923 22:16:09.045490 18870 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260923 22:16:09.046654 18870 hybrid_clock.cc:648] HybridClock initialized: now 1790201769046650 us; error 35 us; skew 500 ppm
Sep 23 22:16:09 dist-test-slave-s2fj krb5kdc[18495](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790201769, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.135.195@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260923 22:16:09.048972 18870 init.cc:377] Logged in from keytab as kudu/127.1.135.195@KRBTEST.COM (short username kudu)
I20260923 22:16:09.049372 18870 webserver.cc:460] Webserver started at http://127.1.135.195:45475/ using document root <none> and password file <none>
I20260923 22:16:09.049593 18870 fs_manager.cc:359] Metadata directory not provided
I20260923 22:16:09.049661 18870 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260923 22:16:09.049806 18870 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260923 22:16:09.050788 18870 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskRiQe_a/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790201539513534-1567-0/minicluster-data/ts-2/data/instance:
uuid: "b68584186a78410aa48ab89ef2139d35"
format_stamp: "Formatted at 2026-09-23 22:16:09 on dist-test-slave-s2fj"
server_key: "acb35f81fc05273a66f5112337606aca"
server_key_iv: "79693b67ec4752eea6e5e81ee0d186c9"
server_key_version: "encryptionkey@0"
I20260923 22:16:09.051132 18870 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskRiQe_a/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790201539513534-1567-0/minicluster-data/ts-2/wal/instance:
uuid: "b68584186a78410aa48ab89ef2139d35"
format_stamp: "Formatted at 2026-09-23 22:16:09 on dist-test-slave-s2fj"
server_key: "acb35f81fc05273a66f5112337606aca"
server_key_iv: "79693b67ec4752eea6e5e81ee0d186c9"
server_key_version: "encryptionkey@0"
I20260923 22:16:09.052425 18870 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.003s
I20260923 22:16:09.053107 18885 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260923 22:16:09.053267 18870 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.001s
I20260923 22:16:09.053366 18870 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskRiQe_a/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790201539513534-1567-0/minicluster-data/ts-2/data,/tmp/dist-test-taskRiQe_a/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790201539513534-1567-0/minicluster-data/ts-2/wal
uuid: "b68584186a78410aa48ab89ef2139d35"
format_stamp: "Formatted at 2026-09-23 22:16:09 on dist-test-slave-s2fj"
server_key: "acb35f81fc05273a66f5112337606aca"
server_key_iv: "79693b67ec4752eea6e5e81ee0d186c9"
server_key_version: "encryptionkey@0"
I20260923 22:16:09.053435 18870 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskRiQe_a/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790201539513534-1567-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskRiQe_a/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790201539513534-1567-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskRiQe_a/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790201539513534-1567-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260923 22:16:09.067516 18870 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260923 22:16:09.069235 18870 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260923 22:16:09.069391 18870 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260923 22:16:09.069597 18870 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260923 22:16:09.069871 18870 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260923 22:16:09.069926 18870 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260923 22:16:09.069965 18870 ts_tablet_manager.cc:610] Registered 0 tablets
I20260923 22:16:09.069993 18870 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260923 22:16:09.075632 18870 rpc_server.cc:307] RPC server started. Bound to: 127.1.135.195:36141
I20260923 22:16:09.075682 18998 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.1.135.195:36141 every 8 connection(s)
I20260923 22:16:09.076001 18870 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskRiQe_a/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790201539513534-1567-0/minicluster-data/ts-2/data/info.pb
I20260923 22:16:09.081378 1567 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskRiQe_a/build/release/bin/kudu as pid 18870
I20260923 22:16:09.081465 1567 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskRiQe_a/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790201539513534-1567-0/minicluster-data/ts-2/wal/instance
Sep 23 22:16:09 dist-test-slave-s2fj krb5kdc[18495](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790201769, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.135.195@KRBTEST.COM for kudu/127.1.135.254@KRBTEST.COM
I20260923 22:16:09.081686 1567 external_mini_cluster.cc:1428] Setting key 869975abd62f0d104cdf3b091d4a40e0
I20260923 22:16:09.084692 18579 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 22:16:09.076660 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.135.195:51599 (local address 127.1.135.254:38385)
0923 22:16:09.076702 (+ 42us) server_negotiation.cc:206] Beginning negotiation
0923 22:16:09.076704 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0923 22:16:09.076920 (+ 216us) server_negotiation.cc:407] Connection header received
0923 22:16:09.077367 (+ 447us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 22:16:09.077368 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 22:16:09.077376 (+ 8us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 22:16:09.077432 (+ 56us) server_negotiation.cc:226] Negotiated authn=SASL
0923 22:16:09.078297 (+ 865us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 22:16:09.078757 (+ 460us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 22:16:09.079677 (+ 920us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 22:16:09.079853 (+ 176us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 22:16:09.079902 (+ 49us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 22:16:09.082081 (+ 2179us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0923 22:16:09.082083 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0923 22:16:09.082083 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0923 22:16:09.082091 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0923 22:16:09.083531 (+ 1440us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0923 22:16:09.083892 (+ 361us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0923 22:16:09.083893 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0923 22:16:09.083893 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0923 22:16:09.083940 (+ 47us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0923 22:16:09.084144 (+ 204us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0923 22:16:09.084145 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0923 22:16:09.084145 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0923 22:16:09.084285 (+ 140us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0923 22:16:09.084336 (+ 51us) server_negotiation.cc:1035] Waiting for connection context
0923 22:16:09.084567 (+ 231us) server_negotiation.cc:299] Negotiation successful
0923 22:16:09.084630 (+ 63us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":19}
I20260923 22:16:09.084849 19001 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 22:16:09.076634 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.135.254:38385 (local address 127.1.135.195:51599)
0923 22:16:09.076797 (+ 163us) negotiation.cc:107] Waiting for socket to connect
0923 22:16:09.076837 (+ 40us) client_negotiation.cc:174] Beginning negotiation
0923 22:16:09.077284 (+ 447us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0923 22:16:09.077500 (+ 216us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0923 22:16:09.077503 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0923 22:16:09.078057 (+ 554us) client_negotiation.cc:189] Negotiated authn=SASL
0923 22:16:09.078227 (+ 170us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 22:16:09.078231 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 22:16:09.078830 (+ 599us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 22:16:09.078831 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 22:16:09.079606 (+ 775us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 22:16:09.079608 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 22:16:09.079923 (+ 315us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 22:16:09.079923 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 22:16:09.080014 (+ 91us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 22:16:09.080558 (+ 544us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0923 22:16:09.080575 (+ 17us) client_negotiation.cc:656] calling sasl_client_start()
0923 22:16:09.081989 (+ 1414us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0923 22:16:09.083601 (+ 1612us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0923 22:16:09.083606 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0923 22:16:09.083607 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0923 22:16:09.083826 (+ 219us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0923 22:16:09.083981 (+ 155us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0923 22:16:09.083981 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0923 22:16:09.083981 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0923 22:16:09.084057 (+ 76us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0923 22:16:09.084342 (+ 285us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0923 22:16:09.084352 (+ 10us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0923 22:16:09.084457 (+ 105us) client_negotiation.cc:769] Sending connection context
0923 22:16:09.084536 (+ 79us) client_negotiation.cc:240] Negotiation successful
0923 22:16:09.084646 (+ 110us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":124,"thread_start_us":89,"threads_started":1}
I20260923 22:16:09.085258 18999 heartbeater.cc:347] Connected to a master server at 127.1.135.254:38385
I20260923 22:16:09.085350 18999 heartbeater.cc:464] Registering TS with master...
I20260923 22:16:09.085534 18999 heartbeater.cc:511] Master 127.1.135.254:38385 requested a full tablet report, sending...
I20260923 22:16:09.085879 18540 ts_manager.cc:194] Registered new tserver with Master: b68584186a78410aa48ab89ef2139d35 (127.1.135.195:36141)
I20260923 22:16:09.086231 18540 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.1.135.195@KRBTEST.COM'} at 127.1.135.195:51599
I20260923 22:16:09.093529 1567 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260923 22:16:09.100697 18579 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 22:16:09.094787 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38008 (local address 127.1.135.254:38385)
0923 22:16:09.094850 (+ 63us) server_negotiation.cc:206] Beginning negotiation
0923 22:16:09.094860 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0923 22:16:09.094938 (+ 78us) server_negotiation.cc:407] Connection header received
0923 22:16:09.094975 (+ 37us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 22:16:09.094976 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 22:16:09.094984 (+ 8us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 22:16:09.095031 (+ 47us) server_negotiation.cc:226] Negotiated authn=SASL
0923 22:16:09.095546 (+ 515us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 22:16:09.096016 (+ 470us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 22:16:09.096821 (+ 805us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 22:16:09.096995 (+ 174us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 22:16:09.097042 (+ 47us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 22:16:09.098198 (+ 1156us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0923 22:16:09.098199 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0923 22:16:09.098199 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0923 22:16:09.098204 (+ 5us) server_negotiation.cc:922] calling sasl_server_start()
0923 22:16:09.099619 (+ 1415us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0923 22:16:09.099923 (+ 304us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0923 22:16:09.099924 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0923 22:16:09.099924 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0923 22:16:09.099969 (+ 45us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0923 22:16:09.100162 (+ 193us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0923 22:16:09.100163 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0923 22:16:09.100163 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0923 22:16:09.100302 (+ 139us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0923 22:16:09.100427 (+ 125us) server_negotiation.cc:1035] Waiting for connection context
0923 22:16:09.100529 (+ 102us) server_negotiation.cc:299] Negotiation successful
0923 22:16:09.100603 (+ 74us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":40}
I20260923 22:16:09.102339 18540 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:38008:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260923 22:16:09.103818 18540 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260923 22:16:09.109889 19011 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 22:16:09.105908 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.135.194:45839 (local address 127.0.0.1:34476)
0923 22:16:09.106219 (+ 311us) negotiation.cc:107] Waiting for socket to connect
0923 22:16:09.106228 (+ 9us) client_negotiation.cc:174] Beginning negotiation
0923 22:16:09.106274 (+ 46us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0923 22:16:09.106630 (+ 356us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0923 22:16:09.106630 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0923 22:16:09.106632 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0923 22:16:09.106666 (+ 34us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 22:16:09.106667 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 22:16:09.107575 (+ 908us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 22:16:09.107576 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 22:16:09.108810 (+ 1234us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 22:16:09.108813 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 22:16:09.109572 (+ 759us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 22:16:09.109573 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 22:16:09.109649 (+ 76us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 22:16:09.109662 (+ 13us) client_negotiation.cc:769] Sending connection context
0923 22:16:09.109759 (+ 97us) client_negotiation.cc:240] Negotiation successful
0923 22:16:09.109813 (+ 54us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":301,"mutex_wait_us":27,"thread_start_us":30,"threads_started":1}
I20260923 22:16:09.109905 19012 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 22:16:09.106081 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34476 (local address 127.1.135.194:45839)
0923 22:16:09.106214 (+ 133us) server_negotiation.cc:206] Beginning negotiation
0923 22:16:09.106218 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0923 22:16:09.106425 (+ 207us) server_negotiation.cc:407] Connection header received
0923 22:16:09.106460 (+ 35us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 22:16:09.106462 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 22:16:09.106473 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 22:16:09.106634 (+ 161us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0923 22:16:09.106725 (+ 91us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 22:16:09.107457 (+ 732us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 22:16:09.108913 (+ 1456us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 22:16:09.109472 (+ 559us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 22:16:09.109569 (+ 97us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 22:16:09.109630 (+ 61us) server_negotiation.cc:1035] Waiting for connection context
0923 22:16:09.109777 (+ 147us) server_negotiation.cc:299] Negotiation successful
0923 22:16:09.109821 (+ 44us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":103,"thread_start_us":71,"threads_started":1}
I20260923 22:16:09.110100 19008 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 22:16:09.105734 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.135.195:36141 (local address 127.0.0.1:56970)
0923 22:16:09.105873 (+ 139us) negotiation.cc:107] Waiting for socket to connect
0923 22:16:09.105908 (+ 35us) client_negotiation.cc:174] Beginning negotiation
0923 22:16:09.105971 (+ 63us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0923 22:16:09.106248 (+ 277us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0923 22:16:09.106251 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0923 22:16:09.106252 (+ 1us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0923 22:16:09.106342 (+ 90us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 22:16:09.106344 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 22:16:09.107861 (+ 1517us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 22:16:09.107862 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 22:16:09.108946 (+ 1084us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 22:16:09.108948 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 22:16:09.109867 (+ 919us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 22:16:09.109868 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 22:16:09.109927 (+ 59us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 22:16:09.109935 (+ 8us) client_negotiation.cc:769] Sending connection context
0923 22:16:09.109965 (+ 30us) client_negotiation.cc:240] Negotiation successful
0923 22:16:09.110011 (+ 46us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":118,"mutex_wait_us":7,"thread_start_us":47,"threads_started":1}
I20260923 22:16:09.110302 19010 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 22:16:09.105839 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45458 (local address 127.1.135.193:41015)
0923 22:16:09.106007 (+ 168us) server_negotiation.cc:206] Beginning negotiation
0923 22:16:09.106012 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0923 22:16:09.106082 (+ 70us) server_negotiation.cc:407] Connection header received
0923 22:16:09.106160 (+ 78us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 22:16:09.106162 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 22:16:09.106178 (+ 16us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 22:16:09.106270 (+ 92us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0923 22:16:09.106408 (+ 138us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 22:16:09.107751 (+ 1343us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 22:16:09.109098 (+ 1347us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 22:16:09.109755 (+ 657us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 22:16:09.110054 (+ 299us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 22:16:09.110129 (+ 75us) server_negotiation.cc:1035] Waiting for connection context
0923 22:16:09.110158 (+ 29us) server_negotiation.cc:299] Negotiation successful
0923 22:16:09.110206 (+ 48us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":130,"thread_start_us":82,"threads_started":1}
I20260923 22:16:09.110486 19007 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 22:16:09.105737 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.135.193:41015 (local address 127.0.0.1:45458)
0923 22:16:09.106012 (+ 275us) negotiation.cc:107] Waiting for socket to connect
0923 22:16:09.106034 (+ 22us) client_negotiation.cc:174] Beginning negotiation
0923 22:16:09.106091 (+ 57us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0923 22:16:09.106274 (+ 183us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0923 22:16:09.106275 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0923 22:16:09.106295 (+ 20us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0923 22:16:09.106356 (+ 61us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 22:16:09.106357 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 22:16:09.108139 (+ 1782us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 22:16:09.108140 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 22:16:09.109014 (+ 874us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 22:16:09.109016 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 22:16:09.109861 (+ 845us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 22:16:09.109862 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 22:16:09.109927 (+ 65us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 22:16:09.109933 (+ 6us) client_negotiation.cc:769] Sending connection context
0923 22:16:09.109965 (+ 32us) client_negotiation.cc:240] Negotiation successful
0923 22:16:09.110009 (+ 44us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":263,"mutex_wait_us":2,"thread_start_us":42,"threads_started":1}
I20260923 22:16:09.110610 18799 tablet_service.cc:1467] Processing CreateTablet for tablet 31bfeda7218446908902d4b26aa8b6e1 (DEFAULT_TABLE table=test-table [id=2be252c2650c476b9886da7e474814b3]), partition=RANGE (key) PARTITION UNBOUNDED
I20260923 22:16:09.110807 19009 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 22:16:09.105795 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56970 (local address 127.1.135.195:36141)
0923 22:16:09.106097 (+ 302us) server_negotiation.cc:206] Beginning negotiation
0923 22:16:09.106100 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0923 22:16:09.106107 (+ 7us) server_negotiation.cc:407] Connection header received
0923 22:16:09.106138 (+ 31us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 22:16:09.106140 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 22:16:09.106154 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 22:16:09.106426 (+ 272us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0923 22:16:09.106481 (+ 55us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 22:16:09.107749 (+ 1268us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 22:16:09.109031 (+ 1282us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 22:16:09.109755 (+ 724us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 22:16:09.110616 (+ 861us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 22:16:09.110683 (+ 67us) server_negotiation.cc:1035] Waiting for connection context
0923 22:16:09.110703 (+ 20us) server_negotiation.cc:299] Negotiation successful
0923 22:16:09.110733 (+ 30us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":277,"thread_start_us":85,"threads_started":1}
I20260923 22:16:09.110980 18799 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 31bfeda7218446908902d4b26aa8b6e1. 1 dirs total, 0 dirs full, 0 dirs failed
I20260923 22:16:09.111163 18665 tablet_service.cc:1467] Processing CreateTablet for tablet 31bfeda7218446908902d4b26aa8b6e1 (DEFAULT_TABLE table=test-table [id=2be252c2650c476b9886da7e474814b3]), partition=RANGE (key) PARTITION UNBOUNDED
I20260923 22:16:09.111474 18665 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 31bfeda7218446908902d4b26aa8b6e1. 1 dirs total, 0 dirs full, 0 dirs failed
I20260923 22:16:09.114256 19013 tablet_bootstrap.cc:492] T 31bfeda7218446908902d4b26aa8b6e1 P 4cecc19a08b0495282c8939a7e0b5da6: Bootstrap starting.
I20260923 22:16:09.114578 19014 tablet_bootstrap.cc:492] T 31bfeda7218446908902d4b26aa8b6e1 P 1c89669cd4df4d2a9d7f4c80f199a124: Bootstrap starting.
I20260923 22:16:09.115008 19013 tablet_bootstrap.cc:654] T 31bfeda7218446908902d4b26aa8b6e1 P 4cecc19a08b0495282c8939a7e0b5da6: Neither blocks nor log segments found. Creating new log.
I20260923 22:16:09.115266 19014 tablet_bootstrap.cc:654] T 31bfeda7218446908902d4b26aa8b6e1 P 1c89669cd4df4d2a9d7f4c80f199a124: Neither blocks nor log segments found. Creating new log.
I20260923 22:16:09.115159 18933 tablet_service.cc:1467] Processing CreateTablet for tablet 31bfeda7218446908902d4b26aa8b6e1 (DEFAULT_TABLE table=test-table [id=2be252c2650c476b9886da7e474814b3]), partition=RANGE (key) PARTITION UNBOUNDED
I20260923 22:16:09.115511 18933 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 31bfeda7218446908902d4b26aa8b6e1. 1 dirs total, 0 dirs full, 0 dirs failed
I20260923 22:16:09.115592 19013 log.cc:824] T 31bfeda7218446908902d4b26aa8b6e1 P 4cecc19a08b0495282c8939a7e0b5da6: Log is configured to *not* fsync() on all Append() calls
I20260923 22:16:09.115758 19014 log.cc:824] T 31bfeda7218446908902d4b26aa8b6e1 P 1c89669cd4df4d2a9d7f4c80f199a124: Log is configured to *not* fsync() on all Append() calls
I20260923 22:16:09.116322 19013 tablet_bootstrap.cc:492] T 31bfeda7218446908902d4b26aa8b6e1 P 4cecc19a08b0495282c8939a7e0b5da6: No bootstrap required, opened a new log
I20260923 22:16:09.116398 19013 ts_tablet_manager.cc:1397] T 31bfeda7218446908902d4b26aa8b6e1 P 4cecc19a08b0495282c8939a7e0b5da6: Time spent bootstrapping tablet: real 0.002s user 0.002s sys 0.000s
I20260923 22:16:09.118263 19013 raft_consensus.cc:348] T 31bfeda7218446908902d4b26aa8b6e1 P 4cecc19a08b0495282c8939a7e0b5da6 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b68584186a78410aa48ab89ef2139d35" member_type: VOTER last_known_addr { host: "127.1.135.195" port: 36141 } } peers { permanent_uuid: "1c89669cd4df4d2a9d7f4c80f199a124" member_type: VOTER last_known_addr { host: "127.1.135.193" port: 41015 } } peers { permanent_uuid: "4cecc19a08b0495282c8939a7e0b5da6" member_type: VOTER last_known_addr { host: "127.1.135.194" port: 45839 } }
I20260923 22:16:09.118465 19013 raft_consensus.cc:374] T 31bfeda7218446908902d4b26aa8b6e1 P 4cecc19a08b0495282c8939a7e0b5da6 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260923 22:16:09.118512 19013 raft_consensus.cc:729] T 31bfeda7218446908902d4b26aa8b6e1 P 4cecc19a08b0495282c8939a7e0b5da6 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 4cecc19a08b0495282c8939a7e0b5da6, State: Initialized, Role: FOLLOWER
I20260923 22:16:09.118589 19017 tablet_bootstrap.cc:492] T 31bfeda7218446908902d4b26aa8b6e1 P b68584186a78410aa48ab89ef2139d35: Bootstrap starting.
I20260923 22:16:09.118628 19013 consensus_queue.cc:260] T 31bfeda7218446908902d4b26aa8b6e1 P 4cecc19a08b0495282c8939a7e0b5da6 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b68584186a78410aa48ab89ef2139d35" member_type: VOTER last_known_addr { host: "127.1.135.195" port: 36141 } } peers { permanent_uuid: "1c89669cd4df4d2a9d7f4c80f199a124" member_type: VOTER last_known_addr { host: "127.1.135.193" port: 41015 } } peers { permanent_uuid: "4cecc19a08b0495282c8939a7e0b5da6" member_type: VOTER last_known_addr { host: "127.1.135.194" port: 45839 } }
I20260923 22:16:09.118898 19013 ts_tablet_manager.cc:1428] T 31bfeda7218446908902d4b26aa8b6e1 P 4cecc19a08b0495282c8939a7e0b5da6: Time spent starting tablet: real 0.002s user 0.000s sys 0.003s
I20260923 22:16:09.118988 18865 heartbeater.cc:503] Master 127.1.135.254:38385 was elected leader, sending a full tablet report...
I20260923 22:16:09.119216 19017 tablet_bootstrap.cc:654] T 31bfeda7218446908902d4b26aa8b6e1 P b68584186a78410aa48ab89ef2139d35: Neither blocks nor log segments found. Creating new log.
I20260923 22:16:09.119557 19014 tablet_bootstrap.cc:492] T 31bfeda7218446908902d4b26aa8b6e1 P 1c89669cd4df4d2a9d7f4c80f199a124: No bootstrap required, opened a new log
I20260923 22:16:09.119638 19014 ts_tablet_manager.cc:1397] T 31bfeda7218446908902d4b26aa8b6e1 P 1c89669cd4df4d2a9d7f4c80f199a124: Time spent bootstrapping tablet: real 0.005s user 0.001s sys 0.000s
I20260923 22:16:09.119731 19017 log.cc:824] T 31bfeda7218446908902d4b26aa8b6e1 P b68584186a78410aa48ab89ef2139d35: Log is configured to *not* fsync() on all Append() calls
I20260923 22:16:09.120419 19017 tablet_bootstrap.cc:492] T 31bfeda7218446908902d4b26aa8b6e1 P b68584186a78410aa48ab89ef2139d35: No bootstrap required, opened a new log
I20260923 22:16:09.120472 19017 ts_tablet_manager.cc:1397] T 31bfeda7218446908902d4b26aa8b6e1 P b68584186a78410aa48ab89ef2139d35: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20260923 22:16:09.121218 19014 raft_consensus.cc:348] T 31bfeda7218446908902d4b26aa8b6e1 P 1c89669cd4df4d2a9d7f4c80f199a124 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b68584186a78410aa48ab89ef2139d35" member_type: VOTER last_known_addr { host: "127.1.135.195" port: 36141 } } peers { permanent_uuid: "1c89669cd4df4d2a9d7f4c80f199a124" member_type: VOTER last_known_addr { host: "127.1.135.193" port: 41015 } } peers { permanent_uuid: "4cecc19a08b0495282c8939a7e0b5da6" member_type: VOTER last_known_addr { host: "127.1.135.194" port: 45839 } }
I20260923 22:16:09.121371 19014 raft_consensus.cc:374] T 31bfeda7218446908902d4b26aa8b6e1 P 1c89669cd4df4d2a9d7f4c80f199a124 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260923 22:16:09.121416 19014 raft_consensus.cc:729] T 31bfeda7218446908902d4b26aa8b6e1 P 1c89669cd4df4d2a9d7f4c80f199a124 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 1c89669cd4df4d2a9d7f4c80f199a124, State: Initialized, Role: FOLLOWER
I20260923 22:16:09.121520 19014 consensus_queue.cc:260] T 31bfeda7218446908902d4b26aa8b6e1 P 1c89669cd4df4d2a9d7f4c80f199a124 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b68584186a78410aa48ab89ef2139d35" member_type: VOTER last_known_addr { host: "127.1.135.195" port: 36141 } } peers { permanent_uuid: "1c89669cd4df4d2a9d7f4c80f199a124" member_type: VOTER last_known_addr { host: "127.1.135.193" port: 41015 } } peers { permanent_uuid: "4cecc19a08b0495282c8939a7e0b5da6" member_type: VOTER last_known_addr { host: "127.1.135.194" port: 45839 } }
I20260923 22:16:09.121784 19014 ts_tablet_manager.cc:1428] T 31bfeda7218446908902d4b26aa8b6e1 P 1c89669cd4df4d2a9d7f4c80f199a124: Time spent starting tablet: real 0.002s user 0.002s sys 0.000s
I20260923 22:16:09.121825 18731 heartbeater.cc:503] Master 127.1.135.254:38385 was elected leader, sending a full tablet report...
I20260923 22:16:09.121861 19017 raft_consensus.cc:348] T 31bfeda7218446908902d4b26aa8b6e1 P b68584186a78410aa48ab89ef2139d35 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b68584186a78410aa48ab89ef2139d35" member_type: VOTER last_known_addr { host: "127.1.135.195" port: 36141 } } peers { permanent_uuid: "1c89669cd4df4d2a9d7f4c80f199a124" member_type: VOTER last_known_addr { host: "127.1.135.193" port: 41015 } } peers { permanent_uuid: "4cecc19a08b0495282c8939a7e0b5da6" member_type: VOTER last_known_addr { host: "127.1.135.194" port: 45839 } }
I20260923 22:16:09.122011 19017 raft_consensus.cc:374] T 31bfeda7218446908902d4b26aa8b6e1 P b68584186a78410aa48ab89ef2139d35 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260923 22:16:09.122047 19017 raft_consensus.cc:729] T 31bfeda7218446908902d4b26aa8b6e1 P b68584186a78410aa48ab89ef2139d35 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b68584186a78410aa48ab89ef2139d35, State: Initialized, Role: FOLLOWER
I20260923 22:16:09.122148 19017 consensus_queue.cc:260] T 31bfeda7218446908902d4b26aa8b6e1 P b68584186a78410aa48ab89ef2139d35 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b68584186a78410aa48ab89ef2139d35" member_type: VOTER last_known_addr { host: "127.1.135.195" port: 36141 } } peers { permanent_uuid: "1c89669cd4df4d2a9d7f4c80f199a124" member_type: VOTER last_known_addr { host: "127.1.135.193" port: 41015 } } peers { permanent_uuid: "4cecc19a08b0495282c8939a7e0b5da6" member_type: VOTER last_known_addr { host: "127.1.135.194" port: 45839 } }
I20260923 22:16:09.122429 19017 ts_tablet_manager.cc:1428] T 31bfeda7218446908902d4b26aa8b6e1 P b68584186a78410aa48ab89ef2139d35: Time spent starting tablet: real 0.002s user 0.002s sys 0.000s
I20260923 22:16:09.122512 18999 heartbeater.cc:503] Master 127.1.135.254:38385 was elected leader, sending a full tablet report...
W20260923 22:16:09.179675 18866 tablet.cc:2369] T 31bfeda7218446908902d4b26aa8b6e1 P 4cecc19a08b0495282c8939a7e0b5da6: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260923 22:16:09.211750 19021 raft_consensus.cc:482] T 31bfeda7218446908902d4b26aa8b6e1 P b68584186a78410aa48ab89ef2139d35 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260923 22:16:09.211877 19021 raft_consensus.cc:504] T 31bfeda7218446908902d4b26aa8b6e1 P b68584186a78410aa48ab89ef2139d35 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b68584186a78410aa48ab89ef2139d35" member_type: VOTER last_known_addr { host: "127.1.135.195" port: 36141 } } peers { permanent_uuid: "1c89669cd4df4d2a9d7f4c80f199a124" member_type: VOTER last_known_addr { host: "127.1.135.193" port: 41015 } } peers { permanent_uuid: "4cecc19a08b0495282c8939a7e0b5da6" member_type: VOTER last_known_addr { host: "127.1.135.194" port: 45839 } }
I20260923 22:16:09.212288 19021 leader_election.cc:288] T 31bfeda7218446908902d4b26aa8b6e1 P b68584186a78410aa48ab89ef2139d35 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 1c89669cd4df4d2a9d7f4c80f199a124 (127.1.135.193:41015), 4cecc19a08b0495282c8939a7e0b5da6 (127.1.135.194:45839)
I20260923 22:16:09.216022 19010 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 22:16:09.212541 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.135.195:58053 (local address 127.1.135.193:41015)
0923 22:16:09.212608 (+ 67us) server_negotiation.cc:206] Beginning negotiation
0923 22:16:09.212611 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0923 22:16:09.212638 (+ 27us) server_negotiation.cc:407] Connection header received
0923 22:16:09.212666 (+ 28us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 22:16:09.212667 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 22:16:09.212678 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 22:16:09.212727 (+ 49us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0923 22:16:09.212909 (+ 182us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 22:16:09.213687 (+ 778us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 22:16:09.215076 (+ 1389us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 22:16:09.215651 (+ 575us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 22:16:09.215726 (+ 75us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 22:16:09.215734 (+ 8us) server_negotiation.cc:1035] Waiting for connection context
0923 22:16:09.215912 (+ 178us) server_negotiation.cc:299] Negotiation successful
0923 22:16:09.215953 (+ 41us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":41}
I20260923 22:16:09.216022 19012 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 22:16:09.212662 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.135.195:35955 (local address 127.1.135.194:45839)
0923 22:16:09.212725 (+ 63us) server_negotiation.cc:206] Beginning negotiation
0923 22:16:09.212727 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0923 22:16:09.212807 (+ 80us) server_negotiation.cc:407] Connection header received
0923 22:16:09.212903 (+ 96us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 22:16:09.212904 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 22:16:09.212913 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 22:16:09.212963 (+ 50us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0923 22:16:09.213091 (+ 128us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 22:16:09.213764 (+ 673us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 22:16:09.215076 (+ 1312us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 22:16:09.215651 (+ 575us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 22:16:09.215726 (+ 75us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 22:16:09.215734 (+ 8us) server_negotiation.cc:1035] Waiting for connection context
0923 22:16:09.215918 (+ 184us) server_negotiation.cc:299] Negotiation successful
0923 22:16:09.215954 (+ 36us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":38}
I20260923 22:16:09.216045 19001 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 22:16:09.212466 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.135.193:41015 (local address 127.1.135.195:58053)
0923 22:16:09.212520 (+ 54us) negotiation.cc:107] Waiting for socket to connect
0923 22:16:09.212532 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0923 22:16:09.212603 (+ 71us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0923 22:16:09.212739 (+ 136us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0923 22:16:09.212740 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0923 22:16:09.212743 (+ 3us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0923 22:16:09.212815 (+ 72us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 22:16:09.212817 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 22:16:09.213760 (+ 943us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 22:16:09.213761 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 22:16:09.214990 (+ 1229us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 22:16:09.214992 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 22:16:09.215732 (+ 740us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 22:16:09.215733 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 22:16:09.215820 (+ 87us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 22:16:09.215822 (+ 2us) client_negotiation.cc:769] Sending connection context
0923 22:16:09.215882 (+ 60us) client_negotiation.cc:240] Negotiation successful
0923 22:16:09.215932 (+ 50us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":28,"mutex_wait_us":21}
I20260923 22:16:09.216054 19022 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 22:16:09.212575 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.135.194:45839 (local address 127.1.135.195:35955)
0923 22:16:09.212736 (+ 161us) negotiation.cc:107] Waiting for socket to connect
0923 22:16:09.212746 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0923 22:16:09.212804 (+ 58us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0923 22:16:09.212977 (+ 173us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0923 22:16:09.212978 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0923 22:16:09.212979 (+ 1us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0923 22:16:09.213030 (+ 51us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 22:16:09.213032 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 22:16:09.213830 (+ 798us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 22:16:09.213830 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 22:16:09.214990 (+ 1160us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 22:16:09.214992 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 22:16:09.215733 (+ 741us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 22:16:09.215733 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 22:16:09.215816 (+ 83us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 22:16:09.215821 (+ 5us) client_negotiation.cc:769] Sending connection context
0923 22:16:09.215907 (+ 86us) client_negotiation.cc:240] Negotiation successful
0923 22:16:09.215936 (+ 29us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":143,"mutex_wait_us":11,"thread_start_us":102,"threads_started":1}
I20260923 22:16:09.216419 18819 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "31bfeda7218446908902d4b26aa8b6e1" candidate_uuid: "b68584186a78410aa48ab89ef2139d35" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4cecc19a08b0495282c8939a7e0b5da6" is_pre_election: true
I20260923 22:16:09.216445 18685 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "31bfeda7218446908902d4b26aa8b6e1" candidate_uuid: "b68584186a78410aa48ab89ef2139d35" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1c89669cd4df4d2a9d7f4c80f199a124" is_pre_election: true
I20260923 22:16:09.216568 18685 raft_consensus.cc:2454] T 31bfeda7218446908902d4b26aa8b6e1 P 1c89669cd4df4d2a9d7f4c80f199a124 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b68584186a78410aa48ab89ef2139d35 in term 0.
I20260923 22:16:09.216573 18819 raft_consensus.cc:2454] T 31bfeda7218446908902d4b26aa8b6e1 P 4cecc19a08b0495282c8939a7e0b5da6 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b68584186a78410aa48ab89ef2139d35 in term 0.
I20260923 22:16:09.216809 18886 leader_election.cc:302] T 31bfeda7218446908902d4b26aa8b6e1 P b68584186a78410aa48ab89ef2139d35 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1c89669cd4df4d2a9d7f4c80f199a124, b68584186a78410aa48ab89ef2139d35; no voters:
I20260923 22:16:09.216928 19021 raft_consensus.cc:2781] T 31bfeda7218446908902d4b26aa8b6e1 P b68584186a78410aa48ab89ef2139d35 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260923 22:16:09.216980 19021 raft_consensus.cc:482] T 31bfeda7218446908902d4b26aa8b6e1 P b68584186a78410aa48ab89ef2139d35 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260923 22:16:09.217003 19021 raft_consensus.cc:3037] T 31bfeda7218446908902d4b26aa8b6e1 P b68584186a78410aa48ab89ef2139d35 [term 0 FOLLOWER]: Advancing to term 1
I20260923 22:16:09.217640 19021 raft_consensus.cc:504] T 31bfeda7218446908902d4b26aa8b6e1 P b68584186a78410aa48ab89ef2139d35 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b68584186a78410aa48ab89ef2139d35" member_type: VOTER last_known_addr { host: "127.1.135.195" port: 36141 } } peers { permanent_uuid: "1c89669cd4df4d2a9d7f4c80f199a124" member_type: VOTER last_known_addr { host: "127.1.135.193" port: 41015 } } peers { permanent_uuid: "4cecc19a08b0495282c8939a7e0b5da6" member_type: VOTER last_known_addr { host: "127.1.135.194" port: 45839 } }
I20260923 22:16:09.217765 19021 leader_election.cc:288] T 31bfeda7218446908902d4b26aa8b6e1 P b68584186a78410aa48ab89ef2139d35 [CANDIDATE]: Term 1 election: Requested vote from peers 1c89669cd4df4d2a9d7f4c80f199a124 (127.1.135.193:41015), 4cecc19a08b0495282c8939a7e0b5da6 (127.1.135.194:45839)
I20260923 22:16:09.217908 18685 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "31bfeda7218446908902d4b26aa8b6e1" candidate_uuid: "b68584186a78410aa48ab89ef2139d35" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1c89669cd4df4d2a9d7f4c80f199a124"
I20260923 22:16:09.217999 18685 raft_consensus.cc:3037] T 31bfeda7218446908902d4b26aa8b6e1 P 1c89669cd4df4d2a9d7f4c80f199a124 [term 0 FOLLOWER]: Advancing to term 1
I20260923 22:16:09.217984 18819 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "31bfeda7218446908902d4b26aa8b6e1" candidate_uuid: "b68584186a78410aa48ab89ef2139d35" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4cecc19a08b0495282c8939a7e0b5da6"
I20260923 22:16:09.218070 18819 raft_consensus.cc:3037] T 31bfeda7218446908902d4b26aa8b6e1 P 4cecc19a08b0495282c8939a7e0b5da6 [term 0 FOLLOWER]: Advancing to term 1
I20260923 22:16:09.218724 18685 raft_consensus.cc:2454] T 31bfeda7218446908902d4b26aa8b6e1 P 1c89669cd4df4d2a9d7f4c80f199a124 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b68584186a78410aa48ab89ef2139d35 in term 1.
I20260923 22:16:09.218787 18819 raft_consensus.cc:2454] T 31bfeda7218446908902d4b26aa8b6e1 P 4cecc19a08b0495282c8939a7e0b5da6 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b68584186a78410aa48ab89ef2139d35 in term 1.
I20260923 22:16:09.218959 18886 leader_election.cc:302] T 31bfeda7218446908902d4b26aa8b6e1 P b68584186a78410aa48ab89ef2139d35 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1c89669cd4df4d2a9d7f4c80f199a124, b68584186a78410aa48ab89ef2139d35; no voters:
I20260923 22:16:09.219049 19021 raft_consensus.cc:2781] T 31bfeda7218446908902d4b26aa8b6e1 P b68584186a78410aa48ab89ef2139d35 [term 1 FOLLOWER]: Leader election won for term 1
I20260923 22:16:09.219208 19021 raft_consensus.cc:686] T 31bfeda7218446908902d4b26aa8b6e1 P b68584186a78410aa48ab89ef2139d35 [term 1 LEADER]: Becoming Leader. State: Replica: b68584186a78410aa48ab89ef2139d35, State: Running, Role: LEADER
I20260923 22:16:09.219308 19021 consensus_queue.cc:237] T 31bfeda7218446908902d4b26aa8b6e1 P b68584186a78410aa48ab89ef2139d35 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b68584186a78410aa48ab89ef2139d35" member_type: VOTER last_known_addr { host: "127.1.135.195" port: 36141 } } peers { permanent_uuid: "1c89669cd4df4d2a9d7f4c80f199a124" member_type: VOTER last_known_addr { host: "127.1.135.193" port: 41015 } } peers { permanent_uuid: "4cecc19a08b0495282c8939a7e0b5da6" member_type: VOTER last_known_addr { host: "127.1.135.194" port: 45839 } }
I20260923 22:16:09.220268 18540 catalog_manager.cc:5495] T 31bfeda7218446908902d4b26aa8b6e1 P b68584186a78410aa48ab89ef2139d35 reported cstate change: term changed from 0 to 1, leader changed from <none> to b68584186a78410aa48ab89ef2139d35 (127.1.135.195). New cstate: current_term: 1 leader_uuid: "b68584186a78410aa48ab89ef2139d35" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b68584186a78410aa48ab89ef2139d35" member_type: VOTER last_known_addr { host: "127.1.135.195" port: 36141 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "1c89669cd4df4d2a9d7f4c80f199a124" member_type: VOTER last_known_addr { host: "127.1.135.193" port: 41015 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "4cecc19a08b0495282c8939a7e0b5da6" member_type: VOTER last_known_addr { host: "127.1.135.194" port: 45839 } health_report { overall_health: UNKNOWN } } }
I20260923 22:16:09.225220 19009 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 22:16:09.223188 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56980 (local address 127.1.135.195:36141)
0923 22:16:09.223266 (+ 78us) server_negotiation.cc:206] Beginning negotiation
0923 22:16:09.223268 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0923 22:16:09.223278 (+ 10us) server_negotiation.cc:407] Connection header received
0923 22:16:09.223305 (+ 27us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 22:16:09.223306 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 22:16:09.223321 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 22:16:09.223382 (+ 61us) server_negotiation.cc:226] Negotiated authn=TOKEN
0923 22:16:09.223499 (+ 117us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 22:16:09.223961 (+ 462us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 22:16:09.224636 (+ 675us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 22:16:09.224821 (+ 185us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 22:16:09.224874 (+ 53us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 22:16:09.224986 (+ 112us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0923 22:16:09.225071 (+ 85us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0923 22:16:09.225127 (+ 56us) server_negotiation.cc:1035] Waiting for connection context
0923 22:16:09.225146 (+ 19us) server_negotiation.cc:299] Negotiation successful
0923 22:16:09.225160 (+ 14us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":21}
I20260923 22:16:09.226814 18819 raft_consensus.cc:1261] T 31bfeda7218446908902d4b26aa8b6e1 P 4cecc19a08b0495282c8939a7e0b5da6 [term 1 FOLLOWER]: Refusing update from remote peer b68584186a78410aa48ab89ef2139d35: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260923 22:16:09.226864 18685 raft_consensus.cc:1261] T 31bfeda7218446908902d4b26aa8b6e1 P 1c89669cd4df4d2a9d7f4c80f199a124 [term 1 FOLLOWER]: Refusing update from remote peer b68584186a78410aa48ab89ef2139d35: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260923 22:16:09.227157 19021 consensus_queue.cc:1035] T 31bfeda7218446908902d4b26aa8b6e1 P b68584186a78410aa48ab89ef2139d35 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4cecc19a08b0495282c8939a7e0b5da6" member_type: VOTER last_known_addr { host: "127.1.135.194" port: 45839 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260923 22:16:09.227272 19023 consensus_queue.cc:1035] T 31bfeda7218446908902d4b26aa8b6e1 P b68584186a78410aa48ab89ef2139d35 [LEADER]: Connected to new peer: Peer: permanent_uuid: "1c89669cd4df4d2a9d7f4c80f199a124" member_type: VOTER last_known_addr { host: "127.1.135.193" port: 41015 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260923 22:16:09.229816 19028 mvcc.cc:205] Tried to move back new op lower bound from 7332666446750744576 to 7332666446723170304. Current Snapshot: MvccSnapshot[applied={T|T < 7332666446750744576}]
I20260923 22:16:09.231127 19027 mvcc.cc:205] Tried to move back new op lower bound from 7332666446750744576 to 7332666446723170304. Current Snapshot: MvccSnapshot[applied={T|T < 7332666446750744576 or (T in {7332666446750744576})}]
I20260923 22:16:09.231343 18539 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:38008:
table { table_name: "test-table" } modify_external_catalogs: true
I20260923 22:16:09.231443 18539 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:38008:
table { table_name: "test-table" } modify_external_catalogs: true
I20260923 22:16:09.232204 18539 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 20d73f4b1d8a4851ba5e5aa4c58c4b82: Sending DeleteTablet for 3 replicas of tablet 31bfeda7218446908902d4b26aa8b6e1
I20260923 22:16:09.232599 18933 tablet_service.cc:1514] Processing DeleteTablet for tablet 31bfeda7218446908902d4b26aa8b6e1 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-23 22:16:09 UTC) from {username='kudu', principal='kudu/127.1.135.254@KRBTEST.COM'} at 127.0.0.1:56970
I20260923 22:16:09.232626 18799 tablet_service.cc:1514] Processing DeleteTablet for tablet 31bfeda7218446908902d4b26aa8b6e1 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-23 22:16:09 UTC) from {username='kudu', principal='kudu/127.1.135.254@KRBTEST.COM'} at 127.0.0.1:34476
I20260923 22:16:09.232638 18665 tablet_service.cc:1514] Processing DeleteTablet for tablet 31bfeda7218446908902d4b26aa8b6e1 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-23 22:16:09 UTC) from {username='kudu', principal='kudu/127.1.135.254@KRBTEST.COM'} at 127.0.0.1:45458
I20260923 22:16:09.232895 1567 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskRiQe_a/build/release/bin/kudu with pid 18602
I20260923 22:16:09.238965 19036 tablet_replica.cc:330] T 31bfeda7218446908902d4b26aa8b6e1 P b68584186a78410aa48ab89ef2139d35: stopping tablet replica
I20260923 22:16:09.239074 19036 raft_consensus.cc:2229] T 31bfeda7218446908902d4b26aa8b6e1 P b68584186a78410aa48ab89ef2139d35 [term 1 LEADER]: Raft consensus shutting down.
I20260923 22:16:09.239195 19036 raft_consensus.cc:2258] T 31bfeda7218446908902d4b26aa8b6e1 P b68584186a78410aa48ab89ef2139d35 [term 1 FOLLOWER]: Raft consensus is shut down!
W20260923 22:16:09.239255 18524 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.1.135.193:41015 (error 108)
I20260923 22:16:09.239336 19037 tablet_replica.cc:330] T 31bfeda7218446908902d4b26aa8b6e1 P 4cecc19a08b0495282c8939a7e0b5da6: stopping tablet replica
I20260923 22:16:09.239432 19037 raft_consensus.cc:2229] T 31bfeda7218446908902d4b26aa8b6e1 P 4cecc19a08b0495282c8939a7e0b5da6 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260923 22:16:09.239427 1567 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskRiQe_a/build/release/bin/kudu with pid 18736
I20260923 22:16:09.239527 19037 raft_consensus.cc:2258] T 31bfeda7218446908902d4b26aa8b6e1 P 4cecc19a08b0495282c8939a7e0b5da6 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260923 22:16:09.239945 19007 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 22:16:09.239835 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.135.193:41015 (local address 127.0.0.1:45474)
0923 22:16:09.239880 (+ 45us) negotiation.cc:107] Waiting for socket to connect
0923 22:16:09.239913 (+ 33us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.135.193:41015: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":29}
W20260923 22:16:09.240159 18524 catalog_manager.cc:4584] TS 1c89669cd4df4d2a9d7f4c80f199a124 (127.1.135.193:41015): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet 31bfeda7218446908902d4b26aa8b6e1: Network error: Client connection negotiation failed: client connection to 127.1.135.193:41015: connect: Connection refused (error 111)
I20260923 22:16:09.240303 19036 ts_tablet_manager.cc:1905] T 31bfeda7218446908902d4b26aa8b6e1 P b68584186a78410aa48ab89ef2139d35: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260923 22:16:09.242350 19036 ts_tablet_manager.cc:1918] T 31bfeda7218446908902d4b26aa8b6e1 P b68584186a78410aa48ab89ef2139d35: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260923 22:16:09.242442 19036 log.cc:1196] T 31bfeda7218446908902d4b26aa8b6e1 P b68584186a78410aa48ab89ef2139d35: Deleting WAL directory at /tmp/dist-test-taskRiQe_a/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790201539513534-1567-0/minicluster-data/ts-2/wal/wals/31bfeda7218446908902d4b26aa8b6e1
I20260923 22:16:09.242695 19036 ts_tablet_manager.cc:1939] T 31bfeda7218446908902d4b26aa8b6e1 P b68584186a78410aa48ab89ef2139d35: Deleting consensus metadata
I20260923 22:16:09.243155 18526 catalog_manager.cc:4841] TS b68584186a78410aa48ab89ef2139d35 (127.1.135.195:36141): tablet 31bfeda7218446908902d4b26aa8b6e1 (table test-table [id=2be252c2650c476b9886da7e474814b3]) successfully deleted
I20260923 22:16:09.245771 19007 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 22:16:09.245659 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.135.194:45839 (local address 127.0.0.1:34488)
0923 22:16:09.245715 (+ 56us) negotiation.cc:107] Waiting for socket to connect
0923 22:16:09.245740 (+ 25us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.135.194:45839: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":36}
I20260923 22:16:09.245880 1567 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskRiQe_a/build/release/bin/kudu with pid 18870
I20260923 22:16:09.250664 1567 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskRiQe_a/build/release/bin/kudu with pid 18508
2026-09-23T22:16:09Z chronyd exiting
[ OK ] EncryptionPolicyTest.TestEncryptionPolicy/0 (1954 ms)
[ RUN ] EncryptionPolicyTest.TestEncryptionPolicy/1
2026-09-23T22:16:09Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-09-23T22:16:09Z Disabled control of system clock
I20260923 22:16:09.277941 1567 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskRiQe_a/build/release/bin/kudu
/tmp/dist-test-taskRiQe_a/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskRiQe_a/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790201539513534-1567-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskRiQe_a/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790201539513534-1567-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskRiQe_a/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790201539513534-1567-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskRiQe_a/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790201539513534-1567-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.1.135.254:43673
--webserver_interface=127.1.135.254
--webserver_port=0
--builtin_ntp_servers=127.1.135.212:41195
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.1.135.254:43673
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
W20260923 22:16:09.359344 19043 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260923 22:16:09.359568 19043 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260923 22:16:09.359589 19043 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260923 22:16:09.360868 19043 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260923 22:16:09.360926 19043 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260923 22:16:09.360939 19043 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260923 22:16:09.360950 19043 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260923 22:16:09.360981 19043 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260923 22:16:09.362437 19043 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.1.135.212:41195
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskRiQe_a/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790201539513534-1567-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskRiQe_a/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790201539513534-1567-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.1.135.254:43673
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.1.135.254:43673
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskRiQe_a/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790201539513534-1567-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.1.135.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.19043
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskRiQe_a/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790201539513534-1567-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 23 Sep 2026 20:53:37 UTC on ccycloud.quasar-rkviyx.root.comops.site
build id 4668083
I20260923 22:16:09.362709 19043 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260923 22:16:09.363041 19043 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260923 22:16:09.367841 19049 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260923 22:16:09.367859 19048 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260923 22:16:09.367833 19051 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260923 22:16:09.368040 19043 server_base.cc:1034] running on GCE node
I20260923 22:16:09.368304 19043 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260923 22:16:09.368604 19043 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260923 22:16:09.369774 19043 hybrid_clock.cc:648] HybridClock initialized: now 1790201769369768 us; error 37 us; skew 500 ppm
I20260923 22:16:09.371089 19043 webserver.cc:460] Webserver started at http://127.1.135.254:32811/ using document root <none> and password file <none>
I20260923 22:16:09.371364 19043 fs_manager.cc:359] Metadata directory not provided
I20260923 22:16:09.371434 19043 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260923 22:16:09.371579 19043 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260923 22:16:09.372748 19043 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskRiQe_a/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790201539513534-1567-0/minicluster-data/master-0/data/instance:
uuid: "2b2e1e684715488fb0c19715737e4bd7"
format_stamp: "Formatted at 2026-09-23 22:16:09 on dist-test-slave-s2fj"
server_key: "97eb16ade24973ece802bbf3834f1130"
server_key_iv: "ccdda8e58c9f7df8fb97f24cf1f8bc90"
server_key_version: "encryptionkey@0"
I20260923 22:16:09.373108 19043 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskRiQe_a/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790201539513534-1567-0/minicluster-data/master-0/wal/instance:
uuid: "2b2e1e684715488fb0c19715737e4bd7"
format_stamp: "Formatted at 2026-09-23 22:16:09 on dist-test-slave-s2fj"
server_key: "97eb16ade24973ece802bbf3834f1130"
server_key_iv: "ccdda8e58c9f7df8fb97f24cf1f8bc90"
server_key_version: "encryptionkey@0"
I20260923 22:16:09.374380 19043 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.000s
I20260923 22:16:09.375137 19057 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260923 22:16:09.375322 19043 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260923 22:16:09.375433 19043 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskRiQe_a/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790201539513534-1567-0/minicluster-data/master-0/data,/tmp/dist-test-taskRiQe_a/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790201539513534-1567-0/minicluster-data/master-0/wal
uuid: "2b2e1e684715488fb0c19715737e4bd7"
format_stamp: "Formatted at 2026-09-23 22:16:09 on dist-test-slave-s2fj"
server_key: "97eb16ade24973ece802bbf3834f1130"
server_key_iv: "ccdda8e58c9f7df8fb97f24cf1f8bc90"
server_key_version: "encryptionkey@0"
I20260923 22:16:09.375522 19043 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskRiQe_a/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790201539513534-1567-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskRiQe_a/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790201539513534-1567-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskRiQe_a/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790201539513534-1567-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260923 22:16:09.390975 19043 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260923 22:16:09.391361 19043 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260923 22:16:09.391520 19043 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260923 22:16:09.395869 19043 rpc_server.cc:307] RPC server started. Bound to: 127.1.135.254:43673
I20260923 22:16:09.395901 19109 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.1.135.254:43673 every 8 connection(s)
I20260923 22:16:09.396342 19043 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskRiQe_a/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790201539513534-1567-0/minicluster-data/master-0/data/info.pb
I20260923 22:16:09.397318 19110 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260923 22:16:09.399571 19110 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 2b2e1e684715488fb0c19715737e4bd7: Bootstrap starting.
I20260923 22:16:09.400130 19110 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 2b2e1e684715488fb0c19715737e4bd7: Neither blocks nor log segments found. Creating new log.
I20260923 22:16:09.400375 19110 log.cc:824] T 00000000000000000000000000000000 P 2b2e1e684715488fb0c19715737e4bd7: Log is configured to *not* fsync() on all Append() calls
I20260923 22:16:09.400974 19110 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 2b2e1e684715488fb0c19715737e4bd7: No bootstrap required, opened a new log
I20260923 22:16:09.402479 1567 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskRiQe_a/build/release/bin/kudu as pid 19043
I20260923 22:16:09.402616 1567 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskRiQe_a/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790201539513534-1567-0/minicluster-data/master-0/wal/instance
I20260923 22:16:09.402510 19110 raft_consensus.cc:348] T 00000000000000000000000000000000 P 2b2e1e684715488fb0c19715737e4bd7 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2b2e1e684715488fb0c19715737e4bd7" member_type: VOTER last_known_addr { host: "127.1.135.254" port: 43673 } }
I20260923 22:16:09.402707 19110 raft_consensus.cc:374] T 00000000000000000000000000000000 P 2b2e1e684715488fb0c19715737e4bd7 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260923 22:16:09.402756 19110 raft_consensus.cc:729] T 00000000000000000000000000000000 P 2b2e1e684715488fb0c19715737e4bd7 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 2b2e1e684715488fb0c19715737e4bd7, State: Initialized, Role: FOLLOWER
I20260923 22:16:09.402854 1567 external_mini_cluster.cc:1428] Setting key bdc13c87c86359c6c22891d9a9653b1a
I20260923 22:16:09.402910 19110 consensus_queue.cc:260] T 00000000000000000000000000000000 P 2b2e1e684715488fb0c19715737e4bd7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2b2e1e684715488fb0c19715737e4bd7" member_type: VOTER last_known_addr { host: "127.1.135.254" port: 43673 } }
I20260923 22:16:09.402999 19110 raft_consensus.cc:388] T 00000000000000000000000000000000 P 2b2e1e684715488fb0c19715737e4bd7 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260923 22:16:09.403060 19110 raft_consensus.cc:482] T 00000000000000000000000000000000 P 2b2e1e684715488fb0c19715737e4bd7 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260923 22:16:09.403110 19110 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 2b2e1e684715488fb0c19715737e4bd7 [term 0 FOLLOWER]: Advancing to term 1
I20260923 22:16:09.403860 19110 raft_consensus.cc:504] T 00000000000000000000000000000000 P 2b2e1e684715488fb0c19715737e4bd7 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2b2e1e684715488fb0c19715737e4bd7" member_type: VOTER last_known_addr { host: "127.1.135.254" port: 43673 } }
I20260923 22:16:09.403970 19110 leader_election.cc:302] T 00000000000000000000000000000000 P 2b2e1e684715488fb0c19715737e4bd7 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 2b2e1e684715488fb0c19715737e4bd7; no voters:
I20260923 22:16:09.404142 19110 leader_election.cc:288] T 00000000000000000000000000000000 P 2b2e1e684715488fb0c19715737e4bd7 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260923 22:16:09.404183 19115 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 2b2e1e684715488fb0c19715737e4bd7 [term 1 FOLLOWER]: Leader election won for term 1
I20260923 22:16:09.404369 19115 raft_consensus.cc:686] T 00000000000000000000000000000000 P 2b2e1e684715488fb0c19715737e4bd7 [term 1 LEADER]: Becoming Leader. State: Replica: 2b2e1e684715488fb0c19715737e4bd7, State: Running, Role: LEADER
I20260923 22:16:09.404487 19110 sys_catalog.cc:564] T 00000000000000000000000000000000 P 2b2e1e684715488fb0c19715737e4bd7 [sys.catalog]: configured and running, proceeding with master startup.
I20260923 22:16:09.404529 19115 consensus_queue.cc:237] T 00000000000000000000000000000000 P 2b2e1e684715488fb0c19715737e4bd7 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2b2e1e684715488fb0c19715737e4bd7" member_type: VOTER last_known_addr { host: "127.1.135.254" port: 43673 } }
I20260923 22:16:09.404929 19115 sys_catalog.cc:455] T 00000000000000000000000000000000 P 2b2e1e684715488fb0c19715737e4bd7 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 2b2e1e684715488fb0c19715737e4bd7. Latest consensus state: current_term: 1 leader_uuid: "2b2e1e684715488fb0c19715737e4bd7" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2b2e1e684715488fb0c19715737e4bd7" member_type: VOTER last_known_addr { host: "127.1.135.254" port: 43673 } } }
I20260923 22:16:09.405015 19115 sys_catalog.cc:458] T 00000000000000000000000000000000 P 2b2e1e684715488fb0c19715737e4bd7 [sys.catalog]: This master's current role is: LEADER
I20260923 22:16:09.404940 19116 sys_catalog.cc:455] T 00000000000000000000000000000000 P 2b2e1e684715488fb0c19715737e4bd7 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "2b2e1e684715488fb0c19715737e4bd7" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2b2e1e684715488fb0c19715737e4bd7" member_type: VOTER last_known_addr { host: "127.1.135.254" port: 43673 } } }
I20260923 22:16:09.405189 19116 sys_catalog.cc:458] T 00000000000000000000000000000000 P 2b2e1e684715488fb0c19715737e4bd7 [sys.catalog]: This master's current role is: LEADER
W20260923 22:16:09.406395 19131 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 2b2e1e684715488fb0c19715737e4bd7: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260923 22:16:09.406435 19131 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260923 22:16:09.406504 19123 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260923 22:16:09.406648 19123 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260923 22:16:09.408128 19123 catalog_manager.cc:1347] Generated new cluster ID: c055ff77733a4814a415be55fa058dcf
I20260923 22:16:09.408183 19123 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260923 22:16:09.409415 19114 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 22:16:09.403474 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55650 (local address 127.1.135.254:43673)
0923 22:16:09.403679 (+ 205us) server_negotiation.cc:206] Beginning negotiation
0923 22:16:09.403688 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0923 22:16:09.403723 (+ 35us) server_negotiation.cc:407] Connection header received
0923 22:16:09.403852 (+ 129us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 22:16:09.403859 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 22:16:09.403949 (+ 90us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 22:16:09.404131 (+ 182us) server_negotiation.cc:226] Negotiated authn=SASL
0923 22:16:09.404394 (+ 263us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 22:16:09.406124 (+ 1730us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 22:16:09.407217 (+ 1093us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 22:16:09.407854 (+ 637us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 22:16:09.407982 (+ 128us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 22:16:09.408685 (+ 703us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0923 22:16:09.408689 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0923 22:16:09.408689 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0923 22:16:09.408887 (+ 198us) server_negotiation.cc:922] calling sasl_server_start()
0923 22:16:09.408926 (+ 39us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0923 22:16:09.408939 (+ 13us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0923 22:16:09.409002 (+ 63us) server_negotiation.cc:1035] Waiting for connection context
0923 22:16:09.409137 (+ 135us) server_negotiation.cc:299] Negotiation successful
0923 22:16:09.409210 (+ 73us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":124,"thread_start_us":83,"threads_started":1}
I20260923 22:16:09.436676 19123 catalog_manager.cc:1370] Generated new certificate authority record
I20260923 22:16:09.437492 19123 catalog_manager.cc:1504] Loading token signing keys...
I20260923 22:16:09.450588 19123 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 2b2e1e684715488fb0c19715737e4bd7: Generated new TSK 0
I20260923 22:16:09.450824 19123 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260923 22:16:09.459868 1567 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskRiQe_a/build/release/bin/kudu
/tmp/dist-test-taskRiQe_a/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskRiQe_a/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790201539513534-1567-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskRiQe_a/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790201539513534-1567-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskRiQe_a/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790201539513534-1567-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskRiQe_a/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790201539513534-1567-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.1.135.193:0
--local_ip_for_outbound_sockets=127.1.135.193
--webserver_interface=127.1.135.193
--webserver_port=0
--tserver_master_addrs=127.1.135.254:43673
--builtin_ntp_servers=127.1.135.212:41195
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
W20260923 22:16:09.542775 19134 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260923 22:16:09.543001 19134 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260923 22:16:09.543023 19134 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260923 22:16:09.544308 19134 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260923 22:16:09.544353 19134 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260923 22:16:09.544381 19134 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.1.135.193
I20260923 22:16:09.545784 19134 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.1.135.212:41195
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskRiQe_a/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790201539513534-1567-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskRiQe_a/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790201539513534-1567-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.1.135.193:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskRiQe_a/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790201539513534-1567-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.1.135.193
--webserver_port=0
--tserver_master_addrs=127.1.135.254:43673
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.19134
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.1.135.193
--log_dir=/tmp/dist-test-taskRiQe_a/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790201539513534-1567-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 23 Sep 2026 20:53:37 UTC on ccycloud.quasar-rkviyx.root.comops.site
build id 4668083
I20260923 22:16:09.546051 19134 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260923 22:16:09.546373 19134 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260923 22:16:09.551307 19142 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260923 22:16:09.551301 19139 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260923 22:16:09.551311 19140 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260923 22:16:09.551592 19134 server_base.cc:1034] running on GCE node
I20260923 22:16:09.551808 19134 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260923 22:16:09.552121 19134 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260923 22:16:09.553277 19134 hybrid_clock.cc:648] HybridClock initialized: now 1790201769553267 us; error 28 us; skew 500 ppm
I20260923 22:16:09.554579 19134 webserver.cc:460] Webserver started at http://127.1.135.193:45835/ using document root <none> and password file <none>
I20260923 22:16:09.554821 19134 fs_manager.cc:359] Metadata directory not provided
I20260923 22:16:09.554873 19134 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260923 22:16:09.555024 19134 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260923 22:16:09.556188 19134 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskRiQe_a/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790201539513534-1567-0/minicluster-data/ts-0/data/instance:
uuid: "097e47bb0fba4a90aac02fb28166828c"
format_stamp: "Formatted at 2026-09-23 22:16:09 on dist-test-slave-s2fj"
server_key: "29485f3d19aab3f20a9f1c6444004fe8"
server_key_iv: "6bb55a88a6467b49ced17c662705923f"
server_key_version: "encryptionkey@0"
I20260923 22:16:09.556550 19134 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskRiQe_a/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790201539513534-1567-0/minicluster-data/ts-0/wal/instance:
uuid: "097e47bb0fba4a90aac02fb28166828c"
format_stamp: "Formatted at 2026-09-23 22:16:09 on dist-test-slave-s2fj"
server_key: "29485f3d19aab3f20a9f1c6444004fe8"
server_key_iv: "6bb55a88a6467b49ced17c662705923f"
server_key_version: "encryptionkey@0"
I20260923 22:16:09.557832 19134 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.002s
I20260923 22:16:09.558537 19148 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260923 22:16:09.558708 19134 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.001s
I20260923 22:16:09.558805 19134 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskRiQe_a/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790201539513534-1567-0/minicluster-data/ts-0/data,/tmp/dist-test-taskRiQe_a/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790201539513534-1567-0/minicluster-data/ts-0/wal
uuid: "097e47bb0fba4a90aac02fb28166828c"
format_stamp: "Formatted at 2026-09-23 22:16:09 on dist-test-slave-s2fj"
server_key: "29485f3d19aab3f20a9f1c6444004fe8"
server_key_iv: "6bb55a88a6467b49ced17c662705923f"
server_key_version: "encryptionkey@0"
I20260923 22:16:09.558899 19134 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskRiQe_a/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790201539513534-1567-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskRiQe_a/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790201539513534-1567-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskRiQe_a/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790201539513534-1567-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260923 22:16:09.580509 19134 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260923 22:16:09.580868 19134 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260923 22:16:09.581019 19134 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260923 22:16:09.581266 19134 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260923 22:16:09.581624 19134 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260923 22:16:09.581679 19134 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260923 22:16:09.581727 19134 ts_tablet_manager.cc:610] Registered 0 tablets
I20260923 22:16:09.581750 19134 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260923 22:16:09.587786 19134 rpc_server.cc:307] RPC server started. Bound to: 127.1.135.193:44509
I20260923 22:16:09.587827 19261 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.1.135.193:44509 every 8 connection(s)
I20260923 22:16:09.588220 19134 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskRiQe_a/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790201539513534-1567-0/minicluster-data/ts-0/data/info.pb
I20260923 22:16:09.593613 19114 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 22:16:09.589003 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.135.193:35347 (local address 127.1.135.254:43673)
0923 22:16:09.589067 (+ 64us) server_negotiation.cc:206] Beginning negotiation
0923 22:16:09.589069 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0923 22:16:09.589312 (+ 243us) server_negotiation.cc:407] Connection header received
0923 22:16:09.589855 (+ 543us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 22:16:09.589857 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 22:16:09.589872 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 22:16:09.589919 (+ 47us) server_negotiation.cc:226] Negotiated authn=SASL
0923 22:16:09.590609 (+ 690us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 22:16:09.591108 (+ 499us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 22:16:09.592158 (+ 1050us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 22:16:09.592376 (+ 218us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 22:16:09.592465 (+ 89us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 22:16:09.593298 (+ 833us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0923 22:16:09.593300 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0923 22:16:09.593300 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0923 22:16:09.593305 (+ 5us) server_negotiation.cc:922] calling sasl_server_start()
0923 22:16:09.593321 (+ 16us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0923 22:16:09.593326 (+ 5us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0923 22:16:09.593400 (+ 74us) server_negotiation.cc:1035] Waiting for connection context
0923 22:16:09.593497 (+ 97us) server_negotiation.cc:299] Negotiation successful
0923 22:16:09.593531 (+ 34us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":34}
I20260923 22:16:09.593837 19264 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 22:16:09.588988 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.135.254:43673 (local address 127.1.135.193:35347)
0923 22:16:09.589191 (+ 203us) negotiation.cc:107] Waiting for socket to connect
0923 22:16:09.589227 (+ 36us) client_negotiation.cc:174] Beginning negotiation
0923 22:16:09.589740 (+ 513us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0923 22:16:09.590005 (+ 265us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0923 22:16:09.590008 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0923 22:16:09.590016 (+ 8us) client_negotiation.cc:189] Negotiated authn=SASL
0923 22:16:09.590517 (+ 501us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 22:16:09.590521 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 22:16:09.591219 (+ 698us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 22:16:09.591220 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 22:16:09.592013 (+ 793us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 22:16:09.592015 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 22:16:09.592477 (+ 462us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 22:16:09.592478 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 22:16:09.592615 (+ 137us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 22:16:09.593146 (+ 531us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0923 22:16:09.593147 (+ 1us) client_negotiation.cc:656] calling sasl_client_start()
0923 22:16:09.593174 (+ 27us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0923 22:16:09.593197 (+ 23us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0923 22:16:09.593403 (+ 206us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0923 22:16:09.593409 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0923 22:16:09.593413 (+ 4us) client_negotiation.cc:769] Sending connection context
0923 22:16:09.593502 (+ 89us) client_negotiation.cc:240] Negotiation successful
0923 22:16:09.593606 (+ 104us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":144,"thread_start_us":93,"threads_started":1}
I20260923 22:16:09.594429 19262 heartbeater.cc:347] Connected to a master server at 127.1.135.254:43673
I20260923 22:16:09.594530 19262 heartbeater.cc:464] Registering TS with master...
I20260923 22:16:09.594681 19262 heartbeater.cc:511] Master 127.1.135.254:43673 requested a full tablet report, sending...
I20260923 22:16:09.595067 1567 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskRiQe_a/build/release/bin/kudu as pid 19134
I20260923 22:16:09.595187 1567 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskRiQe_a/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790201539513534-1567-0/minicluster-data/ts-0/wal/instance
I20260923 22:16:09.595285 19074 ts_manager.cc:194] Registered new tserver with Master: 097e47bb0fba4a90aac02fb28166828c (127.1.135.193:44509)
I20260923 22:16:09.595427 1567 external_mini_cluster.cc:1428] Setting key 03627517338099d820b5364e6e2a65c2
I20260923 22:16:09.596189 19074 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.1.135.193:35347
I20260923 22:16:09.596673 1567 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskRiQe_a/build/release/bin/kudu
/tmp/dist-test-taskRiQe_a/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskRiQe_a/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790201539513534-1567-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskRiQe_a/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790201539513534-1567-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskRiQe_a/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790201539513534-1567-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskRiQe_a/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790201539513534-1567-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.1.135.194:0
--local_ip_for_outbound_sockets=127.1.135.194
--webserver_interface=127.1.135.194
--webserver_port=0
--tserver_master_addrs=127.1.135.254:43673
--builtin_ntp_servers=127.1.135.212:41195
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
W20260923 22:16:09.678027 19265 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260923 22:16:09.678258 19265 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260923 22:16:09.678293 19265 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260923 22:16:09.679613 19265 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260923 22:16:09.679682 19265 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260923 22:16:09.679720 19265 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.1.135.194
I20260923 22:16:09.681120 19265 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.1.135.212:41195
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskRiQe_a/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790201539513534-1567-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskRiQe_a/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790201539513534-1567-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.1.135.194:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskRiQe_a/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790201539513534-1567-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.1.135.194
--webserver_port=0
--tserver_master_addrs=127.1.135.254:43673
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.19265
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.1.135.194
--log_dir=/tmp/dist-test-taskRiQe_a/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790201539513534-1567-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 23 Sep 2026 20:53:37 UTC on ccycloud.quasar-rkviyx.root.comops.site
build id 4668083
I20260923 22:16:09.681394 19265 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260923 22:16:09.681705 19265 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260923 22:16:09.686594 19273 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260923 22:16:09.686627 19271 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260923 22:16:09.686632 19270 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260923 22:16:09.686854 19265 server_base.cc:1034] running on GCE node
I20260923 22:16:09.687108 19265 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260923 22:16:09.687376 19265 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260923 22:16:09.688540 19265 hybrid_clock.cc:648] HybridClock initialized: now 1790201769688530 us; error 40 us; skew 500 ppm
I20260923 22:16:09.689793 19265 webserver.cc:460] Webserver started at http://127.1.135.194:43045/ using document root <none> and password file <none>
I20260923 22:16:09.690055 19265 fs_manager.cc:359] Metadata directory not provided
I20260923 22:16:09.690124 19265 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260923 22:16:09.690269 19265 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260923 22:16:09.691377 19265 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskRiQe_a/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790201539513534-1567-0/minicluster-data/ts-1/data/instance:
uuid: "bc06d877919f41a6974c66c7ebc04461"
format_stamp: "Formatted at 2026-09-23 22:16:09 on dist-test-slave-s2fj"
server_key: "c084c937cc43423be6b8b7d7293f6867"
server_key_iv: "322622697268205c5114449c237d7ccb"
server_key_version: "encryptionkey@0"
I20260923 22:16:09.691740 19265 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskRiQe_a/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790201539513534-1567-0/minicluster-data/ts-1/wal/instance:
uuid: "bc06d877919f41a6974c66c7ebc04461"
format_stamp: "Formatted at 2026-09-23 22:16:09 on dist-test-slave-s2fj"
server_key: "c084c937cc43423be6b8b7d7293f6867"
server_key_iv: "322622697268205c5114449c237d7ccb"
server_key_version: "encryptionkey@0"
I20260923 22:16:09.693094 19265 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260923 22:16:09.693804 19279 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260923 22:16:09.693975 19265 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260923 22:16:09.694062 19265 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskRiQe_a/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790201539513534-1567-0/minicluster-data/ts-1/data,/tmp/dist-test-taskRiQe_a/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790201539513534-1567-0/minicluster-data/ts-1/wal
uuid: "bc06d877919f41a6974c66c7ebc04461"
format_stamp: "Formatted at 2026-09-23 22:16:09 on dist-test-slave-s2fj"
server_key: "c084c937cc43423be6b8b7d7293f6867"
server_key_iv: "322622697268205c5114449c237d7ccb"
server_key_version: "encryptionkey@0"
I20260923 22:16:09.694136 19265 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskRiQe_a/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790201539513534-1567-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskRiQe_a/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790201539513534-1567-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskRiQe_a/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790201539513534-1567-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260923 22:16:09.735424 19265 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260923 22:16:09.735765 19265 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260923 22:16:09.735916 19265 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260923 22:16:09.736222 19265 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260923 22:16:09.736575 19265 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260923 22:16:09.736627 19265 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260923 22:16:09.736661 19265 ts_tablet_manager.cc:610] Registered 0 tablets
I20260923 22:16:09.736688 19265 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260923 22:16:09.742136 19265 rpc_server.cc:307] RPC server started. Bound to: 127.1.135.194:46683
I20260923 22:16:09.742225 19392 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.1.135.194:46683 every 8 connection(s)
I20260923 22:16:09.742594 19265 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskRiQe_a/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790201539513534-1567-0/minicluster-data/ts-1/data/info.pb
I20260923 22:16:09.748035 19114 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 22:16:09.743384 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.135.194:60561 (local address 127.1.135.254:43673)
0923 22:16:09.743447 (+ 63us) server_negotiation.cc:206] Beginning negotiation
0923 22:16:09.743449 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0923 22:16:09.743702 (+ 253us) server_negotiation.cc:407] Connection header received
0923 22:16:09.744331 (+ 629us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 22:16:09.744333 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 22:16:09.744351 (+ 18us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 22:16:09.744398 (+ 47us) server_negotiation.cc:226] Negotiated authn=SASL
0923 22:16:09.745105 (+ 707us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 22:16:09.745626 (+ 521us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 22:16:09.746645 (+ 1019us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 22:16:09.746850 (+ 205us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 22:16:09.746921 (+ 71us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 22:16:09.747731 (+ 810us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0923 22:16:09.747733 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0923 22:16:09.747733 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0923 22:16:09.747739 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0923 22:16:09.747754 (+ 15us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0923 22:16:09.747759 (+ 5us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0923 22:16:09.747826 (+ 67us) server_negotiation.cc:1035] Waiting for connection context
0923 22:16:09.747926 (+ 100us) server_negotiation.cc:299] Negotiation successful
0923 22:16:09.747955 (+ 29us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":34}
I20260923 22:16:09.748283 19395 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 22:16:09.743357 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.135.254:43673 (local address 127.1.135.194:60561)
0923 22:16:09.743565 (+ 208us) negotiation.cc:107] Waiting for socket to connect
0923 22:16:09.743600 (+ 35us) client_negotiation.cc:174] Beginning negotiation
0923 22:16:09.744208 (+ 608us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0923 22:16:09.744484 (+ 276us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0923 22:16:09.744488 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0923 22:16:09.744504 (+ 16us) client_negotiation.cc:189] Negotiated authn=SASL
0923 22:16:09.745003 (+ 499us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 22:16:09.745007 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 22:16:09.745731 (+ 724us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 22:16:09.745732 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 22:16:09.746539 (+ 807us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 22:16:09.746541 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 22:16:09.746943 (+ 402us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 22:16:09.746944 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 22:16:09.747079 (+ 135us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 22:16:09.747593 (+ 514us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0923 22:16:09.747594 (+ 1us) client_negotiation.cc:656] calling sasl_client_start()
0923 22:16:09.747621 (+ 27us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0923 22:16:09.747642 (+ 21us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0923 22:16:09.747825 (+ 183us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0923 22:16:09.747832 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0923 22:16:09.747835 (+ 3us) client_negotiation.cc:769] Sending connection context
0923 22:16:09.747929 (+ 94us) client_negotiation.cc:240] Negotiation successful
0923 22:16:09.748030 (+ 101us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":140,"thread_start_us":94,"threads_started":1}
I20260923 22:16:09.748788 19393 heartbeater.cc:347] Connected to a master server at 127.1.135.254:43673
I20260923 22:16:09.748868 19393 heartbeater.cc:464] Registering TS with master...
I20260923 22:16:09.749009 19393 heartbeater.cc:511] Master 127.1.135.254:43673 requested a full tablet report, sending...
I20260923 22:16:09.749329 19074 ts_manager.cc:194] Registered new tserver with Master: bc06d877919f41a6974c66c7ebc04461 (127.1.135.194:46683)
I20260923 22:16:09.749655 19074 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.1.135.194:60561
I20260923 22:16:09.752413 1567 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskRiQe_a/build/release/bin/kudu as pid 19265
I20260923 22:16:09.752496 1567 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskRiQe_a/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790201539513534-1567-0/minicluster-data/ts-1/wal/instance
I20260923 22:16:09.752661 1567 external_mini_cluster.cc:1428] Setting key eaaee31de6696811cc929dfd0315424d
I20260923 22:16:09.753523 1567 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskRiQe_a/build/release/bin/kudu
/tmp/dist-test-taskRiQe_a/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskRiQe_a/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790201539513534-1567-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskRiQe_a/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790201539513534-1567-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskRiQe_a/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790201539513534-1567-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskRiQe_a/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790201539513534-1567-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.1.135.195:0
--local_ip_for_outbound_sockets=127.1.135.195
--webserver_interface=127.1.135.195
--webserver_port=0
--tserver_master_addrs=127.1.135.254:43673
--builtin_ntp_servers=127.1.135.212:41195
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
W20260923 22:16:09.834164 19396 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260923 22:16:09.834369 19396 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260923 22:16:09.834390 19396 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260923 22:16:09.835458 19396 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260923 22:16:09.835490 19396 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260923 22:16:09.835513 19396 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.1.135.195
I20260923 22:16:09.836826 19396 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.1.135.212:41195
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskRiQe_a/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790201539513534-1567-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskRiQe_a/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790201539513534-1567-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.1.135.195:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskRiQe_a/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790201539513534-1567-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.1.135.195
--webserver_port=0
--tserver_master_addrs=127.1.135.254:43673
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.19396
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.1.135.195
--log_dir=/tmp/dist-test-taskRiQe_a/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790201539513534-1567-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 23 Sep 2026 20:53:37 UTC on ccycloud.quasar-rkviyx.root.comops.site
build id 4668083
I20260923 22:16:09.837021 19396 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260923 22:16:09.837275 19396 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260923 22:16:09.844412 19404 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260923 22:16:09.844379 19402 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260923 22:16:09.844434 19401 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260923 22:16:09.845480 19396 server_base.cc:1034] running on GCE node
I20260923 22:16:09.845746 19396 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260923 22:16:09.846050 19396 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260923 22:16:09.847208 19396 hybrid_clock.cc:648] HybridClock initialized: now 1790201769847185 us; error 42 us; skew 500 ppm
I20260923 22:16:09.848522 19396 webserver.cc:460] Webserver started at http://127.1.135.195:33335/ using document root <none> and password file <none>
I20260923 22:16:09.848789 19396 fs_manager.cc:359] Metadata directory not provided
I20260923 22:16:09.848857 19396 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260923 22:16:09.849033 19396 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260923 22:16:09.850108 19396 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskRiQe_a/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790201539513534-1567-0/minicluster-data/ts-2/data/instance:
uuid: "708230f249c6409c8dcc3eee79c7f370"
format_stamp: "Formatted at 2026-09-23 22:16:09 on dist-test-slave-s2fj"
server_key: "dfe0a3db147ec78f25bb85378d8d00a7"
server_key_iv: "3a6598ee57a37731f00a86d2db6e7009"
server_key_version: "encryptionkey@0"
I20260923 22:16:09.850476 19396 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskRiQe_a/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790201539513534-1567-0/minicluster-data/ts-2/wal/instance:
uuid: "708230f249c6409c8dcc3eee79c7f370"
format_stamp: "Formatted at 2026-09-23 22:16:09 on dist-test-slave-s2fj"
server_key: "dfe0a3db147ec78f25bb85378d8d00a7"
server_key_iv: "3a6598ee57a37731f00a86d2db6e7009"
server_key_version: "encryptionkey@0"
I20260923 22:16:09.851728 19396 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.001s
I20260923 22:16:09.852523 19410 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260923 22:16:09.852689 19396 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.000s
I20260923 22:16:09.852782 19396 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskRiQe_a/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790201539513534-1567-0/minicluster-data/ts-2/data,/tmp/dist-test-taskRiQe_a/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790201539513534-1567-0/minicluster-data/ts-2/wal
uuid: "708230f249c6409c8dcc3eee79c7f370"
format_stamp: "Formatted at 2026-09-23 22:16:09 on dist-test-slave-s2fj"
server_key: "dfe0a3db147ec78f25bb85378d8d00a7"
server_key_iv: "3a6598ee57a37731f00a86d2db6e7009"
server_key_version: "encryptionkey@0"
I20260923 22:16:09.852861 19396 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskRiQe_a/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790201539513534-1567-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskRiQe_a/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790201539513534-1567-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskRiQe_a/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790201539513534-1567-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260923 22:16:09.863014 19396 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260923 22:16:09.863322 19396 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260923 22:16:09.863497 19396 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260923 22:16:09.863713 19396 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260923 22:16:09.864029 19396 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260923 22:16:09.864154 19396 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260923 22:16:09.864204 19396 ts_tablet_manager.cc:610] Registered 0 tablets
I20260923 22:16:09.864250 19396 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260923 22:16:09.870162 19396 rpc_server.cc:307] RPC server started. Bound to: 127.1.135.195:42159
I20260923 22:16:09.870201 19523 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.1.135.195:42159 every 8 connection(s)
I20260923 22:16:09.870568 19396 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskRiQe_a/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790201539513534-1567-0/minicluster-data/ts-2/data/info.pb
I20260923 22:16:09.875415 19114 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 22:16:09.871276 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.135.195:59277 (local address 127.1.135.254:43673)
0923 22:16:09.871321 (+ 45us) server_negotiation.cc:206] Beginning negotiation
0923 22:16:09.871323 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0923 22:16:09.871587 (+ 264us) server_negotiation.cc:407] Connection header received
0923 22:16:09.872044 (+ 457us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 22:16:09.872045 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 22:16:09.872055 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 22:16:09.872155 (+ 100us) server_negotiation.cc:226] Negotiated authn=SASL
0923 22:16:09.872800 (+ 645us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 22:16:09.873287 (+ 487us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 22:16:09.874220 (+ 933us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 22:16:09.874395 (+ 175us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 22:16:09.874445 (+ 50us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 22:16:09.875216 (+ 771us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0923 22:16:09.875217 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0923 22:16:09.875218 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0923 22:16:09.875222 (+ 4us) server_negotiation.cc:922] calling sasl_server_start()
0923 22:16:09.875237 (+ 15us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0923 22:16:09.875241 (+ 4us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0923 22:16:09.875283 (+ 42us) server_negotiation.cc:1035] Waiting for connection context
0923 22:16:09.875329 (+ 46us) server_negotiation.cc:299] Negotiation successful
0923 22:16:09.875356 (+ 27us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":19}
I20260923 22:16:09.875649 19526 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 22:16:09.871280 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.135.254:43673 (local address 127.1.135.195:59277)
0923 22:16:09.871454 (+ 174us) negotiation.cc:107] Waiting for socket to connect
0923 22:16:09.871490 (+ 36us) client_negotiation.cc:174] Beginning negotiation
0923 22:16:09.871960 (+ 470us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0923 22:16:09.872212 (+ 252us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0923 22:16:09.872215 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0923 22:16:09.872223 (+ 8us) client_negotiation.cc:189] Negotiated authn=SASL
0923 22:16:09.872728 (+ 505us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 22:16:09.872733 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 22:16:09.873372 (+ 639us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 22:16:09.873373 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 22:16:09.874146 (+ 773us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 22:16:09.874148 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 22:16:09.874469 (+ 321us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 22:16:09.874470 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 22:16:09.874578 (+ 108us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 22:16:09.875097 (+ 519us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0923 22:16:09.875097 (+ 0us) client_negotiation.cc:656] calling sasl_client_start()
0923 22:16:09.875125 (+ 28us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0923 22:16:09.875146 (+ 21us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0923 22:16:09.875283 (+ 137us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0923 22:16:09.875285 (+ 2us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0923 22:16:09.875288 (+ 3us) client_negotiation.cc:769] Sending connection context
0923 22:16:09.875332 (+ 44us) client_negotiation.cc:240] Negotiation successful
0923 22:16:09.875411 (+ 79us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":114,"thread_start_us":88,"threads_started":1}
I20260923 22:16:09.876217 19524 heartbeater.cc:347] Connected to a master server at 127.1.135.254:43673
I20260923 22:16:09.876333 19524 heartbeater.cc:464] Registering TS with master...
I20260923 22:16:09.876520 19524 heartbeater.cc:511] Master 127.1.135.254:43673 requested a full tablet report, sending...
I20260923 22:16:09.876924 19074 ts_manager.cc:194] Registered new tserver with Master: 708230f249c6409c8dcc3eee79c7f370 (127.1.135.195:42159)
I20260923 22:16:09.877296 19074 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.1.135.195:59277
I20260923 22:16:09.878475 1567 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskRiQe_a/build/release/bin/kudu as pid 19396
I20260923 22:16:09.878558 1567 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskRiQe_a/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790201539513534-1567-0/minicluster-data/ts-2/wal/instance
I20260923 22:16:09.878769 1567 external_mini_cluster.cc:1428] Setting key f5ca89f13e54eda50f91af1da7a72a8d
I20260923 22:16:09.879642 1567 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260923 22:16:09.883682 19114 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 22:16:09.881092 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55658 (local address 127.1.135.254:43673)
0923 22:16:09.881137 (+ 45us) server_negotiation.cc:206] Beginning negotiation
0923 22:16:09.881139 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0923 22:16:09.881260 (+ 121us) server_negotiation.cc:407] Connection header received
0923 22:16:09.881299 (+ 39us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 22:16:09.881300 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 22:16:09.881309 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 22:16:09.881365 (+ 56us) server_negotiation.cc:226] Negotiated authn=SASL
0923 22:16:09.881498 (+ 133us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 22:16:09.881961 (+ 463us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 22:16:09.882650 (+ 689us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 22:16:09.882818 (+ 168us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 22:16:09.882874 (+ 56us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 22:16:09.883497 (+ 623us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0923 22:16:09.883498 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0923 22:16:09.883499 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0923 22:16:09.883502 (+ 3us) server_negotiation.cc:922] calling sasl_server_start()
0923 22:16:09.883516 (+ 14us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0923 22:16:09.883521 (+ 5us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0923 22:16:09.883562 (+ 41us) server_negotiation.cc:1035] Waiting for connection context
0923 22:16:09.883608 (+ 46us) server_negotiation.cc:299] Negotiation successful
0923 22:16:09.883629 (+ 21us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":19}
I20260923 22:16:09.885236 19074 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:55658:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260923 22:16:09.886802 19074 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260923 22:16:09.893129 19532 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 22:16:09.889041 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.135.195:42159 (local address 127.0.0.1:43738)
0923 22:16:09.889341 (+ 300us) negotiation.cc:107] Waiting for socket to connect
0923 22:16:09.889365 (+ 24us) client_negotiation.cc:174] Beginning negotiation
0923 22:16:09.889420 (+ 55us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0923 22:16:09.889625 (+ 205us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0923 22:16:09.889628 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0923 22:16:09.889631 (+ 3us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0923 22:16:09.889708 (+ 77us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 22:16:09.889710 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 22:16:09.890827 (+ 1117us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 22:16:09.890828 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 22:16:09.892038 (+ 1210us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 22:16:09.892041 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 22:16:09.892827 (+ 786us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 22:16:09.892828 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 22:16:09.892906 (+ 78us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 22:16:09.892919 (+ 13us) client_negotiation.cc:769] Sending connection context
0923 22:16:09.892997 (+ 78us) client_negotiation.cc:240] Negotiation successful
0923 22:16:09.893051 (+ 54us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":288,"mutex_wait_us":2,"thread_start_us":66,"threads_started":1}
I20260923 22:16:09.893146 19534 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 22:16:09.889124 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43738 (local address 127.1.135.195:42159)
0923 22:16:09.889338 (+ 214us) server_negotiation.cc:206] Beginning negotiation
0923 22:16:09.889342 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0923 22:16:09.889408 (+ 66us) server_negotiation.cc:407] Connection header received
0923 22:16:09.889488 (+ 80us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 22:16:09.889490 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 22:16:09.889508 (+ 18us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 22:16:09.889620 (+ 112us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0923 22:16:09.889787 (+ 167us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 22:16:09.890737 (+ 950us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 22:16:09.892155 (+ 1418us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 22:16:09.892726 (+ 571us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 22:16:09.892824 (+ 98us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 22:16:09.892935 (+ 111us) server_negotiation.cc:1035] Waiting for connection context
0923 22:16:09.893011 (+ 76us) server_negotiation.cc:299] Negotiation successful
0923 22:16:09.893050 (+ 39us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":177,"thread_start_us":74,"threads_started":1}
I20260923 22:16:09.893436 19533 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 22:16:09.888996 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.135.194:46683 (local address 127.0.0.1:37182)
0923 22:16:09.889193 (+ 197us) negotiation.cc:107] Waiting for socket to connect
0923 22:16:09.889241 (+ 48us) client_negotiation.cc:174] Beginning negotiation
0923 22:16:09.889308 (+ 67us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0923 22:16:09.889637 (+ 329us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0923 22:16:09.889638 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0923 22:16:09.889640 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0923 22:16:09.889715 (+ 75us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 22:16:09.889717 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 22:16:09.891216 (+ 1499us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 22:16:09.891217 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 22:16:09.892276 (+ 1059us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 22:16:09.892278 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 22:16:09.893207 (+ 929us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 22:16:09.893208 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 22:16:09.893265 (+ 57us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 22:16:09.893269 (+ 4us) client_negotiation.cc:769] Sending connection context
0923 22:16:09.893297 (+ 28us) client_negotiation.cc:240] Negotiation successful
0923 22:16:09.893334 (+ 37us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":171,"mutex_wait_us":33,"thread_start_us":85,"threads_started":1}
I20260923 22:16:09.893908 19535 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 22:16:09.889109 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37182 (local address 127.1.135.194:46683)
0923 22:16:09.889456 (+ 347us) server_negotiation.cc:206] Beginning negotiation
0923 22:16:09.889461 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0923 22:16:09.889472 (+ 11us) server_negotiation.cc:407] Connection header received
0923 22:16:09.889512 (+ 40us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 22:16:09.889514 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 22:16:09.889529 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 22:16:09.889868 (+ 339us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0923 22:16:09.889933 (+ 65us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 22:16:09.891106 (+ 1173us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 22:16:09.892373 (+ 1267us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 22:16:09.893109 (+ 736us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 22:16:09.893646 (+ 537us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 22:16:09.893762 (+ 116us) server_negotiation.cc:1035] Waiting for connection context
0923 22:16:09.893790 (+ 28us) server_negotiation.cc:299] Negotiation successful
0923 22:16:09.893827 (+ 37us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":299,"thread_start_us":94,"threads_started":1}
I20260923 22:16:09.894501 19536 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 22:16:09.889232 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.135.193:44509 (local address 127.0.0.1:46672)
0923 22:16:09.889589 (+ 357us) negotiation.cc:107] Waiting for socket to connect
0923 22:16:09.889599 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0923 22:16:09.889649 (+ 50us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0923 22:16:09.889842 (+ 193us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0923 22:16:09.889843 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0923 22:16:09.889845 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0923 22:16:09.889881 (+ 36us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 22:16:09.889882 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 22:16:09.891203 (+ 1321us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 22:16:09.891204 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 22:16:09.893326 (+ 2122us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 22:16:09.893328 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 22:16:09.894283 (+ 955us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 22:16:09.894284 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 22:16:09.894354 (+ 70us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 22:16:09.894359 (+ 5us) client_negotiation.cc:769] Sending connection context
0923 22:16:09.894400 (+ 41us) client_negotiation.cc:240] Negotiation successful
0923 22:16:09.894441 (+ 41us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":340,"thread_start_us":63,"threads_started":1}
I20260923 22:16:09.894394 19458 tablet_service.cc:1467] Processing CreateTablet for tablet 6c660fe5406c4f4193e901eb27d32379 (DEFAULT_TABLE table=test-table [id=2851d7eeb93b43f18125bc549a197e2a]), partition=RANGE (key) PARTITION UNBOUNDED
I20260923 22:16:09.894552 19327 tablet_service.cc:1467] Processing CreateTablet for tablet 6c660fe5406c4f4193e901eb27d32379 (DEFAULT_TABLE table=test-table [id=2851d7eeb93b43f18125bc549a197e2a]), partition=RANGE (key) PARTITION UNBOUNDED
I20260923 22:16:09.894770 19458 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 6c660fe5406c4f4193e901eb27d32379. 1 dirs total, 0 dirs full, 0 dirs failed
I20260923 22:16:09.894830 19537 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 22:16:09.889384 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46672 (local address 127.1.135.193:44509)
0923 22:16:09.889569 (+ 185us) server_negotiation.cc:206] Beginning negotiation
0923 22:16:09.889574 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0923 22:16:09.889707 (+ 133us) server_negotiation.cc:407] Connection header received
0923 22:16:09.889751 (+ 44us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 22:16:09.889753 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 22:16:09.889769 (+ 16us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 22:16:09.889846 (+ 77us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0923 22:16:09.889937 (+ 91us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 22:16:09.891113 (+ 1176us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 22:16:09.893412 (+ 2299us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 22:16:09.894074 (+ 662us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 22:16:09.894156 (+ 82us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 22:16:09.894670 (+ 514us) server_negotiation.cc:1035] Waiting for connection context
0923 22:16:09.894696 (+ 26us) server_negotiation.cc:299] Negotiation successful
0923 22:16:09.894742 (+ 46us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":142,"thread_start_us":106,"threads_started":1}
I20260923 22:16:09.894928 19327 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 6c660fe5406c4f4193e901eb27d32379. 1 dirs total, 0 dirs full, 0 dirs failed
I20260923 22:16:09.895478 19196 tablet_service.cc:1467] Processing CreateTablet for tablet 6c660fe5406c4f4193e901eb27d32379 (DEFAULT_TABLE table=test-table [id=2851d7eeb93b43f18125bc549a197e2a]), partition=RANGE (key) PARTITION UNBOUNDED
I20260923 22:16:09.895807 19196 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 6c660fe5406c4f4193e901eb27d32379. 1 dirs total, 0 dirs full, 0 dirs failed
I20260923 22:16:09.898067 19538 tablet_bootstrap.cc:492] T 6c660fe5406c4f4193e901eb27d32379 P 708230f249c6409c8dcc3eee79c7f370: Bootstrap starting.
I20260923 22:16:09.898654 19538 tablet_bootstrap.cc:654] T 6c660fe5406c4f4193e901eb27d32379 P 708230f249c6409c8dcc3eee79c7f370: Neither blocks nor log segments found. Creating new log.
I20260923 22:16:09.898955 19538 log.cc:824] T 6c660fe5406c4f4193e901eb27d32379 P 708230f249c6409c8dcc3eee79c7f370: Log is configured to *not* fsync() on all Append() calls
I20260923 22:16:09.899372 19540 tablet_bootstrap.cc:492] T 6c660fe5406c4f4193e901eb27d32379 P 097e47bb0fba4a90aac02fb28166828c: Bootstrap starting.
I20260923 22:16:09.899711 19538 tablet_bootstrap.cc:492] T 6c660fe5406c4f4193e901eb27d32379 P 708230f249c6409c8dcc3eee79c7f370: No bootstrap required, opened a new log
I20260923 22:16:09.899773 19538 ts_tablet_manager.cc:1397] T 6c660fe5406c4f4193e901eb27d32379 P 708230f249c6409c8dcc3eee79c7f370: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20260923 22:16:09.899892 19541 tablet_bootstrap.cc:492] T 6c660fe5406c4f4193e901eb27d32379 P bc06d877919f41a6974c66c7ebc04461: Bootstrap starting.
I20260923 22:16:09.900018 19540 tablet_bootstrap.cc:654] T 6c660fe5406c4f4193e901eb27d32379 P 097e47bb0fba4a90aac02fb28166828c: Neither blocks nor log segments found. Creating new log.
I20260923 22:16:09.900362 19540 log.cc:824] T 6c660fe5406c4f4193e901eb27d32379 P 097e47bb0fba4a90aac02fb28166828c: Log is configured to *not* fsync() on all Append() calls
I20260923 22:16:09.900521 19541 tablet_bootstrap.cc:654] T 6c660fe5406c4f4193e901eb27d32379 P bc06d877919f41a6974c66c7ebc04461: Neither blocks nor log segments found. Creating new log.
I20260923 22:16:09.900942 19541 log.cc:824] T 6c660fe5406c4f4193e901eb27d32379 P bc06d877919f41a6974c66c7ebc04461: Log is configured to *not* fsync() on all Append() calls
I20260923 22:16:09.901651 19541 tablet_bootstrap.cc:492] T 6c660fe5406c4f4193e901eb27d32379 P bc06d877919f41a6974c66c7ebc04461: No bootstrap required, opened a new log
I20260923 22:16:09.901736 19541 ts_tablet_manager.cc:1397] T 6c660fe5406c4f4193e901eb27d32379 P bc06d877919f41a6974c66c7ebc04461: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20260923 22:16:09.902177 19538 raft_consensus.cc:348] T 6c660fe5406c4f4193e901eb27d32379 P 708230f249c6409c8dcc3eee79c7f370 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bc06d877919f41a6974c66c7ebc04461" member_type: VOTER last_known_addr { host: "127.1.135.194" port: 46683 } } peers { permanent_uuid: "708230f249c6409c8dcc3eee79c7f370" member_type: VOTER last_known_addr { host: "127.1.135.195" port: 42159 } } peers { permanent_uuid: "097e47bb0fba4a90aac02fb28166828c" member_type: VOTER last_known_addr { host: "127.1.135.193" port: 44509 } }
I20260923 22:16:09.902323 19538 raft_consensus.cc:374] T 6c660fe5406c4f4193e901eb27d32379 P 708230f249c6409c8dcc3eee79c7f370 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260923 22:16:09.902347 19538 raft_consensus.cc:729] T 6c660fe5406c4f4193e901eb27d32379 P 708230f249c6409c8dcc3eee79c7f370 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 708230f249c6409c8dcc3eee79c7f370, State: Initialized, Role: FOLLOWER
I20260923 22:16:09.902474 19538 consensus_queue.cc:260] T 6c660fe5406c4f4193e901eb27d32379 P 708230f249c6409c8dcc3eee79c7f370 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bc06d877919f41a6974c66c7ebc04461" member_type: VOTER last_known_addr { host: "127.1.135.194" port: 46683 } } peers { permanent_uuid: "708230f249c6409c8dcc3eee79c7f370" member_type: VOTER last_known_addr { host: "127.1.135.195" port: 42159 } } peers { permanent_uuid: "097e47bb0fba4a90aac02fb28166828c" member_type: VOTER last_known_addr { host: "127.1.135.193" port: 44509 } }
I20260923 22:16:09.902748 19538 ts_tablet_manager.cc:1428] T 6c660fe5406c4f4193e901eb27d32379 P 708230f249c6409c8dcc3eee79c7f370: Time spent starting tablet: real 0.003s user 0.003s sys 0.000s
I20260923 22:16:09.902817 19524 heartbeater.cc:503] Master 127.1.135.254:43673 was elected leader, sending a full tablet report...
I20260923 22:16:09.903321 19540 tablet_bootstrap.cc:492] T 6c660fe5406c4f4193e901eb27d32379 P 097e47bb0fba4a90aac02fb28166828c: No bootstrap required, opened a new log
I20260923 22:16:09.903379 19540 ts_tablet_manager.cc:1397] T 6c660fe5406c4f4193e901eb27d32379 P 097e47bb0fba4a90aac02fb28166828c: Time spent bootstrapping tablet: real 0.004s user 0.001s sys 0.000s
I20260923 22:16:09.904290 19541 raft_consensus.cc:348] T 6c660fe5406c4f4193e901eb27d32379 P bc06d877919f41a6974c66c7ebc04461 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bc06d877919f41a6974c66c7ebc04461" member_type: VOTER last_known_addr { host: "127.1.135.194" port: 46683 } } peers { permanent_uuid: "708230f249c6409c8dcc3eee79c7f370" member_type: VOTER last_known_addr { host: "127.1.135.195" port: 42159 } } peers { permanent_uuid: "097e47bb0fba4a90aac02fb28166828c" member_type: VOTER last_known_addr { host: "127.1.135.193" port: 44509 } }
I20260923 22:16:09.904438 19541 raft_consensus.cc:374] T 6c660fe5406c4f4193e901eb27d32379 P bc06d877919f41a6974c66c7ebc04461 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260923 22:16:09.904480 19541 raft_consensus.cc:729] T 6c660fe5406c4f4193e901eb27d32379 P bc06d877919f41a6974c66c7ebc04461 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: bc06d877919f41a6974c66c7ebc04461, State: Initialized, Role: FOLLOWER
I20260923 22:16:09.904619 19541 consensus_queue.cc:260] T 6c660fe5406c4f4193e901eb27d32379 P bc06d877919f41a6974c66c7ebc04461 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bc06d877919f41a6974c66c7ebc04461" member_type: VOTER last_known_addr { host: "127.1.135.194" port: 46683 } } peers { permanent_uuid: "708230f249c6409c8dcc3eee79c7f370" member_type: VOTER last_known_addr { host: "127.1.135.195" port: 42159 } } peers { permanent_uuid: "097e47bb0fba4a90aac02fb28166828c" member_type: VOTER last_known_addr { host: "127.1.135.193" port: 44509 } }
I20260923 22:16:09.904935 19541 ts_tablet_manager.cc:1428] T 6c660fe5406c4f4193e901eb27d32379 P bc06d877919f41a6974c66c7ebc04461: Time spent starting tablet: real 0.003s user 0.003s sys 0.000s
I20260923 22:16:09.905676 19540 raft_consensus.cc:348] T 6c660fe5406c4f4193e901eb27d32379 P 097e47bb0fba4a90aac02fb28166828c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bc06d877919f41a6974c66c7ebc04461" member_type: VOTER last_known_addr { host: "127.1.135.194" port: 46683 } } peers { permanent_uuid: "708230f249c6409c8dcc3eee79c7f370" member_type: VOTER last_known_addr { host: "127.1.135.195" port: 42159 } } peers { permanent_uuid: "097e47bb0fba4a90aac02fb28166828c" member_type: VOTER last_known_addr { host: "127.1.135.193" port: 44509 } }
I20260923 22:16:09.905814 19540 raft_consensus.cc:374] T 6c660fe5406c4f4193e901eb27d32379 P 097e47bb0fba4a90aac02fb28166828c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260923 22:16:09.905836 19540 raft_consensus.cc:729] T 6c660fe5406c4f4193e901eb27d32379 P 097e47bb0fba4a90aac02fb28166828c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 097e47bb0fba4a90aac02fb28166828c, State: Initialized, Role: FOLLOWER
I20260923 22:16:09.905946 19540 consensus_queue.cc:260] T 6c660fe5406c4f4193e901eb27d32379 P 097e47bb0fba4a90aac02fb28166828c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bc06d877919f41a6974c66c7ebc04461" member_type: VOTER last_known_addr { host: "127.1.135.194" port: 46683 } } peers { permanent_uuid: "708230f249c6409c8dcc3eee79c7f370" member_type: VOTER last_known_addr { host: "127.1.135.195" port: 42159 } } peers { permanent_uuid: "097e47bb0fba4a90aac02fb28166828c" member_type: VOTER last_known_addr { host: "127.1.135.193" port: 44509 } }
I20260923 22:16:09.906219 19540 ts_tablet_manager.cc:1428] T 6c660fe5406c4f4193e901eb27d32379 P 097e47bb0fba4a90aac02fb28166828c: Time spent starting tablet: real 0.003s user 0.003s sys 0.000s
I20260923 22:16:09.906246 19262 heartbeater.cc:503] Master 127.1.135.254:43673 was elected leader, sending a full tablet report...
I20260923 22:16:09.906353 19393 heartbeater.cc:503] Master 127.1.135.254:43673 was elected leader, sending a full tablet report...
I20260923 22:16:09.912205 19545 raft_consensus.cc:482] T 6c660fe5406c4f4193e901eb27d32379 P bc06d877919f41a6974c66c7ebc04461 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260923 22:16:09.912290 19545 raft_consensus.cc:504] T 6c660fe5406c4f4193e901eb27d32379 P bc06d877919f41a6974c66c7ebc04461 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bc06d877919f41a6974c66c7ebc04461" member_type: VOTER last_known_addr { host: "127.1.135.194" port: 46683 } } peers { permanent_uuid: "708230f249c6409c8dcc3eee79c7f370" member_type: VOTER last_known_addr { host: "127.1.135.195" port: 42159 } } peers { permanent_uuid: "097e47bb0fba4a90aac02fb28166828c" member_type: VOTER last_known_addr { host: "127.1.135.193" port: 44509 } }
I20260923 22:16:09.912719 19545 leader_election.cc:288] T 6c660fe5406c4f4193e901eb27d32379 P bc06d877919f41a6974c66c7ebc04461 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 708230f249c6409c8dcc3eee79c7f370 (127.1.135.195:42159), 097e47bb0fba4a90aac02fb28166828c (127.1.135.193:44509)
I20260923 22:16:09.916461 19395 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 22:16:09.912843 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.135.195:42159 (local address 127.1.135.194:49791)
0923 22:16:09.912891 (+ 48us) negotiation.cc:107] Waiting for socket to connect
0923 22:16:09.912904 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0923 22:16:09.912946 (+ 42us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0923 22:16:09.913144 (+ 198us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0923 22:16:09.913145 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0923 22:16:09.913147 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0923 22:16:09.913215 (+ 68us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 22:16:09.913218 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 22:16:09.914143 (+ 925us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 22:16:09.914144 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 22:16:09.915450 (+ 1306us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 22:16:09.915453 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 22:16:09.916194 (+ 741us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 22:16:09.916195 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 22:16:09.916271 (+ 76us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 22:16:09.916275 (+ 4us) client_negotiation.cc:769] Sending connection context
0923 22:16:09.916334 (+ 59us) client_negotiation.cc:240] Negotiation successful
0923 22:16:09.916381 (+ 47us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":18,"mutex_wait_us":28}
I20260923 22:16:09.916472 19534 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 22:16:09.912960 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.135.194:49791 (local address 127.1.135.195:42159)
0923 22:16:09.913009 (+ 49us) server_negotiation.cc:206] Beginning negotiation
0923 22:16:09.913012 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0923 22:16:09.913021 (+ 9us) server_negotiation.cc:407] Connection header received
0923 22:16:09.913051 (+ 30us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 22:16:09.913052 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 22:16:09.913063 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 22:16:09.913126 (+ 63us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0923 22:16:09.913307 (+ 181us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 22:16:09.914062 (+ 755us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 22:16:09.915545 (+ 1483us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 22:16:09.916131 (+ 586us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 22:16:09.916192 (+ 61us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 22:16:09.916201 (+ 9us) server_negotiation.cc:1035] Waiting for connection context
0923 22:16:09.916351 (+ 150us) server_negotiation.cc:299] Negotiation successful
0923 22:16:09.916390 (+ 39us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":23}
I20260923 22:16:09.916553 19537 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 22:16:09.912942 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.135.194:34637 (local address 127.1.135.193:44509)
0923 22:16:09.913059 (+ 117us) server_negotiation.cc:206] Beginning negotiation
0923 22:16:09.913061 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0923 22:16:09.913095 (+ 34us) server_negotiation.cc:407] Connection header received
0923 22:16:09.913164 (+ 69us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 22:16:09.913165 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 22:16:09.913174 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 22:16:09.913220 (+ 46us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0923 22:16:09.913365 (+ 145us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 22:16:09.914058 (+ 693us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 22:16:09.915546 (+ 1488us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 22:16:09.916116 (+ 570us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 22:16:09.916192 (+ 76us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 22:16:09.916201 (+ 9us) server_negotiation.cc:1035] Waiting for connection context
0923 22:16:09.916434 (+ 233us) server_negotiation.cc:299] Negotiation successful
0923 22:16:09.916481 (+ 47us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":96}
I20260923 22:16:09.916757 19547 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 22:16:09.912912 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.135.193:44509 (local address 127.1.135.194:34637)
0923 22:16:09.913050 (+ 138us) negotiation.cc:107] Waiting for socket to connect
0923 22:16:09.913062 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0923 22:16:09.913097 (+ 35us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0923 22:16:09.913235 (+ 138us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0923 22:16:09.913235 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0923 22:16:09.913236 (+ 1us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0923 22:16:09.913293 (+ 57us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 22:16:09.913294 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 22:16:09.914143 (+ 849us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 22:16:09.914144 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 22:16:09.915449 (+ 1305us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 22:16:09.915453 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 22:16:09.916196 (+ 743us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 22:16:09.916196 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 22:16:09.916279 (+ 83us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 22:16:09.916281 (+ 2us) client_negotiation.cc:769] Sending connection context
0923 22:16:09.916653 (+ 372us) client_negotiation.cc:240] Negotiation successful
0923 22:16:09.916692 (+ 39us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":122,"mutex_wait_us":74,"thread_start_us":96,"threads_started":1}
I20260923 22:16:09.916775 19478 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "6c660fe5406c4f4193e901eb27d32379" candidate_uuid: "bc06d877919f41a6974c66c7ebc04461" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "708230f249c6409c8dcc3eee79c7f370" is_pre_election: true
I20260923 22:16:09.916940 19478 raft_consensus.cc:2454] T 6c660fe5406c4f4193e901eb27d32379 P 708230f249c6409c8dcc3eee79c7f370 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate bc06d877919f41a6974c66c7ebc04461 in term 0.
I20260923 22:16:09.917064 19216 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "6c660fe5406c4f4193e901eb27d32379" candidate_uuid: "bc06d877919f41a6974c66c7ebc04461" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "097e47bb0fba4a90aac02fb28166828c" is_pre_election: true
I20260923 22:16:09.917179 19283 leader_election.cc:302] T 6c660fe5406c4f4193e901eb27d32379 P bc06d877919f41a6974c66c7ebc04461 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 708230f249c6409c8dcc3eee79c7f370, bc06d877919f41a6974c66c7ebc04461; no voters:
I20260923 22:16:09.917220 19216 raft_consensus.cc:2454] T 6c660fe5406c4f4193e901eb27d32379 P 097e47bb0fba4a90aac02fb28166828c [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate bc06d877919f41a6974c66c7ebc04461 in term 0.
I20260923 22:16:09.917295 19545 raft_consensus.cc:2781] T 6c660fe5406c4f4193e901eb27d32379 P bc06d877919f41a6974c66c7ebc04461 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260923 22:16:09.917354 19545 raft_consensus.cc:482] T 6c660fe5406c4f4193e901eb27d32379 P bc06d877919f41a6974c66c7ebc04461 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260923 22:16:09.917388 19545 raft_consensus.cc:3037] T 6c660fe5406c4f4193e901eb27d32379 P bc06d877919f41a6974c66c7ebc04461 [term 0 FOLLOWER]: Advancing to term 1
I20260923 22:16:09.918102 19545 raft_consensus.cc:504] T 6c660fe5406c4f4193e901eb27d32379 P bc06d877919f41a6974c66c7ebc04461 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bc06d877919f41a6974c66c7ebc04461" member_type: VOTER last_known_addr { host: "127.1.135.194" port: 46683 } } peers { permanent_uuid: "708230f249c6409c8dcc3eee79c7f370" member_type: VOTER last_known_addr { host: "127.1.135.195" port: 42159 } } peers { permanent_uuid: "097e47bb0fba4a90aac02fb28166828c" member_type: VOTER last_known_addr { host: "127.1.135.193" port: 44509 } }
I20260923 22:16:09.918251 19545 leader_election.cc:288] T 6c660fe5406c4f4193e901eb27d32379 P bc06d877919f41a6974c66c7ebc04461 [CANDIDATE]: Term 1 election: Requested vote from peers 708230f249c6409c8dcc3eee79c7f370 (127.1.135.195:42159), 097e47bb0fba4a90aac02fb28166828c (127.1.135.193:44509)
I20260923 22:16:09.918392 19478 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "6c660fe5406c4f4193e901eb27d32379" candidate_uuid: "bc06d877919f41a6974c66c7ebc04461" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "708230f249c6409c8dcc3eee79c7f370"
I20260923 22:16:09.918414 19216 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "6c660fe5406c4f4193e901eb27d32379" candidate_uuid: "bc06d877919f41a6974c66c7ebc04461" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "097e47bb0fba4a90aac02fb28166828c"
I20260923 22:16:09.918481 19216 raft_consensus.cc:3037] T 6c660fe5406c4f4193e901eb27d32379 P 097e47bb0fba4a90aac02fb28166828c [term 0 FOLLOWER]: Advancing to term 1
I20260923 22:16:09.918481 19478 raft_consensus.cc:3037] T 6c660fe5406c4f4193e901eb27d32379 P 708230f249c6409c8dcc3eee79c7f370 [term 0 FOLLOWER]: Advancing to term 1
I20260923 22:16:09.919384 19478 raft_consensus.cc:2454] T 6c660fe5406c4f4193e901eb27d32379 P 708230f249c6409c8dcc3eee79c7f370 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate bc06d877919f41a6974c66c7ebc04461 in term 1.
I20260923 22:16:09.919391 19216 raft_consensus.cc:2454] T 6c660fe5406c4f4193e901eb27d32379 P 097e47bb0fba4a90aac02fb28166828c [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate bc06d877919f41a6974c66c7ebc04461 in term 1.
I20260923 22:16:09.919572 19282 leader_election.cc:302] T 6c660fe5406c4f4193e901eb27d32379 P bc06d877919f41a6974c66c7ebc04461 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 097e47bb0fba4a90aac02fb28166828c, bc06d877919f41a6974c66c7ebc04461; no voters:
I20260923 22:16:09.919700 19545 raft_consensus.cc:2781] T 6c660fe5406c4f4193e901eb27d32379 P bc06d877919f41a6974c66c7ebc04461 [term 1 FOLLOWER]: Leader election won for term 1
I20260923 22:16:09.919864 19545 raft_consensus.cc:686] T 6c660fe5406c4f4193e901eb27d32379 P bc06d877919f41a6974c66c7ebc04461 [term 1 LEADER]: Becoming Leader. State: Replica: bc06d877919f41a6974c66c7ebc04461, State: Running, Role: LEADER
I20260923 22:16:09.919971 19545 consensus_queue.cc:237] T 6c660fe5406c4f4193e901eb27d32379 P bc06d877919f41a6974c66c7ebc04461 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bc06d877919f41a6974c66c7ebc04461" member_type: VOTER last_known_addr { host: "127.1.135.194" port: 46683 } } peers { permanent_uuid: "708230f249c6409c8dcc3eee79c7f370" member_type: VOTER last_known_addr { host: "127.1.135.195" port: 42159 } } peers { permanent_uuid: "097e47bb0fba4a90aac02fb28166828c" member_type: VOTER last_known_addr { host: "127.1.135.193" port: 44509 } }
I20260923 22:16:09.920821 19073 catalog_manager.cc:5495] T 6c660fe5406c4f4193e901eb27d32379 P bc06d877919f41a6974c66c7ebc04461 reported cstate change: term changed from 0 to 1, leader changed from <none> to bc06d877919f41a6974c66c7ebc04461 (127.1.135.194). New cstate: current_term: 1 leader_uuid: "bc06d877919f41a6974c66c7ebc04461" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bc06d877919f41a6974c66c7ebc04461" member_type: VOTER last_known_addr { host: "127.1.135.194" port: 46683 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "708230f249c6409c8dcc3eee79c7f370" member_type: VOTER last_known_addr { host: "127.1.135.195" port: 42159 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "097e47bb0fba4a90aac02fb28166828c" member_type: VOTER last_known_addr { host: "127.1.135.193" port: 44509 } health_report { overall_health: UNKNOWN } } }
I20260923 22:16:09.927971 19535 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 22:16:09.925947 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37190 (local address 127.1.135.194:46683)
0923 22:16:09.925987 (+ 40us) server_negotiation.cc:206] Beginning negotiation
0923 22:16:09.925989 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0923 22:16:09.926016 (+ 27us) server_negotiation.cc:407] Connection header received
0923 22:16:09.926076 (+ 60us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 22:16:09.926077 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 22:16:09.926092 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 22:16:09.926147 (+ 55us) server_negotiation.cc:226] Negotiated authn=TOKEN
0923 22:16:09.926257 (+ 110us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 22:16:09.926699 (+ 442us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 22:16:09.927370 (+ 671us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 22:16:09.927537 (+ 167us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 22:16:09.927584 (+ 47us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 22:16:09.927706 (+ 122us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0923 22:16:09.927795 (+ 89us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0923 22:16:09.927853 (+ 58us) server_negotiation.cc:1035] Waiting for connection context
0923 22:16:09.927888 (+ 35us) server_negotiation.cc:299] Negotiation successful
0923 22:16:09.927903 (+ 15us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":17}
I20260923 22:16:09.929693 19216 raft_consensus.cc:1261] T 6c660fe5406c4f4193e901eb27d32379 P 097e47bb0fba4a90aac02fb28166828c [term 1 FOLLOWER]: Refusing update from remote peer bc06d877919f41a6974c66c7ebc04461: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260923 22:16:09.929736 19478 raft_consensus.cc:1261] T 6c660fe5406c4f4193e901eb27d32379 P 708230f249c6409c8dcc3eee79c7f370 [term 1 FOLLOWER]: Refusing update from remote peer bc06d877919f41a6974c66c7ebc04461: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260923 22:16:09.930055 19548 consensus_queue.cc:1035] T 6c660fe5406c4f4193e901eb27d32379 P bc06d877919f41a6974c66c7ebc04461 [LEADER]: Connected to new peer: Peer: permanent_uuid: "097e47bb0fba4a90aac02fb28166828c" member_type: VOTER last_known_addr { host: "127.1.135.193" port: 44509 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260923 22:16:09.930238 19545 consensus_queue.cc:1035] T 6c660fe5406c4f4193e901eb27d32379 P bc06d877919f41a6974c66c7ebc04461 [LEADER]: Connected to new peer: Peer: permanent_uuid: "708230f249c6409c8dcc3eee79c7f370" member_type: VOTER last_known_addr { host: "127.1.135.195" port: 42159 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260923 22:16:09.932765 19551 mvcc.cc:205] Tried to move back new op lower bound from 7332666449629454336 to 7332666449593024512. Current Snapshot: MvccSnapshot[applied={T|T < 7332666449629454336 or (T in {7332666449629454336})}]
I20260923 22:16:09.934667 19073 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:55658:
table { table_name: "test-table" } modify_external_catalogs: true
I20260923 22:16:09.934759 19073 catalog_manager.cc:2645] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:55658:
table { table_name: "test-table" } modify_external_catalogs: true
I20260923 22:16:09.935497 19073 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 2b2e1e684715488fb0c19715737e4bd7: Sending DeleteTablet for 3 replicas of tablet 6c660fe5406c4f4193e901eb27d32379
I20260923 22:16:09.935832 19196 tablet_service.cc:1514] Processing DeleteTablet for tablet 6c660fe5406c4f4193e901eb27d32379 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-23 22:16:09 UTC) from {username='slave'} at 127.0.0.1:46672
I20260923 22:16:09.935829 19458 tablet_service.cc:1514] Processing DeleteTablet for tablet 6c660fe5406c4f4193e901eb27d32379 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-23 22:16:09 UTC) from {username='slave'} at 127.0.0.1:43738
I20260923 22:16:09.936116 19327 tablet_service.cc:1514] Processing DeleteTablet for tablet 6c660fe5406c4f4193e901eb27d32379 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-23 22:16:09 UTC) from {username='slave'} at 127.0.0.1:37182
I20260923 22:16:09.936143 1567 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskRiQe_a/build/release/bin/kudu with pid 19134
I20260923 22:16:09.937309 19560 tablet_replica.cc:330] T 6c660fe5406c4f4193e901eb27d32379 P 708230f249c6409c8dcc3eee79c7f370: stopping tablet replica
I20260923 22:16:09.937419 19560 raft_consensus.cc:2229] T 6c660fe5406c4f4193e901eb27d32379 P 708230f249c6409c8dcc3eee79c7f370 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260923 22:16:09.937525 19560 raft_consensus.cc:2258] T 6c660fe5406c4f4193e901eb27d32379 P 708230f249c6409c8dcc3eee79c7f370 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260923 22:16:09.937947 19560 ts_tablet_manager.cc:1905] T 6c660fe5406c4f4193e901eb27d32379 P 708230f249c6409c8dcc3eee79c7f370: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260923 22:16:09.939833 19560 ts_tablet_manager.cc:1918] T 6c660fe5406c4f4193e901eb27d32379 P 708230f249c6409c8dcc3eee79c7f370: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260923 22:16:09.939973 19560 log.cc:1196] T 6c660fe5406c4f4193e901eb27d32379 P 708230f249c6409c8dcc3eee79c7f370: Deleting WAL directory at /tmp/dist-test-taskRiQe_a/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790201539513534-1567-0/minicluster-data/ts-2/wal/wals/6c660fe5406c4f4193e901eb27d32379
I20260923 22:16:09.940371 19560 ts_tablet_manager.cc:1939] T 6c660fe5406c4f4193e901eb27d32379 P 708230f249c6409c8dcc3eee79c7f370: Deleting consensus metadata
I20260923 22:16:09.940852 19061 catalog_manager.cc:4841] TS 708230f249c6409c8dcc3eee79c7f370 (127.1.135.195:42159): tablet 6c660fe5406c4f4193e901eb27d32379 (table test-table [id=2851d7eeb93b43f18125bc549a197e2a]) successfully deleted
W20260923 22:16:09.941942 19060 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.1.135.193:44509 (error 108)
I20260923 22:16:09.942112 19562 tablet_replica.cc:330] T 6c660fe5406c4f4193e901eb27d32379 P bc06d877919f41a6974c66c7ebc04461: stopping tablet replica
I20260923 22:16:09.942215 19562 raft_consensus.cc:2229] T 6c660fe5406c4f4193e901eb27d32379 P bc06d877919f41a6974c66c7ebc04461 [term 1 LEADER]: Raft consensus shutting down.
I20260923 22:16:09.942337 19562 raft_consensus.cc:2258] T 6c660fe5406c4f4193e901eb27d32379 P bc06d877919f41a6974c66c7ebc04461 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260923 22:16:09.942572 19536 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 22:16:09.942444 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.135.193:44509 (local address 127.0.0.1:46674)
0923 22:16:09.942490 (+ 46us) negotiation.cc:107] Waiting for socket to connect
0923 22:16:09.942536 (+ 46us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.135.193:44509: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":24}
I20260923 22:16:09.942564 1567 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskRiQe_a/build/release/bin/kudu with pid 19265
I20260923 22:16:09.942780 19562 ts_tablet_manager.cc:1905] T 6c660fe5406c4f4193e901eb27d32379 P bc06d877919f41a6974c66c7ebc04461: Deleting tablet data with delete state TABLET_DATA_DELETED
W20260923 22:16:09.943249 19060 catalog_manager.cc:4584] TS 097e47bb0fba4a90aac02fb28166828c (127.1.135.193:44509): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet 6c660fe5406c4f4193e901eb27d32379: Network error: Client connection negotiation failed: client connection to 127.1.135.193:44509: connect: Connection refused (error 111)
I20260923 22:16:09.947975 19536 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 22:16:09.947862 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.135.194:46683 (local address 127.0.0.1:37198)
0923 22:16:09.947908 (+ 46us) negotiation.cc:107] Waiting for socket to connect
0923 22:16:09.947934 (+ 26us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.135.194:46683: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":25}
I20260923 22:16:09.948223 1567 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskRiQe_a/build/release/bin/kudu with pid 19396
I20260923 22:16:09.953102 1567 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskRiQe_a/build/release/bin/kudu with pid 19043
2026-09-23T22:16:09Z chronyd exiting
[ OK ] EncryptionPolicyTest.TestEncryptionPolicy/1 (702 ms)
[ RUN ] EncryptionPolicyTest.TestEncryptionPolicy/2
Loading random data
Initializing database '/tmp/dist-test-taskRiQe_a/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790201539513534-1567-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Sep 23 22:16:10 dist-test-slave-s2fj krb5kdc[19566](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Sep 23 22:16:10 dist-test-slave-s2fj krb5kdc[19566](info): set up 2 sockets
Sep 23 22:16:10 dist-test-slave-s2fj krb5kdc[19566](info): commencing operation
krb5kdc: starting...
W20260923 22:16:10.971871 1567 mini_kdc.cc:121] Time spent starting KDC: real 0.999s user 0.000s sys 0.005s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskRiQe_a/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790201539513534-1567-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskRiQe_a/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790201539513534-1567-0/krb5kdc/test-user.keytab.
Sep 23 22:16:11 dist-test-slave-s2fj krb5kdc[19566](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790201771, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-09-23T22:16:11Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-09-23T22:16:11Z Disabled control of system clock
WARNING: no policy specified for kudu/127.1.135.254@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.1.135.254@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.1.135.254 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskRiQe_a/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790201539513534-1567-0/krb5kdc/kudu_127.1.135.254.keytab.
Entry for principal kudu/127.1.135.254 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskRiQe_a/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790201539513534-1567-0/krb5kdc/kudu_127.1.135.254.keytab.
I20260923 22:16:11.082540 1567 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskRiQe_a/build/release/bin/kudu
/tmp/dist-test-taskRiQe_a/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskRiQe_a/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790201539513534-1567-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskRiQe_a/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790201539513534-1567-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskRiQe_a/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790201539513534-1567-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskRiQe_a/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790201539513534-1567-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.1.135.254:44103
--webserver_interface=127.1.135.254
--webserver_port=0
--keytab_file=/tmp/dist-test-taskRiQe_a/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790201539513534-1567-0/krb5kdc/kudu_127.1.135.254.keytab
--principal=kudu/127.1.135.254
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.1.135.212:41637
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.1.135.254:44103
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskRiQe_a/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790201539513534-1567-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskRiQe_a/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790201539513534-1567-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskRiQe_a/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790201539513534-1567-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260923 22:16:11.159931 19579 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260923 22:16:11.160173 19579 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260923 22:16:11.160209 19579 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260923 22:16:11.161341 19579 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260923 22:16:11.161410 19579 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260923 22:16:11.161434 19579 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260923 22:16:11.161454 19579 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260923 22:16:11.161495 19579 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260923 22:16:11.163076 19579 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.1.135.212:41637
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskRiQe_a/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790201539513534-1567-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskRiQe_a/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790201539513534-1567-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.1.135.254:44103
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskRiQe_a/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790201539513534-1567-0/krb5kdc/kudu_127.1.135.254.keytab
--principal=kudu/127.1.135.254
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.1.135.254:44103
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskRiQe_a/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790201539513534-1567-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.1.135.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.19579
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskRiQe_a/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790201539513534-1567-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 23 Sep 2026 20:53:37 UTC on ccycloud.quasar-rkviyx.root.comops.site
build id 4668083
I20260923 22:16:11.163339 19579 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260923 22:16:11.163574 19579 file_cache.cc:493] Constructed file cache file cache with capacity 419430
I20260923 22:16:11.168325 19579 server_base.cc:1034] running on GCE node
W20260923 22:16:11.168308 19587 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260923 22:16:11.168299 19585 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260923 22:16:11.168308 19584 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260923 22:16:11.168816 19579 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260923 22:16:11.169114 19579 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260923 22:16:11.170276 19579 hybrid_clock.cc:648] HybridClock initialized: now 1790201771170266 us; error 45 us; skew 500 ppm
Sep 23 22:16:11 dist-test-slave-s2fj krb5kdc[19566](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790201771, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.135.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260923 22:16:11.172765 19579 init.cc:377] Logged in from keytab as kudu/127.1.135.254@KRBTEST.COM (short username kudu)
I20260923 22:16:11.173200 19579 webserver.cc:460] Webserver started at http://127.1.135.254:38997/ using document root <none> and password file <none>
I20260923 22:16:11.173381 19579 fs_manager.cc:359] Metadata directory not provided
I20260923 22:16:11.173422 19579 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260923 22:16:11.173574 19579 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260923 22:16:11.174546 19579 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskRiQe_a/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790201539513534-1567-0/minicluster-data/master-0/data/instance:
uuid: "a2024669c690445cbe6333e111b7cd96"
format_stamp: "Formatted at 2026-09-23 22:16:11 on dist-test-slave-s2fj"
server_key: "68e10bd53733efd12c307a0fd0018994"
server_key_iv: "a667625e13ae5bd1b5ca503dd34f0cc2"
server_key_version: "encryptionkey@0"
I20260923 22:16:11.174834 19579 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskRiQe_a/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790201539513534-1567-0/minicluster-data/master-0/wal/instance:
uuid: "a2024669c690445cbe6333e111b7cd96"
format_stamp: "Formatted at 2026-09-23 22:16:11 on dist-test-slave-s2fj"
server_key: "68e10bd53733efd12c307a0fd0018994"
server_key_iv: "a667625e13ae5bd1b5ca503dd34f0cc2"
server_key_version: "encryptionkey@0"
I20260923 22:16:11.176048 19579 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.001s
I20260923 22:16:11.177078 19594 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260923 22:16:11.177294 19579 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.001s
I20260923 22:16:11.177395 19579 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskRiQe_a/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790201539513534-1567-0/minicluster-data/master-0/data,/tmp/dist-test-taskRiQe_a/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790201539513534-1567-0/minicluster-data/master-0/wal
uuid: "a2024669c690445cbe6333e111b7cd96"
format_stamp: "Formatted at 2026-09-23 22:16:11 on dist-test-slave-s2fj"
server_key: "68e10bd53733efd12c307a0fd0018994"
server_key_iv: "a667625e13ae5bd1b5ca503dd34f0cc2"
server_key_version: "encryptionkey@0"
I20260923 22:16:11.177477 19579 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskRiQe_a/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790201539513534-1567-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskRiQe_a/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790201539513534-1567-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskRiQe_a/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790201539513534-1567-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260923 22:16:11.210589 19579 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260923 22:16:11.213014 19579 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260923 22:16:11.213130 19579 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260923 22:16:11.217083 19579 rpc_server.cc:307] RPC server started. Bound to: 127.1.135.254:44103
I20260923 22:16:11.217118 19646 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.1.135.254:44103 every 8 connection(s)
I20260923 22:16:11.217505 19579 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskRiQe_a/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790201539513534-1567-0/minicluster-data/master-0/data/info.pb
I20260923 22:16:11.218413 19647 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260923 22:16:11.220880 19647 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P a2024669c690445cbe6333e111b7cd96: Bootstrap starting.
I20260923 22:16:11.221655 19647 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P a2024669c690445cbe6333e111b7cd96: Neither blocks nor log segments found. Creating new log.
I20260923 22:16:11.222038 19647 log.cc:824] T 00000000000000000000000000000000 P a2024669c690445cbe6333e111b7cd96: Log is configured to *not* fsync() on all Append() calls
I20260923 22:16:11.222813 19647 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P a2024669c690445cbe6333e111b7cd96: No bootstrap required, opened a new log
I20260923 22:16:11.224509 19647 raft_consensus.cc:348] T 00000000000000000000000000000000 P a2024669c690445cbe6333e111b7cd96 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a2024669c690445cbe6333e111b7cd96" member_type: VOTER last_known_addr { host: "127.1.135.254" port: 44103 } }
I20260923 22:16:11.224651 19647 raft_consensus.cc:374] T 00000000000000000000000000000000 P a2024669c690445cbe6333e111b7cd96 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260923 22:16:11.224696 19647 raft_consensus.cc:729] T 00000000000000000000000000000000 P a2024669c690445cbe6333e111b7cd96 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: a2024669c690445cbe6333e111b7cd96, State: Initialized, Role: FOLLOWER
I20260923 22:16:11.224853 19647 consensus_queue.cc:260] T 00000000000000000000000000000000 P a2024669c690445cbe6333e111b7cd96 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a2024669c690445cbe6333e111b7cd96" member_type: VOTER last_known_addr { host: "127.1.135.254" port: 44103 } }
I20260923 22:16:11.224933 19647 raft_consensus.cc:388] T 00000000000000000000000000000000 P a2024669c690445cbe6333e111b7cd96 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260923 22:16:11.224977 19647 raft_consensus.cc:482] T 00000000000000000000000000000000 P a2024669c690445cbe6333e111b7cd96 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260923 22:16:11.225026 19647 raft_consensus.cc:3037] T 00000000000000000000000000000000 P a2024669c690445cbe6333e111b7cd96 [term 0 FOLLOWER]: Advancing to term 1
I20260923 22:16:11.225564 19647 raft_consensus.cc:504] T 00000000000000000000000000000000 P a2024669c690445cbe6333e111b7cd96 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a2024669c690445cbe6333e111b7cd96" member_type: VOTER last_known_addr { host: "127.1.135.254" port: 44103 } }
I20260923 22:16:11.225679 19647 leader_election.cc:302] T 00000000000000000000000000000000 P a2024669c690445cbe6333e111b7cd96 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: a2024669c690445cbe6333e111b7cd96; no voters:
I20260923 22:16:11.225872 19647 leader_election.cc:288] T 00000000000000000000000000000000 P a2024669c690445cbe6333e111b7cd96 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260923 22:16:11.225916 19650 raft_consensus.cc:2781] T 00000000000000000000000000000000 P a2024669c690445cbe6333e111b7cd96 [term 1 FOLLOWER]: Leader election won for term 1
I20260923 22:16:11.226095 19650 raft_consensus.cc:686] T 00000000000000000000000000000000 P a2024669c690445cbe6333e111b7cd96 [term 1 LEADER]: Becoming Leader. State: Replica: a2024669c690445cbe6333e111b7cd96, State: Running, Role: LEADER
I20260923 22:16:11.226135 19647 sys_catalog.cc:564] T 00000000000000000000000000000000 P a2024669c690445cbe6333e111b7cd96 [sys.catalog]: configured and running, proceeding with master startup.
I20260923 22:16:11.226210 19650 consensus_queue.cc:237] T 00000000000000000000000000000000 P a2024669c690445cbe6333e111b7cd96 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a2024669c690445cbe6333e111b7cd96" member_type: VOTER last_known_addr { host: "127.1.135.254" port: 44103 } }
I20260923 22:16:11.226682 19652 sys_catalog.cc:455] T 00000000000000000000000000000000 P a2024669c690445cbe6333e111b7cd96 [sys.catalog]: SysCatalogTable state changed. Reason: New leader a2024669c690445cbe6333e111b7cd96. Latest consensus state: current_term: 1 leader_uuid: "a2024669c690445cbe6333e111b7cd96" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a2024669c690445cbe6333e111b7cd96" member_type: VOTER last_known_addr { host: "127.1.135.254" port: 44103 } } }
I20260923 22:16:11.226759 19652 sys_catalog.cc:458] T 00000000000000000000000000000000 P a2024669c690445cbe6333e111b7cd96 [sys.catalog]: This master's current role is: LEADER
I20260923 22:16:11.226848 19651 sys_catalog.cc:455] T 00000000000000000000000000000000 P a2024669c690445cbe6333e111b7cd96 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "a2024669c690445cbe6333e111b7cd96" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a2024669c690445cbe6333e111b7cd96" member_type: VOTER last_known_addr { host: "127.1.135.254" port: 44103 } } }
I20260923 22:16:11.226925 19651 sys_catalog.cc:458] T 00000000000000000000000000000000 P a2024669c690445cbe6333e111b7cd96 [sys.catalog]: This master's current role is: LEADER
I20260923 22:16:11.227084 19659 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260923 22:16:11.227141 1567 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskRiQe_a/build/release/bin/kudu as pid 19579
I20260923 22:16:11.227262 1567 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskRiQe_a/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790201539513534-1567-0/minicluster-data/master-0/wal/instance
I20260923 22:16:11.227504 1567 external_mini_cluster.cc:1428] Setting key 42cb21ff1d19c5fb061a5025fa2ba3be
I20260923 22:16:11.227685 19659 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260923 22:16:11.229463 19659 catalog_manager.cc:1347] Generated new cluster ID: 13d673ce7279454085e73bb75718e273
I20260923 22:16:11.229517 19659 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260923 22:16:11.233189 19659 catalog_manager.cc:1370] Generated new certificate authority record
I20260923 22:16:11.233851 19659 catalog_manager.cc:1504] Loading token signing keys...
Sep 23 22:16:11 dist-test-slave-s2fj krb5kdc[19566](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790201771, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.1.135.254@KRBTEST.COM
I20260923 22:16:11.239809 19669 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 22:16:11.228140 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60256 (local address 127.1.135.254:44103)
0923 22:16:11.228311 (+ 171us) server_negotiation.cc:206] Beginning negotiation
0923 22:16:11.228320 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0923 22:16:11.228363 (+ 43us) server_negotiation.cc:407] Connection header received
0923 22:16:11.228479 (+ 116us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 22:16:11.228484 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 22:16:11.228526 (+ 42us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 22:16:11.228675 (+ 149us) server_negotiation.cc:226] Negotiated authn=SASL
0923 22:16:11.229517 (+ 842us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 22:16:11.230954 (+ 1437us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 22:16:11.232020 (+ 1066us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 22:16:11.232689 (+ 669us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 22:16:11.232830 (+ 141us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 22:16:11.235794 (+ 2964us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0923 22:16:11.235798 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0923 22:16:11.235799 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0923 22:16:11.235814 (+ 15us) server_negotiation.cc:922] calling sasl_server_start()
0923 22:16:11.238088 (+ 2274us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0923 22:16:11.238477 (+ 389us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0923 22:16:11.238480 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0923 22:16:11.238480 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0923 22:16:11.238602 (+ 122us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0923 22:16:11.238804 (+ 202us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0923 22:16:11.238804 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0923 22:16:11.238805 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0923 22:16:11.239122 (+ 317us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0923 22:16:11.239277 (+ 155us) server_negotiation.cc:1035] Waiting for connection context
0923 22:16:11.239391 (+ 114us) server_negotiation.cc:299] Negotiation successful
0923 22:16:11.239560 (+ 169us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":129,"thread_start_us":81,"threads_started":1}
I20260923 22:16:11.240280 19659 catalog_manager.cc:5868] T 00000000000000000000000000000000 P a2024669c690445cbe6333e111b7cd96: Generated new TSK 0
I20260923 22:16:11.240558 19659 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.1.135.193@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.1.135.193@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.1.135.193 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskRiQe_a/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790201539513534-1567-0/krb5kdc/kudu_127.1.135.193.keytab.
Entry for principal kudu/127.1.135.193 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskRiQe_a/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790201539513534-1567-0/krb5kdc/kudu_127.1.135.193.keytab.
I20260923 22:16:11.270437 1567 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskRiQe_a/build/release/bin/kudu
/tmp/dist-test-taskRiQe_a/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskRiQe_a/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790201539513534-1567-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskRiQe_a/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790201539513534-1567-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskRiQe_a/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790201539513534-1567-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskRiQe_a/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790201539513534-1567-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.1.135.193:0
--local_ip_for_outbound_sockets=127.1.135.193
--webserver_interface=127.1.135.193
--webserver_port=0
--tserver_master_addrs=127.1.135.254:44103
--keytab_file=/tmp/dist-test-taskRiQe_a/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790201539513534-1567-0/krb5kdc/kudu_127.1.135.193.keytab
--principal=kudu/127.1.135.193
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.1.135.212:41637
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskRiQe_a/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790201539513534-1567-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskRiQe_a/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790201539513534-1567-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskRiQe_a/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790201539513534-1567-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260923 22:16:11.358457 19673 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260923 22:16:11.358713 19673 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260923 22:16:11.358755 19673 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260923 22:16:11.359948 19673 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260923 22:16:11.360020 19673 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260923 22:16:11.360070 19673 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.1.135.193
I20260923 22:16:11.361596 19673 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.1.135.212:41637
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskRiQe_a/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790201539513534-1567-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskRiQe_a/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790201539513534-1567-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskRiQe_a/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790201539513534-1567-0/krb5kdc/kudu_127.1.135.193.keytab
--principal=kudu/127.1.135.193
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.1.135.193:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskRiQe_a/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790201539513534-1567-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.1.135.193
--webserver_port=0
--tserver_master_addrs=127.1.135.254:44103
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.19673
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.1.135.193
--log_dir=/tmp/dist-test-taskRiQe_a/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790201539513534-1567-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 23 Sep 2026 20:53:37 UTC on ccycloud.quasar-rkviyx.root.comops.site
build id 4668083
I20260923 22:16:11.361874 19673 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260923 22:16:11.362112 19673 file_cache.cc:493] Constructed file cache file cache with capacity 419430
I20260923 22:16:11.368223 19673 server_base.cc:1034] running on GCE node
W20260923 22:16:11.368206 19679 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260923 22:16:11.368364 19678 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260923 22:16:11.368230 19681 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260923 22:16:11.368671 19673 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260923 22:16:11.368944 19673 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260923 22:16:11.370124 19673 hybrid_clock.cc:648] HybridClock initialized: now 1790201771370110 us; error 27 us; skew 500 ppm
Sep 23 22:16:11 dist-test-slave-s2fj krb5kdc[19566](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790201771, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.135.193@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260923 22:16:11.373117 19673 init.cc:377] Logged in from keytab as kudu/127.1.135.193@KRBTEST.COM (short username kudu)
I20260923 22:16:11.373657 19673 webserver.cc:460] Webserver started at http://127.1.135.193:38839/ using document root <none> and password file <none>
I20260923 22:16:11.373927 19673 fs_manager.cc:359] Metadata directory not provided
I20260923 22:16:11.374013 19673 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260923 22:16:11.374166 19673 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260923 22:16:11.375247 19673 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskRiQe_a/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790201539513534-1567-0/minicluster-data/ts-0/data/instance:
uuid: "1088b73602c14857a2ad641d87b2a5de"
format_stamp: "Formatted at 2026-09-23 22:16:11 on dist-test-slave-s2fj"
server_key: "20fcc6e8e73024f9926816333fd6cc90"
server_key_iv: "8f1129ccb3fbb8e545d0dccbf811f730"
server_key_version: "encryptionkey@0"
I20260923 22:16:11.375610 19673 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskRiQe_a/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790201539513534-1567-0/minicluster-data/ts-0/wal/instance:
uuid: "1088b73602c14857a2ad641d87b2a5de"
format_stamp: "Formatted at 2026-09-23 22:16:11 on dist-test-slave-s2fj"
server_key: "20fcc6e8e73024f9926816333fd6cc90"
server_key_iv: "8f1129ccb3fbb8e545d0dccbf811f730"
server_key_version: "encryptionkey@0"
I20260923 22:16:11.377038 19673 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.001s
I20260923 22:16:11.377790 19688 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260923 22:16:11.377975 19673 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.000s
I20260923 22:16:11.378074 19673 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskRiQe_a/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790201539513534-1567-0/minicluster-data/ts-0/data,/tmp/dist-test-taskRiQe_a/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790201539513534-1567-0/minicluster-data/ts-0/wal
uuid: "1088b73602c14857a2ad641d87b2a5de"
format_stamp: "Formatted at 2026-09-23 22:16:11 on dist-test-slave-s2fj"
server_key: "20fcc6e8e73024f9926816333fd6cc90"
server_key_iv: "8f1129ccb3fbb8e545d0dccbf811f730"
server_key_version: "encryptionkey@0"
I20260923 22:16:11.378150 19673 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskRiQe_a/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790201539513534-1567-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskRiQe_a/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790201539513534-1567-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskRiQe_a/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790201539513534-1567-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260923 22:16:11.387935 19673 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260923 22:16:11.389629 19673 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260923 22:16:11.389782 19673 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260923 22:16:11.389987 19673 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260923 22:16:11.390261 19673 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260923 22:16:11.390326 19673 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260923 22:16:11.390357 19673 ts_tablet_manager.cc:610] Registered 0 tablets
I20260923 22:16:11.390383 19673 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260923 22:16:11.396021 19673 rpc_server.cc:307] RPC server started. Bound to: 127.1.135.193:39039
I20260923 22:16:11.396095 19801 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.1.135.193:39039 every 8 connection(s)
I20260923 22:16:11.396380 19673 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskRiQe_a/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790201539513534-1567-0/minicluster-data/ts-0/data/info.pb
Sep 23 22:16:11 dist-test-slave-s2fj krb5kdc[19566](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790201771, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.135.193@KRBTEST.COM for kudu/127.1.135.254@KRBTEST.COM
I20260923 22:16:11.404875 19669 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 22:16:11.397038 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.135.193:44719 (local address 127.1.135.254:44103)
0923 22:16:11.397080 (+ 42us) server_negotiation.cc:206] Beginning negotiation
0923 22:16:11.397082 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0923 22:16:11.397280 (+ 198us) server_negotiation.cc:407] Connection header received
0923 22:16:11.397805 (+ 525us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 22:16:11.397807 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 22:16:11.397819 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 22:16:11.397877 (+ 58us) server_negotiation.cc:226] Negotiated authn=SASL
0923 22:16:11.398733 (+ 856us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 22:16:11.399246 (+ 513us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 22:16:11.400292 (+ 1046us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 22:16:11.400496 (+ 204us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 22:16:11.400572 (+ 76us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 22:16:11.402560 (+ 1988us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0923 22:16:11.402562 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0923 22:16:11.402562 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0923 22:16:11.402569 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0923 22:16:11.403717 (+ 1148us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0923 22:16:11.404047 (+ 330us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0923 22:16:11.404048 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0923 22:16:11.404048 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0923 22:16:11.404142 (+ 94us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0923 22:16:11.404349 (+ 207us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0923 22:16:11.404350 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0923 22:16:11.404350 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0923 22:16:11.404476 (+ 126us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0923 22:16:11.404538 (+ 62us) server_negotiation.cc:1035] Waiting for connection context
0923 22:16:11.404746 (+ 208us) server_negotiation.cc:299] Negotiation successful
0923 22:16:11.404809 (+ 63us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":19}
I20260923 22:16:11.404990 19804 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 22:16:11.397010 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.135.254:44103 (local address 127.1.135.193:44719)
0923 22:16:11.397150 (+ 140us) negotiation.cc:107] Waiting for socket to connect
0923 22:16:11.397202 (+ 52us) client_negotiation.cc:174] Beginning negotiation
0923 22:16:11.397699 (+ 497us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0923 22:16:11.397940 (+ 241us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0923 22:16:11.397944 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0923 22:16:11.398469 (+ 525us) client_negotiation.cc:189] Negotiated authn=SASL
0923 22:16:11.398635 (+ 166us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 22:16:11.398639 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 22:16:11.399349 (+ 710us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 22:16:11.399350 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 22:16:11.400194 (+ 844us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 22:16:11.400197 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 22:16:11.400578 (+ 381us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 22:16:11.400579 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 22:16:11.400717 (+ 138us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 22:16:11.401122 (+ 405us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0923 22:16:11.401129 (+ 7us) client_negotiation.cc:656] calling sasl_client_start()
0923 22:16:11.402477 (+ 1348us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0923 22:16:11.403784 (+ 1307us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0923 22:16:11.403790 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0923 22:16:11.403791 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0923 22:16:11.403986 (+ 195us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0923 22:16:11.404196 (+ 210us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0923 22:16:11.404196 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0923 22:16:11.404197 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0923 22:16:11.404288 (+ 91us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0923 22:16:11.404541 (+ 253us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0923 22:16:11.404545 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0923 22:16:11.404643 (+ 98us) client_negotiation.cc:769] Sending connection context
0923 22:16:11.404718 (+ 75us) client_negotiation.cc:240] Negotiation successful
0923 22:16:11.404824 (+ 106us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":104,"thread_start_us":71,"threads_started":1}
I20260923 22:16:11.405426 19802 heartbeater.cc:347] Connected to a master server at 127.1.135.254:44103
I20260923 22:16:11.405500 19802 heartbeater.cc:464] Registering TS with master...
I20260923 22:16:11.405681 19802 heartbeater.cc:511] Master 127.1.135.254:44103 requested a full tablet report, sending...
I20260923 22:16:11.406147 19611 ts_manager.cc:194] Registered new tserver with Master: 1088b73602c14857a2ad641d87b2a5de (127.1.135.193:39039)
I20260923 22:16:11.406409 1567 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskRiQe_a/build/release/bin/kudu as pid 19673
I20260923 22:16:11.406530 1567 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskRiQe_a/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790201539513534-1567-0/minicluster-data/ts-0/wal/instance
I20260923 22:16:11.406716 1567 external_mini_cluster.cc:1428] Setting key 0ad6ecc2cd1a0ed3b8423c1915fce6ba
I20260923 22:16:11.406888 19611 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.1.135.193@KRBTEST.COM'} at 127.1.135.193:44719
WARNING: no policy specified for kudu/127.1.135.194@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.1.135.194@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.1.135.194 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskRiQe_a/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790201539513534-1567-0/krb5kdc/kudu_127.1.135.194.keytab.
Entry for principal kudu/127.1.135.194 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskRiQe_a/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790201539513534-1567-0/krb5kdc/kudu_127.1.135.194.keytab.
I20260923 22:16:11.428987 1567 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskRiQe_a/build/release/bin/kudu
/tmp/dist-test-taskRiQe_a/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskRiQe_a/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790201539513534-1567-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskRiQe_a/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790201539513534-1567-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskRiQe_a/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790201539513534-1567-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskRiQe_a/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790201539513534-1567-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.1.135.194:0
--local_ip_for_outbound_sockets=127.1.135.194
--webserver_interface=127.1.135.194
--webserver_port=0
--tserver_master_addrs=127.1.135.254:44103
--keytab_file=/tmp/dist-test-taskRiQe_a/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790201539513534-1567-0/krb5kdc/kudu_127.1.135.194.keytab
--principal=kudu/127.1.135.194
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.1.135.212:41637
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskRiQe_a/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790201539513534-1567-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskRiQe_a/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790201539513534-1567-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskRiQe_a/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790201539513534-1567-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260923 22:16:11.507547 19807 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260923 22:16:11.507767 19807 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260923 22:16:11.507793 19807 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260923 22:16:11.509023 19807 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260923 22:16:11.509071 19807 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260923 22:16:11.509099 19807 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.1.135.194
I20260923 22:16:11.510617 19807 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.1.135.212:41637
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskRiQe_a/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790201539513534-1567-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskRiQe_a/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790201539513534-1567-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskRiQe_a/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790201539513534-1567-0/krb5kdc/kudu_127.1.135.194.keytab
--principal=kudu/127.1.135.194
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.1.135.194:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskRiQe_a/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790201539513534-1567-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.1.135.194
--webserver_port=0
--tserver_master_addrs=127.1.135.254:44103
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.19807
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.1.135.194
--log_dir=/tmp/dist-test-taskRiQe_a/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790201539513534-1567-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 23 Sep 2026 20:53:37 UTC on ccycloud.quasar-rkviyx.root.comops.site
build id 4668083
I20260923 22:16:11.510874 19807 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260923 22:16:11.511104 19807 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260923 22:16:11.516119 19812 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260923 22:16:11.516259 19807 server_base.cc:1034] running on GCE node
W20260923 22:16:11.516166 19815 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260923 22:16:11.516166 19813 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260923 22:16:11.516579 19807 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260923 22:16:11.516868 19807 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260923 22:16:11.518035 19807 hybrid_clock.cc:648] HybridClock initialized: now 1790201771518031 us; error 35 us; skew 500 ppm
Sep 23 22:16:11 dist-test-slave-s2fj krb5kdc[19566](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790201771, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.135.194@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260923 22:16:11.520598 19807 init.cc:377] Logged in from keytab as kudu/127.1.135.194@KRBTEST.COM (short username kudu)
I20260923 22:16:11.521078 19807 webserver.cc:460] Webserver started at http://127.1.135.194:38359/ using document root <none> and password file <none>
I20260923 22:16:11.521314 19807 fs_manager.cc:359] Metadata directory not provided
I20260923 22:16:11.521382 19807 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260923 22:16:11.521518 19807 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260923 22:16:11.522620 19807 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskRiQe_a/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790201539513534-1567-0/minicluster-data/ts-1/data/instance:
uuid: "33e26a8f637243aab1d372b50c990324"
format_stamp: "Formatted at 2026-09-23 22:16:11 on dist-test-slave-s2fj"
server_key: "0ef33b52170e60e58b1aab90f319dc05"
server_key_iv: "c89037eb192b4e4e3d1860297e0428b9"
server_key_version: "encryptionkey@0"
I20260923 22:16:11.523025 19807 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskRiQe_a/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790201539513534-1567-0/minicluster-data/ts-1/wal/instance:
uuid: "33e26a8f637243aab1d372b50c990324"
format_stamp: "Formatted at 2026-09-23 22:16:11 on dist-test-slave-s2fj"
server_key: "0ef33b52170e60e58b1aab90f319dc05"
server_key_iv: "c89037eb192b4e4e3d1860297e0428b9"
server_key_version: "encryptionkey@0"
I20260923 22:16:11.524444 19807 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.003s
I20260923 22:16:11.525190 19822 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260923 22:16:11.525360 19807 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.001s
I20260923 22:16:11.525467 19807 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskRiQe_a/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790201539513534-1567-0/minicluster-data/ts-1/data,/tmp/dist-test-taskRiQe_a/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790201539513534-1567-0/minicluster-data/ts-1/wal
uuid: "33e26a8f637243aab1d372b50c990324"
format_stamp: "Formatted at 2026-09-23 22:16:11 on dist-test-slave-s2fj"
server_key: "0ef33b52170e60e58b1aab90f319dc05"
server_key_iv: "c89037eb192b4e4e3d1860297e0428b9"
server_key_version: "encryptionkey@0"
I20260923 22:16:11.525542 19807 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskRiQe_a/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790201539513534-1567-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskRiQe_a/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790201539513534-1567-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskRiQe_a/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790201539513534-1567-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260923 22:16:11.539716 19807 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260923 22:16:11.541661 19807 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260923 22:16:11.541837 19807 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260923 22:16:11.542069 19807 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260923 22:16:11.542420 19807 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260923 22:16:11.542480 19807 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260923 22:16:11.542521 19807 ts_tablet_manager.cc:610] Registered 0 tablets
I20260923 22:16:11.542549 19807 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260923 22:16:11.548300 19807 rpc_server.cc:307] RPC server started. Bound to: 127.1.135.194:43765
I20260923 22:16:11.548357 19935 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.1.135.194:43765 every 8 connection(s)
I20260923 22:16:11.548672 19807 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskRiQe_a/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790201539513534-1567-0/minicluster-data/ts-1/data/info.pb
I20260923 22:16:11.553313 1567 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskRiQe_a/build/release/bin/kudu as pid 19807
I20260923 22:16:11.553457 1567 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskRiQe_a/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790201539513534-1567-0/minicluster-data/ts-1/wal/instance
I20260923 22:16:11.553678 1567 external_mini_cluster.cc:1428] Setting key 24d911783d244acfa13081bad933f62f
Sep 23 22:16:11 dist-test-slave-s2fj krb5kdc[19566](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790201771, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.135.194@KRBTEST.COM for kudu/127.1.135.254@KRBTEST.COM
I20260923 22:16:11.558357 19669 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 22:16:11.549362 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.135.194:57805 (local address 127.1.135.254:44103)
0923 22:16:11.549407 (+ 45us) server_negotiation.cc:206] Beginning negotiation
0923 22:16:11.549409 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0923 22:16:11.549619 (+ 210us) server_negotiation.cc:407] Connection header received
0923 22:16:11.550088 (+ 469us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 22:16:11.550089 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 22:16:11.550098 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 22:16:11.550159 (+ 61us) server_negotiation.cc:226] Negotiated authn=SASL
0923 22:16:11.551109 (+ 950us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 22:16:11.551579 (+ 470us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 22:16:11.552603 (+ 1024us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 22:16:11.552775 (+ 172us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 22:16:11.552831 (+ 56us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 22:16:11.555311 (+ 2480us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0923 22:16:11.555313 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0923 22:16:11.555313 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0923 22:16:11.555320 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0923 22:16:11.556958 (+ 1638us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0923 22:16:11.557375 (+ 417us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0923 22:16:11.557376 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0923 22:16:11.557376 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0923 22:16:11.557440 (+ 64us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0923 22:16:11.557699 (+ 259us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0923 22:16:11.557700 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0923 22:16:11.557700 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0923 22:16:11.557820 (+ 120us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0923 22:16:11.557895 (+ 75us) server_negotiation.cc:1035] Waiting for connection context
0923 22:16:11.558185 (+ 290us) server_negotiation.cc:299] Negotiation successful
0923 22:16:11.558263 (+ 78us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":17}
I20260923 22:16:11.558578 19938 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 22:16:11.549313 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.135.254:44103 (local address 127.1.135.194:57805)
0923 22:16:11.549482 (+ 169us) negotiation.cc:107] Waiting for socket to connect
0923 22:16:11.549524 (+ 42us) client_negotiation.cc:174] Beginning negotiation
0923 22:16:11.550004 (+ 480us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0923 22:16:11.550228 (+ 224us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0923 22:16:11.550231 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0923 22:16:11.550817 (+ 586us) client_negotiation.cc:189] Negotiated authn=SASL
0923 22:16:11.551031 (+ 214us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 22:16:11.551036 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 22:16:11.551667 (+ 631us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 22:16:11.551668 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 22:16:11.552529 (+ 861us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 22:16:11.552531 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 22:16:11.552843 (+ 312us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 22:16:11.552844 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 22:16:11.552973 (+ 129us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 22:16:11.553577 (+ 604us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0923 22:16:11.553589 (+ 12us) client_negotiation.cc:656] calling sasl_client_start()
0923 22:16:11.555200 (+ 1611us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0923 22:16:11.557036 (+ 1836us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0923 22:16:11.557044 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0923 22:16:11.557046 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0923 22:16:11.557274 (+ 228us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0923 22:16:11.557506 (+ 232us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0923 22:16:11.557507 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0923 22:16:11.557507 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0923 22:16:11.557622 (+ 115us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0923 22:16:11.557898 (+ 276us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0923 22:16:11.557905 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0923 22:16:11.558047 (+ 142us) client_negotiation.cc:769] Sending connection context
0923 22:16:11.558152 (+ 105us) client_negotiation.cc:240] Negotiation successful
0923 22:16:11.558330 (+ 178us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":128,"thread_start_us":91,"threads_started":1}
I20260923 22:16:11.559224 19936 heartbeater.cc:347] Connected to a master server at 127.1.135.254:44103
I20260923 22:16:11.559309 19936 heartbeater.cc:464] Registering TS with master...
I20260923 22:16:11.559521 19936 heartbeater.cc:511] Master 127.1.135.254:44103 requested a full tablet report, sending...
I20260923 22:16:11.559940 19610 ts_manager.cc:194] Registered new tserver with Master: 33e26a8f637243aab1d372b50c990324 (127.1.135.194:43765)
I20260923 22:16:11.560370 19610 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.1.135.194@KRBTEST.COM'} at 127.1.135.194:57805
WARNING: no policy specified for kudu/127.1.135.195@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.1.135.195@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.1.135.195 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskRiQe_a/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790201539513534-1567-0/krb5kdc/kudu_127.1.135.195.keytab.
Entry for principal kudu/127.1.135.195 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskRiQe_a/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790201539513534-1567-0/krb5kdc/kudu_127.1.135.195.keytab.
I20260923 22:16:11.578673 1567 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskRiQe_a/build/release/bin/kudu
/tmp/dist-test-taskRiQe_a/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskRiQe_a/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790201539513534-1567-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskRiQe_a/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790201539513534-1567-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskRiQe_a/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790201539513534-1567-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskRiQe_a/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790201539513534-1567-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.1.135.195:0
--local_ip_for_outbound_sockets=127.1.135.195
--webserver_interface=127.1.135.195
--webserver_port=0
--tserver_master_addrs=127.1.135.254:44103
--keytab_file=/tmp/dist-test-taskRiQe_a/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790201539513534-1567-0/krb5kdc/kudu_127.1.135.195.keytab
--principal=kudu/127.1.135.195
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.1.135.212:41637
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskRiQe_a/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790201539513534-1567-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskRiQe_a/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790201539513534-1567-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskRiQe_a/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790201539513534-1567-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260923 22:16:11.660691 19941 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260923 22:16:11.660903 19941 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260923 22:16:11.660928 19941 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260923 22:16:11.662016 19941 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260923 22:16:11.662051 19941 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260923 22:16:11.662076 19941 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.1.135.195
I20260923 22:16:11.663632 19941 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.1.135.212:41637
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskRiQe_a/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790201539513534-1567-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskRiQe_a/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790201539513534-1567-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskRiQe_a/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790201539513534-1567-0/krb5kdc/kudu_127.1.135.195.keytab
--principal=kudu/127.1.135.195
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.1.135.195:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskRiQe_a/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790201539513534-1567-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.1.135.195
--webserver_port=0
--tserver_master_addrs=127.1.135.254:44103
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.19941
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.1.135.195
--log_dir=/tmp/dist-test-taskRiQe_a/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790201539513534-1567-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 23 Sep 2026 20:53:37 UTC on ccycloud.quasar-rkviyx.root.comops.site
build id 4668083
I20260923 22:16:11.663823 19941 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260923 22:16:11.664002 19941 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260923 22:16:11.668928 19947 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260923 22:16:11.669050 19949 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260923 22:16:11.669227 19941 server_base.cc:1034] running on GCE node
W20260923 22:16:11.669081 19946 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260923 22:16:11.669482 19941 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260923 22:16:11.669771 19941 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260923 22:16:11.670943 19941 hybrid_clock.cc:648] HybridClock initialized: now 1790201771670938 us; error 34 us; skew 500 ppm
Sep 23 22:16:11 dist-test-slave-s2fj krb5kdc[19566](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790201771, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.135.195@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260923 22:16:11.673550 19941 init.cc:377] Logged in from keytab as kudu/127.1.135.195@KRBTEST.COM (short username kudu)
I20260923 22:16:11.674068 19941 webserver.cc:460] Webserver started at http://127.1.135.195:44133/ using document root <none> and password file <none>
I20260923 22:16:11.674314 19941 fs_manager.cc:359] Metadata directory not provided
I20260923 22:16:11.674384 19941 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260923 22:16:11.674528 19941 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260923 22:16:11.676029 19941 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskRiQe_a/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790201539513534-1567-0/minicluster-data/ts-2/data/instance:
uuid: "2ecd40dd495945ccba780072512499c2"
format_stamp: "Formatted at 2026-09-23 22:16:11 on dist-test-slave-s2fj"
server_key: "1bda25da1f55fb61c5c84ac41e332854"
server_key_iv: "157d698068c9e0caa9d25122c6ddaaa8"
server_key_version: "encryptionkey@0"
I20260923 22:16:11.676443 19941 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskRiQe_a/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790201539513534-1567-0/minicluster-data/ts-2/wal/instance:
uuid: "2ecd40dd495945ccba780072512499c2"
format_stamp: "Formatted at 2026-09-23 22:16:11 on dist-test-slave-s2fj"
server_key: "1bda25da1f55fb61c5c84ac41e332854"
server_key_iv: "157d698068c9e0caa9d25122c6ddaaa8"
server_key_version: "encryptionkey@0"
I20260923 22:16:11.677799 19941 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.001s sys 0.000s
I20260923 22:16:11.678525 19956 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260923 22:16:11.678686 19941 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260923 22:16:11.678779 19941 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskRiQe_a/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790201539513534-1567-0/minicluster-data/ts-2/data,/tmp/dist-test-taskRiQe_a/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790201539513534-1567-0/minicluster-data/ts-2/wal
uuid: "2ecd40dd495945ccba780072512499c2"
format_stamp: "Formatted at 2026-09-23 22:16:11 on dist-test-slave-s2fj"
server_key: "1bda25da1f55fb61c5c84ac41e332854"
server_key_iv: "157d698068c9e0caa9d25122c6ddaaa8"
server_key_version: "encryptionkey@0"
I20260923 22:16:11.678871 19941 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskRiQe_a/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790201539513534-1567-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskRiQe_a/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790201539513534-1567-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskRiQe_a/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790201539513534-1567-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260923 22:16:11.693634 19941 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260923 22:16:11.695947 19941 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260923 22:16:11.696157 19941 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260923 22:16:11.696404 19941 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260923 22:16:11.696703 19941 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260923 22:16:11.696755 19941 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260923 22:16:11.696799 19941 ts_tablet_manager.cc:610] Registered 0 tablets
I20260923 22:16:11.696823 19941 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260923 22:16:11.702611 19941 rpc_server.cc:307] RPC server started. Bound to: 127.1.135.195:42439
I20260923 22:16:11.702654 20069 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.1.135.195:42439 every 8 connection(s)
I20260923 22:16:11.702960 19941 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskRiQe_a/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790201539513534-1567-0/minicluster-data/ts-2/data/info.pb
I20260923 22:16:11.703840 1567 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskRiQe_a/build/release/bin/kudu as pid 19941
I20260923 22:16:11.703951 1567 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskRiQe_a/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790201539513534-1567-0/minicluster-data/ts-2/wal/instance
I20260923 22:16:11.704214 1567 external_mini_cluster.cc:1428] Setting key 31f00ff0357fd14befe260ee3419027e
Sep 23 22:16:11 dist-test-slave-s2fj krb5kdc[19566](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790201771, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.135.195@KRBTEST.COM for kudu/127.1.135.254@KRBTEST.COM
I20260923 22:16:11.711624 19669 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 22:16:11.703659 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.135.195:35103 (local address 127.1.135.254:44103)
0923 22:16:11.703736 (+ 77us) server_negotiation.cc:206] Beginning negotiation
0923 22:16:11.703739 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0923 22:16:11.703937 (+ 198us) server_negotiation.cc:407] Connection header received
0923 22:16:11.704660 (+ 723us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 22:16:11.704661 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 22:16:11.704670 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 22:16:11.704733 (+ 63us) server_negotiation.cc:226] Negotiated authn=SASL
0923 22:16:11.705755 (+ 1022us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 22:16:11.706243 (+ 488us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 22:16:11.707168 (+ 925us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 22:16:11.707351 (+ 183us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 22:16:11.707401 (+ 50us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 22:16:11.709460 (+ 2059us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0923 22:16:11.709461 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0923 22:16:11.709461 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0923 22:16:11.709467 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0923 22:16:11.710581 (+ 1114us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0923 22:16:11.710922 (+ 341us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0923 22:16:11.710923 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0923 22:16:11.710923 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0923 22:16:11.710968 (+ 45us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0923 22:16:11.711161 (+ 193us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0923 22:16:11.711161 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0923 22:16:11.711161 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0923 22:16:11.711249 (+ 88us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0923 22:16:11.711296 (+ 47us) server_negotiation.cc:1035] Waiting for connection context
0923 22:16:11.711510 (+ 214us) server_negotiation.cc:299] Negotiation successful
0923 22:16:11.711565 (+ 55us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":42}
I20260923 22:16:11.711813 20072 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 22:16:11.703631 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.135.254:44103 (local address 127.1.135.195:35103)
0923 22:16:11.703776 (+ 145us) negotiation.cc:107] Waiting for socket to connect
0923 22:16:11.703816 (+ 40us) client_negotiation.cc:174] Beginning negotiation
0923 22:16:11.704510 (+ 694us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0923 22:16:11.704825 (+ 315us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0923 22:16:11.704836 (+ 11us) client_negotiation.cc:394] Received NEGOTIATE response from server
0923 22:16:11.705488 (+ 652us) client_negotiation.cc:189] Negotiated authn=SASL
0923 22:16:11.705674 (+ 186us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 22:16:11.705678 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 22:16:11.706319 (+ 641us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 22:16:11.706320 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 22:16:11.707093 (+ 773us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 22:16:11.707095 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 22:16:11.707422 (+ 327us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 22:16:11.707422 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 22:16:11.707516 (+ 94us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 22:16:11.708025 (+ 509us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0923 22:16:11.708041 (+ 16us) client_negotiation.cc:656] calling sasl_client_start()
0923 22:16:11.709381 (+ 1340us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0923 22:16:11.710649 (+ 1268us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0923 22:16:11.710654 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0923 22:16:11.710656 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0923 22:16:11.710860 (+ 204us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0923 22:16:11.711022 (+ 162us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0923 22:16:11.711022 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0923 22:16:11.711023 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0923 22:16:11.711110 (+ 87us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0923 22:16:11.711301 (+ 191us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0923 22:16:11.711304 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0923 22:16:11.711408 (+ 104us) client_negotiation.cc:769] Sending connection context
0923 22:16:11.711483 (+ 75us) client_negotiation.cc:240] Negotiation successful
0923 22:16:11.711588 (+ 105us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":105,"thread_start_us":80,"threads_started":1}
I20260923 22:16:11.712303 20070 heartbeater.cc:347] Connected to a master server at 127.1.135.254:44103
I20260923 22:16:11.712401 20070 heartbeater.cc:464] Registering TS with master...
I20260923 22:16:11.712571 20070 heartbeater.cc:511] Master 127.1.135.254:44103 requested a full tablet report, sending...
I20260923 22:16:11.712929 19610 ts_manager.cc:194] Registered new tserver with Master: 2ecd40dd495945ccba780072512499c2 (127.1.135.195:42439)
I20260923 22:16:11.713243 19610 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.1.135.195@KRBTEST.COM'} at 127.1.135.195:35103
I20260923 22:16:11.715891 1567 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260923 22:16:11.722443 19669 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 22:16:11.717188 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60262 (local address 127.1.135.254:44103)
0923 22:16:11.717219 (+ 31us) server_negotiation.cc:206] Beginning negotiation
0923 22:16:11.717220 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0923 22:16:11.717342 (+ 122us) server_negotiation.cc:407] Connection header received
0923 22:16:11.717382 (+ 40us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 22:16:11.717383 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 22:16:11.717389 (+ 6us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 22:16:11.717452 (+ 63us) server_negotiation.cc:226] Negotiated authn=SASL
0923 22:16:11.717944 (+ 492us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 22:16:11.718360 (+ 416us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 22:16:11.719047 (+ 687us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 22:16:11.719226 (+ 179us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 22:16:11.719272 (+ 46us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 22:16:11.720477 (+ 1205us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0923 22:16:11.720478 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0923 22:16:11.720478 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0923 22:16:11.720483 (+ 5us) server_negotiation.cc:922] calling sasl_server_start()
0923 22:16:11.721539 (+ 1056us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0923 22:16:11.721864 (+ 325us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0923 22:16:11.721864 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0923 22:16:11.721865 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0923 22:16:11.721909 (+ 44us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0923 22:16:11.722046 (+ 137us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0923 22:16:11.722046 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0923 22:16:11.722046 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0923 22:16:11.722141 (+ 95us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0923 22:16:11.722211 (+ 70us) server_negotiation.cc:1035] Waiting for connection context
0923 22:16:11.722345 (+ 134us) server_negotiation.cc:299] Negotiation successful
0923 22:16:11.722389 (+ 44us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":14}
I20260923 22:16:11.723776 19610 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:60262:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260923 22:16:11.725189 19610 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260923 22:16:11.731266 20078 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 22:16:11.727165 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.135.193:39039 (local address 127.0.0.1:57802)
0923 22:16:11.727504 (+ 339us) negotiation.cc:107] Waiting for socket to connect
0923 22:16:11.727545 (+ 41us) client_negotiation.cc:174] Beginning negotiation
0923 22:16:11.727607 (+ 62us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0923 22:16:11.727945 (+ 338us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0923 22:16:11.727948 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0923 22:16:11.727962 (+ 14us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0923 22:16:11.728046 (+ 84us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 22:16:11.728049 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 22:16:11.729330 (+ 1281us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 22:16:11.729331 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 22:16:11.730267 (+ 936us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 22:16:11.730270 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 22:16:11.730952 (+ 682us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 22:16:11.730953 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 22:16:11.731040 (+ 87us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 22:16:11.731052 (+ 12us) client_negotiation.cc:769] Sending connection context
0923 22:16:11.731147 (+ 95us) client_negotiation.cc:240] Negotiation successful
0923 22:16:11.731199 (+ 52us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":314,"mutex_wait_us":102,"thread_start_us":51,"threads_started":1}
I20260923 22:16:11.731282 20082 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 22:16:11.727360 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57802 (local address 127.1.135.193:39039)
0923 22:16:11.727754 (+ 394us) server_negotiation.cc:206] Beginning negotiation
0923 22:16:11.727761 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0923 22:16:11.727795 (+ 34us) server_negotiation.cc:407] Connection header received
0923 22:16:11.727843 (+ 48us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 22:16:11.727846 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 22:16:11.727861 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 22:16:11.727941 (+ 80us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0923 22:16:11.728152 (+ 211us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 22:16:11.729224 (+ 1072us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 22:16:11.730380 (+ 1156us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 22:16:11.730852 (+ 472us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 22:16:11.730948 (+ 96us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 22:16:11.731007 (+ 59us) server_negotiation.cc:1035] Waiting for connection context
0923 22:16:11.731166 (+ 159us) server_negotiation.cc:299] Negotiation successful
0923 22:16:11.731209 (+ 43us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":346,"thread_start_us":79,"threads_started":1}
I20260923 22:16:11.731711 20081 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 22:16:11.727377 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.135.195:42439 (local address 127.0.0.1:59342)
0923 22:16:11.727540 (+ 163us) negotiation.cc:107] Waiting for socket to connect
0923 22:16:11.727548 (+ 8us) client_negotiation.cc:174] Beginning negotiation
0923 22:16:11.727605 (+ 57us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0923 22:16:11.727945 (+ 340us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0923 22:16:11.727948 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0923 22:16:11.727964 (+ 16us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0923 22:16:11.728047 (+ 83us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 22:16:11.728049 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 22:16:11.729409 (+ 1360us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 22:16:11.729410 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 22:16:11.730599 (+ 1189us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 22:16:11.730601 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 22:16:11.731466 (+ 865us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 22:16:11.731467 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 22:16:11.731524 (+ 57us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 22:16:11.731528 (+ 4us) client_negotiation.cc:769] Sending connection context
0923 22:16:11.731575 (+ 47us) client_negotiation.cc:240] Negotiation successful
0923 22:16:11.731611 (+ 36us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":153,"mutex_wait_us":90,"thread_start_us":39,"threads_started":1}
I20260923 22:16:11.731981 20079 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 22:16:11.727159 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.135.194:43765 (local address 127.0.0.1:40972)
0923 22:16:11.727503 (+ 344us) negotiation.cc:107] Waiting for socket to connect
0923 22:16:11.727542 (+ 39us) client_negotiation.cc:174] Beginning negotiation
0923 22:16:11.727605 (+ 63us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0923 22:16:11.728161 (+ 556us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0923 22:16:11.728162 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0923 22:16:11.728164 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0923 22:16:11.728225 (+ 61us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 22:16:11.728227 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 22:16:11.729218 (+ 991us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 22:16:11.729219 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 22:16:11.730891 (+ 1672us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 22:16:11.730893 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 22:16:11.731755 (+ 862us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 22:16:11.731757 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 22:16:11.731826 (+ 69us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 22:16:11.731829 (+ 3us) client_negotiation.cc:769] Sending connection context
0923 22:16:11.731884 (+ 55us) client_negotiation.cc:240] Negotiation successful
0923 22:16:11.731920 (+ 36us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":311,"mutex_wait_us":88,"thread_start_us":60,"threads_started":1}
I20260923 22:16:11.732018 20083 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 22:16:11.727433 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59342 (local address 127.1.135.195:42439)
0923 22:16:11.727728 (+ 295us) server_negotiation.cc:206] Beginning negotiation
0923 22:16:11.727750 (+ 22us) server_negotiation.cc:399] Waiting for connection header
0923 22:16:11.727759 (+ 9us) server_negotiation.cc:407] Connection header received
0923 22:16:11.727790 (+ 31us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 22:16:11.727792 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 22:16:11.727806 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 22:16:11.727881 (+ 75us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0923 22:16:11.728149 (+ 268us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 22:16:11.729221 (+ 1072us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 22:16:11.730682 (+ 1461us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 22:16:11.731317 (+ 635us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 22:16:11.731388 (+ 71us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 22:16:11.731432 (+ 44us) server_negotiation.cc:1035] Waiting for connection context
0923 22:16:11.731897 (+ 465us) server_negotiation.cc:299] Negotiation successful
0923 22:16:11.731944 (+ 47us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":267,"thread_start_us":73,"threads_started":1}
I20260923 22:16:11.732285 20080 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 22:16:11.727285 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40972 (local address 127.1.135.194:43765)
0923 22:16:11.727631 (+ 346us) server_negotiation.cc:206] Beginning negotiation
0923 22:16:11.727636 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0923 22:16:11.727646 (+ 10us) server_negotiation.cc:407] Connection header received
0923 22:16:11.727686 (+ 40us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 22:16:11.727689 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 22:16:11.727705 (+ 16us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 22:16:11.727812 (+ 107us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0923 22:16:11.728305 (+ 493us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 22:16:11.729073 (+ 768us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 22:16:11.730972 (+ 1899us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 22:16:11.731635 (+ 663us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 22:16:11.731716 (+ 81us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 22:16:11.731765 (+ 49us) server_negotiation.cc:1035] Waiting for connection context
0923 22:16:11.732162 (+ 397us) server_negotiation.cc:299] Negotiation successful
0923 22:16:11.732213 (+ 51us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":312,"thread_start_us":102,"threads_started":1}
I20260923 22:16:11.732191 19736 tablet_service.cc:1467] Processing CreateTablet for tablet 60360d68eb204ad3b01731a9f29f1a37 (DEFAULT_TABLE table=test-table [id=59f1ee65bb8d42ad8cd9f9815b9102d9]), partition=RANGE (key) PARTITION UNBOUNDED
I20260923 22:16:11.732537 19736 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 60360d68eb204ad3b01731a9f29f1a37. 1 dirs total, 0 dirs full, 0 dirs failed
I20260923 22:16:11.732697 20004 tablet_service.cc:1467] Processing CreateTablet for tablet 60360d68eb204ad3b01731a9f29f1a37 (DEFAULT_TABLE table=test-table [id=59f1ee65bb8d42ad8cd9f9815b9102d9]), partition=RANGE (key) PARTITION UNBOUNDED
I20260923 22:16:11.732975 20004 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 60360d68eb204ad3b01731a9f29f1a37. 1 dirs total, 0 dirs full, 0 dirs failed
I20260923 22:16:11.734870 20084 tablet_bootstrap.cc:492] T 60360d68eb204ad3b01731a9f29f1a37 P 1088b73602c14857a2ad641d87b2a5de: Bootstrap starting.
I20260923 22:16:11.735527 20084 tablet_bootstrap.cc:654] T 60360d68eb204ad3b01731a9f29f1a37 P 1088b73602c14857a2ad641d87b2a5de: Neither blocks nor log segments found. Creating new log.
I20260923 22:16:11.735520 20085 tablet_bootstrap.cc:492] T 60360d68eb204ad3b01731a9f29f1a37 P 2ecd40dd495945ccba780072512499c2: Bootstrap starting.
I20260923 22:16:11.735432 19870 tablet_service.cc:1467] Processing CreateTablet for tablet 60360d68eb204ad3b01731a9f29f1a37 (DEFAULT_TABLE table=test-table [id=59f1ee65bb8d42ad8cd9f9815b9102d9]), partition=RANGE (key) PARTITION UNBOUNDED
I20260923 22:16:11.735770 19870 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 60360d68eb204ad3b01731a9f29f1a37. 1 dirs total, 0 dirs full, 0 dirs failed
I20260923 22:16:11.736106 20085 tablet_bootstrap.cc:654] T 60360d68eb204ad3b01731a9f29f1a37 P 2ecd40dd495945ccba780072512499c2: Neither blocks nor log segments found. Creating new log.
I20260923 22:16:11.736088 20084 log.cc:824] T 60360d68eb204ad3b01731a9f29f1a37 P 1088b73602c14857a2ad641d87b2a5de: Log is configured to *not* fsync() on all Append() calls
I20260923 22:16:11.736621 20085 log.cc:824] T 60360d68eb204ad3b01731a9f29f1a37 P 2ecd40dd495945ccba780072512499c2: Log is configured to *not* fsync() on all Append() calls
I20260923 22:16:11.737255 20085 tablet_bootstrap.cc:492] T 60360d68eb204ad3b01731a9f29f1a37 P 2ecd40dd495945ccba780072512499c2: No bootstrap required, opened a new log
I20260923 22:16:11.737335 20085 ts_tablet_manager.cc:1397] T 60360d68eb204ad3b01731a9f29f1a37 P 2ecd40dd495945ccba780072512499c2: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20260923 22:16:11.738337 20088 tablet_bootstrap.cc:492] T 60360d68eb204ad3b01731a9f29f1a37 P 33e26a8f637243aab1d372b50c990324: Bootstrap starting.
I20260923 22:16:11.738801 20084 tablet_bootstrap.cc:492] T 60360d68eb204ad3b01731a9f29f1a37 P 1088b73602c14857a2ad641d87b2a5de: No bootstrap required, opened a new log
I20260923 22:16:11.738859 20084 ts_tablet_manager.cc:1397] T 60360d68eb204ad3b01731a9f29f1a37 P 1088b73602c14857a2ad641d87b2a5de: Time spent bootstrapping tablet: real 0.004s user 0.001s sys 0.000s
I20260923 22:16:11.738897 20085 raft_consensus.cc:348] T 60360d68eb204ad3b01731a9f29f1a37 P 2ecd40dd495945ccba780072512499c2 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1088b73602c14857a2ad641d87b2a5de" member_type: VOTER last_known_addr { host: "127.1.135.193" port: 39039 } } peers { permanent_uuid: "33e26a8f637243aab1d372b50c990324" member_type: VOTER last_known_addr { host: "127.1.135.194" port: 43765 } } peers { permanent_uuid: "2ecd40dd495945ccba780072512499c2" member_type: VOTER last_known_addr { host: "127.1.135.195" port: 42439 } }
I20260923 22:16:11.739074 20088 tablet_bootstrap.cc:654] T 60360d68eb204ad3b01731a9f29f1a37 P 33e26a8f637243aab1d372b50c990324: Neither blocks nor log segments found. Creating new log.
I20260923 22:16:11.739081 20085 raft_consensus.cc:374] T 60360d68eb204ad3b01731a9f29f1a37 P 2ecd40dd495945ccba780072512499c2 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260923 22:16:11.739117 20085 raft_consensus.cc:729] T 60360d68eb204ad3b01731a9f29f1a37 P 2ecd40dd495945ccba780072512499c2 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 2ecd40dd495945ccba780072512499c2, State: Initialized, Role: FOLLOWER
I20260923 22:16:11.739300 20085 consensus_queue.cc:260] T 60360d68eb204ad3b01731a9f29f1a37 P 2ecd40dd495945ccba780072512499c2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1088b73602c14857a2ad641d87b2a5de" member_type: VOTER last_known_addr { host: "127.1.135.193" port: 39039 } } peers { permanent_uuid: "33e26a8f637243aab1d372b50c990324" member_type: VOTER last_known_addr { host: "127.1.135.194" port: 43765 } } peers { permanent_uuid: "2ecd40dd495945ccba780072512499c2" member_type: VOTER last_known_addr { host: "127.1.135.195" port: 42439 } }
I20260923 22:16:11.739614 20070 heartbeater.cc:503] Master 127.1.135.254:44103 was elected leader, sending a full tablet report...
I20260923 22:16:11.739636 20088 log.cc:824] T 60360d68eb204ad3b01731a9f29f1a37 P 33e26a8f637243aab1d372b50c990324: Log is configured to *not* fsync() on all Append() calls
I20260923 22:16:11.739606 20085 ts_tablet_manager.cc:1428] T 60360d68eb204ad3b01731a9f29f1a37 P 2ecd40dd495945ccba780072512499c2: Time spent starting tablet: real 0.002s user 0.002s sys 0.000s
I20260923 22:16:11.740483 20088 tablet_bootstrap.cc:492] T 60360d68eb204ad3b01731a9f29f1a37 P 33e26a8f637243aab1d372b50c990324: No bootstrap required, opened a new log
I20260923 22:16:11.740551 20088 ts_tablet_manager.cc:1397] T 60360d68eb204ad3b01731a9f29f1a37 P 33e26a8f637243aab1d372b50c990324: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20260923 22:16:11.740901 20084 raft_consensus.cc:348] T 60360d68eb204ad3b01731a9f29f1a37 P 1088b73602c14857a2ad641d87b2a5de [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1088b73602c14857a2ad641d87b2a5de" member_type: VOTER last_known_addr { host: "127.1.135.193" port: 39039 } } peers { permanent_uuid: "33e26a8f637243aab1d372b50c990324" member_type: VOTER last_known_addr { host: "127.1.135.194" port: 43765 } } peers { permanent_uuid: "2ecd40dd495945ccba780072512499c2" member_type: VOTER last_known_addr { host: "127.1.135.195" port: 42439 } }
I20260923 22:16:11.741050 20084 raft_consensus.cc:374] T 60360d68eb204ad3b01731a9f29f1a37 P 1088b73602c14857a2ad641d87b2a5de [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260923 22:16:11.741094 20084 raft_consensus.cc:729] T 60360d68eb204ad3b01731a9f29f1a37 P 1088b73602c14857a2ad641d87b2a5de [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 1088b73602c14857a2ad641d87b2a5de, State: Initialized, Role: FOLLOWER
I20260923 22:16:11.741201 20084 consensus_queue.cc:260] T 60360d68eb204ad3b01731a9f29f1a37 P 1088b73602c14857a2ad641d87b2a5de [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1088b73602c14857a2ad641d87b2a5de" member_type: VOTER last_known_addr { host: "127.1.135.193" port: 39039 } } peers { permanent_uuid: "33e26a8f637243aab1d372b50c990324" member_type: VOTER last_known_addr { host: "127.1.135.194" port: 43765 } } peers { permanent_uuid: "2ecd40dd495945ccba780072512499c2" member_type: VOTER last_known_addr { host: "127.1.135.195" port: 42439 } }
I20260923 22:16:11.741497 20084 ts_tablet_manager.cc:1428] T 60360d68eb204ad3b01731a9f29f1a37 P 1088b73602c14857a2ad641d87b2a5de: Time spent starting tablet: real 0.003s user 0.003s sys 0.000s
I20260923 22:16:11.741545 19802 heartbeater.cc:503] Master 127.1.135.254:44103 was elected leader, sending a full tablet report...
I20260923 22:16:11.742556 20088 raft_consensus.cc:348] T 60360d68eb204ad3b01731a9f29f1a37 P 33e26a8f637243aab1d372b50c990324 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1088b73602c14857a2ad641d87b2a5de" member_type: VOTER last_known_addr { host: "127.1.135.193" port: 39039 } } peers { permanent_uuid: "33e26a8f637243aab1d372b50c990324" member_type: VOTER last_known_addr { host: "127.1.135.194" port: 43765 } } peers { permanent_uuid: "2ecd40dd495945ccba780072512499c2" member_type: VOTER last_known_addr { host: "127.1.135.195" port: 42439 } }
I20260923 22:16:11.742753 20088 raft_consensus.cc:374] T 60360d68eb204ad3b01731a9f29f1a37 P 33e26a8f637243aab1d372b50c990324 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260923 22:16:11.742789 20088 raft_consensus.cc:729] T 60360d68eb204ad3b01731a9f29f1a37 P 33e26a8f637243aab1d372b50c990324 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 33e26a8f637243aab1d372b50c990324, State: Initialized, Role: FOLLOWER
I20260923 22:16:11.742904 20088 consensus_queue.cc:260] T 60360d68eb204ad3b01731a9f29f1a37 P 33e26a8f637243aab1d372b50c990324 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1088b73602c14857a2ad641d87b2a5de" member_type: VOTER last_known_addr { host: "127.1.135.193" port: 39039 } } peers { permanent_uuid: "33e26a8f637243aab1d372b50c990324" member_type: VOTER last_known_addr { host: "127.1.135.194" port: 43765 } } peers { permanent_uuid: "2ecd40dd495945ccba780072512499c2" member_type: VOTER last_known_addr { host: "127.1.135.195" port: 42439 } }
I20260923 22:16:11.743147 20088 ts_tablet_manager.cc:1428] T 60360d68eb204ad3b01731a9f29f1a37 P 33e26a8f637243aab1d372b50c990324: Time spent starting tablet: real 0.003s user 0.003s sys 0.000s
I20260923 22:16:11.743203 19936 heartbeater.cc:503] Master 127.1.135.254:44103 was elected leader, sending a full tablet report...
I20260923 22:16:11.747150 20092 raft_consensus.cc:482] T 60360d68eb204ad3b01731a9f29f1a37 P 33e26a8f637243aab1d372b50c990324 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260923 22:16:11.747213 20092 raft_consensus.cc:504] T 60360d68eb204ad3b01731a9f29f1a37 P 33e26a8f637243aab1d372b50c990324 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1088b73602c14857a2ad641d87b2a5de" member_type: VOTER last_known_addr { host: "127.1.135.193" port: 39039 } } peers { permanent_uuid: "33e26a8f637243aab1d372b50c990324" member_type: VOTER last_known_addr { host: "127.1.135.194" port: 43765 } } peers { permanent_uuid: "2ecd40dd495945ccba780072512499c2" member_type: VOTER last_known_addr { host: "127.1.135.195" port: 42439 } }
I20260923 22:16:11.747537 20092 leader_election.cc:288] T 60360d68eb204ad3b01731a9f29f1a37 P 33e26a8f637243aab1d372b50c990324 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 1088b73602c14857a2ad641d87b2a5de (127.1.135.193:39039), 2ecd40dd495945ccba780072512499c2 (127.1.135.195:42439)
I20260923 22:16:11.750242 19938 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 22:16:11.747651 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.135.193:39039 (local address 127.1.135.194:53313)
0923 22:16:11.747700 (+ 49us) negotiation.cc:107] Waiting for socket to connect
0923 22:16:11.747710 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0923 22:16:11.747753 (+ 43us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0923 22:16:11.747935 (+ 182us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0923 22:16:11.747936 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0923 22:16:11.747938 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0923 22:16:11.747991 (+ 53us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 22:16:11.747993 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 22:16:11.748603 (+ 610us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 22:16:11.748604 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 22:16:11.749409 (+ 805us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 22:16:11.749410 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 22:16:11.750002 (+ 592us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 22:16:11.750003 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 22:16:11.750082 (+ 79us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 22:16:11.750086 (+ 4us) client_negotiation.cc:769] Sending connection context
0923 22:16:11.750137 (+ 51us) client_negotiation.cc:240] Negotiation successful
0923 22:16:11.750178 (+ 41us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":31,"mutex_wait_us":27}
I20260923 22:16:11.750264 20082 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 22:16:11.747707 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.135.194:53313 (local address 127.1.135.193:39039)
0923 22:16:11.747743 (+ 36us) server_negotiation.cc:206] Beginning negotiation
0923 22:16:11.747745 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0923 22:16:11.747752 (+ 7us) server_negotiation.cc:407] Connection header received
0923 22:16:11.747804 (+ 52us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 22:16:11.747804 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 22:16:11.747813 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 22:16:11.747873 (+ 60us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0923 22:16:11.748064 (+ 191us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 22:16:11.748530 (+ 466us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 22:16:11.749494 (+ 964us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 22:16:11.749929 (+ 435us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 22:16:11.749982 (+ 53us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 22:16:11.749987 (+ 5us) server_negotiation.cc:1035] Waiting for connection context
0923 22:16:11.750152 (+ 165us) server_negotiation.cc:299] Negotiation successful
0923 22:16:11.750190 (+ 38us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":18}
I20260923 22:16:11.750361 20093 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 22:16:11.747752 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.135.195:42439 (local address 127.1.135.194:42627)
0923 22:16:11.747929 (+ 177us) negotiation.cc:107] Waiting for socket to connect
0923 22:16:11.747937 (+ 8us) client_negotiation.cc:174] Beginning negotiation
0923 22:16:11.747982 (+ 45us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0923 22:16:11.748103 (+ 121us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0923 22:16:11.748104 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0923 22:16:11.748105 (+ 1us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0923 22:16:11.748162 (+ 57us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 22:16:11.748163 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 22:16:11.748794 (+ 631us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 22:16:11.748795 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 22:16:11.749577 (+ 782us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 22:16:11.749579 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 22:16:11.750099 (+ 520us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 22:16:11.750100 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 22:16:11.750184 (+ 84us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 22:16:11.750188 (+ 4us) client_negotiation.cc:769] Sending connection context
0923 22:16:11.750245 (+ 57us) client_negotiation.cc:240] Negotiation successful
0923 22:16:11.750281 (+ 36us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":161,"thread_start_us":87,"threads_started":1}
I20260923 22:16:11.750380 20083 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 22:16:11.747848 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.135.194:42627 (local address 127.1.135.195:42439)
0923 22:16:11.747880 (+ 32us) server_negotiation.cc:206] Beginning negotiation
0923 22:16:11.747882 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0923 22:16:11.747972 (+ 90us) server_negotiation.cc:407] Connection header received
0923 22:16:11.748022 (+ 50us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 22:16:11.748024 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 22:16:11.748031 (+ 7us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 22:16:11.748096 (+ 65us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0923 22:16:11.748208 (+ 112us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 22:16:11.748717 (+ 509us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 22:16:11.749656 (+ 939us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 22:16:11.750020 (+ 364us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 22:16:11.750086 (+ 66us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 22:16:11.750093 (+ 7us) server_negotiation.cc:1035] Waiting for connection context
0923 22:16:11.750262 (+ 169us) server_negotiation.cc:299] Negotiation successful
0923 22:16:11.750301 (+ 39us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":18}
I20260923 22:16:11.750804 20024 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "60360d68eb204ad3b01731a9f29f1a37" candidate_uuid: "33e26a8f637243aab1d372b50c990324" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "2ecd40dd495945ccba780072512499c2" is_pre_election: true
I20260923 22:16:11.750809 19756 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "60360d68eb204ad3b01731a9f29f1a37" candidate_uuid: "33e26a8f637243aab1d372b50c990324" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1088b73602c14857a2ad641d87b2a5de" is_pre_election: true
I20260923 22:16:11.750910 20024 raft_consensus.cc:2454] T 60360d68eb204ad3b01731a9f29f1a37 P 2ecd40dd495945ccba780072512499c2 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 33e26a8f637243aab1d372b50c990324 in term 0.
I20260923 22:16:11.750916 19756 raft_consensus.cc:2454] T 60360d68eb204ad3b01731a9f29f1a37 P 1088b73602c14857a2ad641d87b2a5de [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 33e26a8f637243aab1d372b50c990324 in term 0.
I20260923 22:16:11.751096 19824 leader_election.cc:302] T 60360d68eb204ad3b01731a9f29f1a37 P 33e26a8f637243aab1d372b50c990324 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2ecd40dd495945ccba780072512499c2, 33e26a8f637243aab1d372b50c990324; no voters:
I20260923 22:16:11.751236 20092 raft_consensus.cc:2781] T 60360d68eb204ad3b01731a9f29f1a37 P 33e26a8f637243aab1d372b50c990324 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260923 22:16:11.751302 20092 raft_consensus.cc:482] T 60360d68eb204ad3b01731a9f29f1a37 P 33e26a8f637243aab1d372b50c990324 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260923 22:16:11.751319 20092 raft_consensus.cc:3037] T 60360d68eb204ad3b01731a9f29f1a37 P 33e26a8f637243aab1d372b50c990324 [term 0 FOLLOWER]: Advancing to term 1
I20260923 22:16:11.751890 20092 raft_consensus.cc:504] T 60360d68eb204ad3b01731a9f29f1a37 P 33e26a8f637243aab1d372b50c990324 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1088b73602c14857a2ad641d87b2a5de" member_type: VOTER last_known_addr { host: "127.1.135.193" port: 39039 } } peers { permanent_uuid: "33e26a8f637243aab1d372b50c990324" member_type: VOTER last_known_addr { host: "127.1.135.194" port: 43765 } } peers { permanent_uuid: "2ecd40dd495945ccba780072512499c2" member_type: VOTER last_known_addr { host: "127.1.135.195" port: 42439 } }
I20260923 22:16:11.752012 20092 leader_election.cc:288] T 60360d68eb204ad3b01731a9f29f1a37 P 33e26a8f637243aab1d372b50c990324 [CANDIDATE]: Term 1 election: Requested vote from peers 1088b73602c14857a2ad641d87b2a5de (127.1.135.193:39039), 2ecd40dd495945ccba780072512499c2 (127.1.135.195:42439)
I20260923 22:16:11.752172 19756 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "60360d68eb204ad3b01731a9f29f1a37" candidate_uuid: "33e26a8f637243aab1d372b50c990324" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1088b73602c14857a2ad641d87b2a5de"
I20260923 22:16:11.752203 20024 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "60360d68eb204ad3b01731a9f29f1a37" candidate_uuid: "33e26a8f637243aab1d372b50c990324" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "2ecd40dd495945ccba780072512499c2"
I20260923 22:16:11.752247 19756 raft_consensus.cc:3037] T 60360d68eb204ad3b01731a9f29f1a37 P 1088b73602c14857a2ad641d87b2a5de [term 0 FOLLOWER]: Advancing to term 1
I20260923 22:16:11.752257 20024 raft_consensus.cc:3037] T 60360d68eb204ad3b01731a9f29f1a37 P 2ecd40dd495945ccba780072512499c2 [term 0 FOLLOWER]: Advancing to term 1
I20260923 22:16:11.753013 20024 raft_consensus.cc:2454] T 60360d68eb204ad3b01731a9f29f1a37 P 2ecd40dd495945ccba780072512499c2 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 33e26a8f637243aab1d372b50c990324 in term 1.
I20260923 22:16:11.753013 19756 raft_consensus.cc:2454] T 60360d68eb204ad3b01731a9f29f1a37 P 1088b73602c14857a2ad641d87b2a5de [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 33e26a8f637243aab1d372b50c990324 in term 1.
I20260923 22:16:11.753183 19824 leader_election.cc:302] T 60360d68eb204ad3b01731a9f29f1a37 P 33e26a8f637243aab1d372b50c990324 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2ecd40dd495945ccba780072512499c2, 33e26a8f637243aab1d372b50c990324; no voters:
I20260923 22:16:11.753261 20092 raft_consensus.cc:2781] T 60360d68eb204ad3b01731a9f29f1a37 P 33e26a8f637243aab1d372b50c990324 [term 1 FOLLOWER]: Leader election won for term 1
I20260923 22:16:11.753376 20092 raft_consensus.cc:686] T 60360d68eb204ad3b01731a9f29f1a37 P 33e26a8f637243aab1d372b50c990324 [term 1 LEADER]: Becoming Leader. State: Replica: 33e26a8f637243aab1d372b50c990324, State: Running, Role: LEADER
I20260923 22:16:11.753449 20092 consensus_queue.cc:237] T 60360d68eb204ad3b01731a9f29f1a37 P 33e26a8f637243aab1d372b50c990324 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1088b73602c14857a2ad641d87b2a5de" member_type: VOTER last_known_addr { host: "127.1.135.193" port: 39039 } } peers { permanent_uuid: "33e26a8f637243aab1d372b50c990324" member_type: VOTER last_known_addr { host: "127.1.135.194" port: 43765 } } peers { permanent_uuid: "2ecd40dd495945ccba780072512499c2" member_type: VOTER last_known_addr { host: "127.1.135.195" port: 42439 } }
I20260923 22:16:11.754098 19610 catalog_manager.cc:5495] T 60360d68eb204ad3b01731a9f29f1a37 P 33e26a8f637243aab1d372b50c990324 reported cstate change: term changed from 0 to 1, leader changed from <none> to 33e26a8f637243aab1d372b50c990324 (127.1.135.194). New cstate: current_term: 1 leader_uuid: "33e26a8f637243aab1d372b50c990324" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1088b73602c14857a2ad641d87b2a5de" member_type: VOTER last_known_addr { host: "127.1.135.193" port: 39039 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "33e26a8f637243aab1d372b50c990324" member_type: VOTER last_known_addr { host: "127.1.135.194" port: 43765 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "2ecd40dd495945ccba780072512499c2" member_type: VOTER last_known_addr { host: "127.1.135.195" port: 42439 } health_report { overall_health: UNKNOWN } } }
I20260923 22:16:11.765810 20080 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 22:16:11.763600 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40980 (local address 127.1.135.194:43765)
0923 22:16:11.763644 (+ 44us) server_negotiation.cc:206] Beginning negotiation
0923 22:16:11.763645 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0923 22:16:11.763654 (+ 9us) server_negotiation.cc:407] Connection header received
0923 22:16:11.763693 (+ 39us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 22:16:11.763693 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 22:16:11.763715 (+ 22us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 22:16:11.763785 (+ 70us) server_negotiation.cc:226] Negotiated authn=TOKEN
0923 22:16:11.763883 (+ 98us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 22:16:11.764352 (+ 469us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 22:16:11.765081 (+ 729us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 22:16:11.765263 (+ 182us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 22:16:11.765344 (+ 81us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 22:16:11.765495 (+ 151us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0923 22:16:11.765581 (+ 86us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0923 22:16:11.765685 (+ 104us) server_negotiation.cc:1035] Waiting for connection context
0923 22:16:11.765715 (+ 30us) server_negotiation.cc:299] Negotiation successful
0923 22:16:11.765725 (+ 10us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":24}
I20260923 22:16:11.767285 19756 raft_consensus.cc:1261] T 60360d68eb204ad3b01731a9f29f1a37 P 1088b73602c14857a2ad641d87b2a5de [term 1 FOLLOWER]: Refusing update from remote peer 33e26a8f637243aab1d372b50c990324: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260923 22:16:11.767285 20024 raft_consensus.cc:1261] T 60360d68eb204ad3b01731a9f29f1a37 P 2ecd40dd495945ccba780072512499c2 [term 1 FOLLOWER]: Refusing update from remote peer 33e26a8f637243aab1d372b50c990324: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260923 22:16:11.767606 20094 consensus_queue.cc:1035] T 60360d68eb204ad3b01731a9f29f1a37 P 33e26a8f637243aab1d372b50c990324 [LEADER]: Connected to new peer: Peer: permanent_uuid: "1088b73602c14857a2ad641d87b2a5de" member_type: VOTER last_known_addr { host: "127.1.135.193" port: 39039 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260923 22:16:11.767715 20092 consensus_queue.cc:1035] T 60360d68eb204ad3b01731a9f29f1a37 P 33e26a8f637243aab1d372b50c990324 [LEADER]: Connected to new peer: Peer: permanent_uuid: "2ecd40dd495945ccba780072512499c2" member_type: VOTER last_known_addr { host: "127.1.135.195" port: 42439 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260923 22:16:11.769742 20098 mvcc.cc:205] Tried to move back new op lower bound from 7332666457156624384 to 7332666457102778368. Current Snapshot: MvccSnapshot[applied={T|T < 7332666457156624384}]
I20260923 22:16:11.770599 20099 mvcc.cc:205] Tried to move back new op lower bound from 7332666457156624384 to 7332666457102778368. Current Snapshot: MvccSnapshot[applied={T|T < 7332666457156624384}]
I20260923 22:16:11.771736 19611 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:60262:
table { table_name: "test-table" } modify_external_catalogs: true
I20260923 22:16:11.771802 19611 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:60262:
table { table_name: "test-table" } modify_external_catalogs: true
I20260923 22:16:11.772434 19611 catalog_manager.cc:5782] T 00000000000000000000000000000000 P a2024669c690445cbe6333e111b7cd96: Sending DeleteTablet for 3 replicas of tablet 60360d68eb204ad3b01731a9f29f1a37
I20260923 22:16:11.772729 19870 tablet_service.cc:1514] Processing DeleteTablet for tablet 60360d68eb204ad3b01731a9f29f1a37 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-23 22:16:11 UTC) from {username='kudu', principal='kudu/127.1.135.254@KRBTEST.COM'} at 127.0.0.1:40972
I20260923 22:16:11.772748 19736 tablet_service.cc:1514] Processing DeleteTablet for tablet 60360d68eb204ad3b01731a9f29f1a37 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-23 22:16:11 UTC) from {username='kudu', principal='kudu/127.1.135.254@KRBTEST.COM'} at 127.0.0.1:57802
I20260923 22:16:11.772933 20107 tablet_replica.cc:330] T 60360d68eb204ad3b01731a9f29f1a37 P 33e26a8f637243aab1d372b50c990324: stopping tablet replica
I20260923 22:16:11.773021 20107 raft_consensus.cc:2229] T 60360d68eb204ad3b01731a9f29f1a37 P 33e26a8f637243aab1d372b50c990324 [term 1 LEADER]: Raft consensus shutting down.
I20260923 22:16:11.773066 20004 tablet_service.cc:1514] Processing DeleteTablet for tablet 60360d68eb204ad3b01731a9f29f1a37 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-23 22:16:11 UTC) from {username='kudu', principal='kudu/127.1.135.254@KRBTEST.COM'} at 127.0.0.1:59342
I20260923 22:16:11.773137 20107 raft_consensus.cc:2258] T 60360d68eb204ad3b01731a9f29f1a37 P 33e26a8f637243aab1d372b50c990324 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260923 22:16:11.773360 20106 tablet_replica.cc:330] T 60360d68eb204ad3b01731a9f29f1a37 P 1088b73602c14857a2ad641d87b2a5de: stopping tablet replica
I20260923 22:16:11.773453 1567 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskRiQe_a/build/release/bin/kudu with pid 19673
I20260923 22:16:11.774154 20108 tablet_replica.cc:330] T 60360d68eb204ad3b01731a9f29f1a37 P 2ecd40dd495945ccba780072512499c2: stopping tablet replica
I20260923 22:16:11.774188 20107 ts_tablet_manager.cc:1905] T 60360d68eb204ad3b01731a9f29f1a37 P 33e26a8f637243aab1d372b50c990324: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260923 22:16:11.774235 20108 raft_consensus.cc:2229] T 60360d68eb204ad3b01731a9f29f1a37 P 2ecd40dd495945ccba780072512499c2 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260923 22:16:11.774330 20108 raft_consensus.cc:2258] T 60360d68eb204ad3b01731a9f29f1a37 P 2ecd40dd495945ccba780072512499c2 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260923 22:16:11.775575 20108 ts_tablet_manager.cc:1905] T 60360d68eb204ad3b01731a9f29f1a37 P 2ecd40dd495945ccba780072512499c2: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260923 22:16:11.775590 20107 ts_tablet_manager.cc:1918] T 60360d68eb204ad3b01731a9f29f1a37 P 33e26a8f637243aab1d372b50c990324: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260923 22:16:11.775648 20107 log.cc:1196] T 60360d68eb204ad3b01731a9f29f1a37 P 33e26a8f637243aab1d372b50c990324: Deleting WAL directory at /tmp/dist-test-taskRiQe_a/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790201539513534-1567-0/minicluster-data/ts-1/wal/wals/60360d68eb204ad3b01731a9f29f1a37
I20260923 22:16:11.775930 20107 ts_tablet_manager.cc:1939] T 60360d68eb204ad3b01731a9f29f1a37 P 33e26a8f637243aab1d372b50c990324: Deleting consensus metadata
I20260923 22:16:11.776297 19597 catalog_manager.cc:4841] TS 33e26a8f637243aab1d372b50c990324 (127.1.135.194:43765): tablet 60360d68eb204ad3b01731a9f29f1a37 (table test-table [id=59f1ee65bb8d42ad8cd9f9815b9102d9]) successfully deleted
I20260923 22:16:11.777189 20108 ts_tablet_manager.cc:1918] T 60360d68eb204ad3b01731a9f29f1a37 P 2ecd40dd495945ccba780072512499c2: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260923 22:16:11.777272 20108 log.cc:1196] T 60360d68eb204ad3b01731a9f29f1a37 P 2ecd40dd495945ccba780072512499c2: Deleting WAL directory at /tmp/dist-test-taskRiQe_a/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790201539513534-1567-0/minicluster-data/ts-2/wal/wals/60360d68eb204ad3b01731a9f29f1a37
I20260923 22:16:11.777500 20108 ts_tablet_manager.cc:1939] T 60360d68eb204ad3b01731a9f29f1a37 P 2ecd40dd495945ccba780072512499c2: Deleting consensus metadata
I20260923 22:16:11.777817 19596 catalog_manager.cc:4841] TS 2ecd40dd495945ccba780072512499c2 (127.1.135.195:42439): tablet 60360d68eb204ad3b01731a9f29f1a37 (table test-table [id=59f1ee65bb8d42ad8cd9f9815b9102d9]) successfully deleted
W20260923 22:16:11.779515 19596 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.1.135.193:39039 (error 108)
I20260923 22:16:11.779772 1567 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskRiQe_a/build/release/bin/kudu with pid 19807
I20260923 22:16:11.779912 20079 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 22:16:11.779784 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.135.193:39039 (local address 127.0.0.1:57816)
0923 22:16:11.779849 (+ 65us) negotiation.cc:107] Waiting for socket to connect
0923 22:16:11.779879 (+ 30us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.135.193:39039: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":48}
W20260923 22:16:11.780330 19596 catalog_manager.cc:4584] TS 1088b73602c14857a2ad641d87b2a5de (127.1.135.193:39039): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet 60360d68eb204ad3b01731a9f29f1a37: Network error: Client connection negotiation failed: client connection to 127.1.135.193:39039: connect: Connection refused (error 111)
I20260923 22:16:11.784875 1567 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskRiQe_a/build/release/bin/kudu with pid 19941
I20260923 22:16:11.789464 1567 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskRiQe_a/build/release/bin/kudu with pid 19579
2026-09-23T22:16:11Z chronyd exiting
[ OK ] EncryptionPolicyTest.TestEncryptionPolicy/2 (1836 ms)
[ RUN ] EncryptionPolicyTest.TestEncryptionPolicy/3
2026-09-23T22:16:11Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-09-23T22:16:11Z Disabled control of system clock
I20260923 22:16:11.815997 1567 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskRiQe_a/build/release/bin/kudu
/tmp/dist-test-taskRiQe_a/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskRiQe_a/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790201539513534-1567-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskRiQe_a/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790201539513534-1567-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskRiQe_a/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790201539513534-1567-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskRiQe_a/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790201539513534-1567-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.1.135.254:46585
--webserver_interface=127.1.135.254
--webserver_port=0
--builtin_ntp_servers=127.1.135.212:41107
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.1.135.254:46585
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
W20260923 22:16:11.892503 20113 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260923 22:16:11.892692 20113 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260923 22:16:11.892730 20113 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260923 22:16:11.893777 20113 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260923 22:16:11.893808 20113 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260923 22:16:11.893821 20113 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260923 22:16:11.893834 20113 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260923 22:16:11.893846 20113 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260923 22:16:11.895038 20113 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.1.135.212:41107
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskRiQe_a/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790201539513534-1567-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskRiQe_a/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790201539513534-1567-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.1.135.254:46585
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.1.135.254:46585
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskRiQe_a/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790201539513534-1567-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.1.135.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.20113
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskRiQe_a/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790201539513534-1567-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 23 Sep 2026 20:53:37 UTC on ccycloud.quasar-rkviyx.root.comops.site
build id 4668083
I20260923 22:16:11.895195 20113 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260923 22:16:11.895388 20113 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260923 22:16:11.899868 20118 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260923 22:16:11.899922 20113 server_base.cc:1034] running on GCE node
W20260923 22:16:11.899875 20121 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260923 22:16:11.899883 20119 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260923 22:16:11.900364 20113 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260923 22:16:11.900642 20113 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260923 22:16:11.901803 20113 hybrid_clock.cc:648] HybridClock initialized: now 1790201771901797 us; error 34 us; skew 500 ppm
I20260923 22:16:11.902933 20113 webserver.cc:460] Webserver started at http://127.1.135.254:45001/ using document root <none> and password file <none>
I20260923 22:16:11.903156 20113 fs_manager.cc:359] Metadata directory not provided
I20260923 22:16:11.903223 20113 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260923 22:16:11.903370 20113 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260923 22:16:11.904389 20113 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskRiQe_a/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790201539513534-1567-0/minicluster-data/master-0/data/instance:
uuid: "141301c78aa44ae1bf7a3b8ba5e1c9ed"
format_stamp: "Formatted at 2026-09-23 22:16:11 on dist-test-slave-s2fj"
server_key: "494519ab2b0349b988e026e3ab75e1f9"
server_key_iv: "be64eca0775403ae6f4014739f6da212"
server_key_version: "encryptionkey@0"
I20260923 22:16:11.904716 20113 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskRiQe_a/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790201539513534-1567-0/minicluster-data/master-0/wal/instance:
uuid: "141301c78aa44ae1bf7a3b8ba5e1c9ed"
format_stamp: "Formatted at 2026-09-23 22:16:11 on dist-test-slave-s2fj"
server_key: "494519ab2b0349b988e026e3ab75e1f9"
server_key_iv: "be64eca0775403ae6f4014739f6da212"
server_key_version: "encryptionkey@0"
I20260923 22:16:11.905887 20113 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.000s
I20260923 22:16:11.906523 20127 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260923 22:16:11.906685 20113 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260923 22:16:11.906777 20113 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskRiQe_a/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790201539513534-1567-0/minicluster-data/master-0/data,/tmp/dist-test-taskRiQe_a/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790201539513534-1567-0/minicluster-data/master-0/wal
uuid: "141301c78aa44ae1bf7a3b8ba5e1c9ed"
format_stamp: "Formatted at 2026-09-23 22:16:11 on dist-test-slave-s2fj"
server_key: "494519ab2b0349b988e026e3ab75e1f9"
server_key_iv: "be64eca0775403ae6f4014739f6da212"
server_key_version: "encryptionkey@0"
I20260923 22:16:11.906857 20113 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskRiQe_a/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790201539513534-1567-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskRiQe_a/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790201539513534-1567-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskRiQe_a/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790201539513534-1567-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260923 22:16:11.937238 20113 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260923 22:16:11.937515 20113 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260923 22:16:11.937661 20113 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260923 22:16:11.941531 20113 rpc_server.cc:307] RPC server started. Bound to: 127.1.135.254:46585
I20260923 22:16:11.941565 20179 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.1.135.254:46585 every 8 connection(s)
I20260923 22:16:11.941876 20113 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskRiQe_a/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790201539513534-1567-0/minicluster-data/master-0/data/info.pb
I20260923 22:16:11.942803 20180 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260923 22:16:11.944865 20180 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 141301c78aa44ae1bf7a3b8ba5e1c9ed: Bootstrap starting.
I20260923 22:16:11.945362 20180 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 141301c78aa44ae1bf7a3b8ba5e1c9ed: Neither blocks nor log segments found. Creating new log.
I20260923 22:16:11.945600 20180 log.cc:824] T 00000000000000000000000000000000 P 141301c78aa44ae1bf7a3b8ba5e1c9ed: Log is configured to *not* fsync() on all Append() calls
I20260923 22:16:11.946201 20180 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 141301c78aa44ae1bf7a3b8ba5e1c9ed: No bootstrap required, opened a new log
I20260923 22:16:11.947650 20180 raft_consensus.cc:348] T 00000000000000000000000000000000 P 141301c78aa44ae1bf7a3b8ba5e1c9ed [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "141301c78aa44ae1bf7a3b8ba5e1c9ed" member_type: VOTER last_known_addr { host: "127.1.135.254" port: 46585 } }
I20260923 22:16:11.947773 20180 raft_consensus.cc:374] T 00000000000000000000000000000000 P 141301c78aa44ae1bf7a3b8ba5e1c9ed [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260923 22:16:11.947794 20180 raft_consensus.cc:729] T 00000000000000000000000000000000 P 141301c78aa44ae1bf7a3b8ba5e1c9ed [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 141301c78aa44ae1bf7a3b8ba5e1c9ed, State: Initialized, Role: FOLLOWER
I20260923 22:16:11.947935 20180 consensus_queue.cc:260] T 00000000000000000000000000000000 P 141301c78aa44ae1bf7a3b8ba5e1c9ed [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "141301c78aa44ae1bf7a3b8ba5e1c9ed" member_type: VOTER last_known_addr { host: "127.1.135.254" port: 46585 } }
I20260923 22:16:11.948019 20180 raft_consensus.cc:388] T 00000000000000000000000000000000 P 141301c78aa44ae1bf7a3b8ba5e1c9ed [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260923 22:16:11.948043 20180 raft_consensus.cc:482] T 00000000000000000000000000000000 P 141301c78aa44ae1bf7a3b8ba5e1c9ed [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260923 22:16:11.948132 20180 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 141301c78aa44ae1bf7a3b8ba5e1c9ed [term 0 FOLLOWER]: Advancing to term 1
I20260923 22:16:11.948577 20180 raft_consensus.cc:504] T 00000000000000000000000000000000 P 141301c78aa44ae1bf7a3b8ba5e1c9ed [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "141301c78aa44ae1bf7a3b8ba5e1c9ed" member_type: VOTER last_known_addr { host: "127.1.135.254" port: 46585 } }
I20260923 22:16:11.948657 20180 leader_election.cc:302] T 00000000000000000000000000000000 P 141301c78aa44ae1bf7a3b8ba5e1c9ed [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 141301c78aa44ae1bf7a3b8ba5e1c9ed; no voters:
I20260923 22:16:11.948767 20180 leader_election.cc:288] T 00000000000000000000000000000000 P 141301c78aa44ae1bf7a3b8ba5e1c9ed [CANDIDATE]: Term 1 election: Requested vote from peers
I20260923 22:16:11.948843 20183 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 141301c78aa44ae1bf7a3b8ba5e1c9ed [term 1 FOLLOWER]: Leader election won for term 1
I20260923 22:16:11.948988 20183 raft_consensus.cc:686] T 00000000000000000000000000000000 P 141301c78aa44ae1bf7a3b8ba5e1c9ed [term 1 LEADER]: Becoming Leader. State: Replica: 141301c78aa44ae1bf7a3b8ba5e1c9ed, State: Running, Role: LEADER
I20260923 22:16:11.949002 20180 sys_catalog.cc:564] T 00000000000000000000000000000000 P 141301c78aa44ae1bf7a3b8ba5e1c9ed [sys.catalog]: configured and running, proceeding with master startup.
I20260923 22:16:11.949090 20183 consensus_queue.cc:237] T 00000000000000000000000000000000 P 141301c78aa44ae1bf7a3b8ba5e1c9ed [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "141301c78aa44ae1bf7a3b8ba5e1c9ed" member_type: VOTER last_known_addr { host: "127.1.135.254" port: 46585 } }
I20260923 22:16:11.949536 20185 sys_catalog.cc:455] T 00000000000000000000000000000000 P 141301c78aa44ae1bf7a3b8ba5e1c9ed [sys.catalog]: SysCatalogTable state changed. Reason: New leader 141301c78aa44ae1bf7a3b8ba5e1c9ed. Latest consensus state: current_term: 1 leader_uuid: "141301c78aa44ae1bf7a3b8ba5e1c9ed" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "141301c78aa44ae1bf7a3b8ba5e1c9ed" member_type: VOTER last_known_addr { host: "127.1.135.254" port: 46585 } } }
I20260923 22:16:11.949623 20185 sys_catalog.cc:458] T 00000000000000000000000000000000 P 141301c78aa44ae1bf7a3b8ba5e1c9ed [sys.catalog]: This master's current role is: LEADER
I20260923 22:16:11.950014 20192 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260923 22:16:11.950063 1567 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskRiQe_a/build/release/bin/kudu as pid 20113
I20260923 22:16:11.950188 1567 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskRiQe_a/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790201539513534-1567-0/minicluster-data/master-0/wal/instance
I20260923 22:16:11.949602 20184 sys_catalog.cc:455] T 00000000000000000000000000000000 P 141301c78aa44ae1bf7a3b8ba5e1c9ed [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "141301c78aa44ae1bf7a3b8ba5e1c9ed" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "141301c78aa44ae1bf7a3b8ba5e1c9ed" member_type: VOTER last_known_addr { host: "127.1.135.254" port: 46585 } } }
I20260923 22:16:11.950527 20192 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260923 22:16:11.950549 1567 external_mini_cluster.cc:1428] Setting key 636f338101296393a2ca0cc9815fcbd3
I20260923 22:16:11.950536 20184 sys_catalog.cc:458] T 00000000000000000000000000000000 P 141301c78aa44ae1bf7a3b8ba5e1c9ed [sys.catalog]: This master's current role is: LEADER
I20260923 22:16:11.952047 20192 catalog_manager.cc:1347] Generated new cluster ID: 380a80ced0df45638646809be9c7e611
I20260923 22:16:11.952118 20192 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260923 22:16:11.956560 20202 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 22:16:11.950995 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37602 (local address 127.1.135.254:46585)
0923 22:16:11.951410 (+ 415us) server_negotiation.cc:206] Beginning negotiation
0923 22:16:11.951418 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0923 22:16:11.951445 (+ 27us) server_negotiation.cc:407] Connection header received
0923 22:16:11.951567 (+ 122us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 22:16:11.951573 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 22:16:11.951638 (+ 65us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 22:16:11.951767 (+ 129us) server_negotiation.cc:226] Negotiated authn=SASL
0923 22:16:11.952051 (+ 284us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 22:16:11.953420 (+ 1369us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 22:16:11.954486 (+ 1066us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 22:16:11.955127 (+ 641us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 22:16:11.955259 (+ 132us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 22:16:11.955860 (+ 601us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0923 22:16:11.955864 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0923 22:16:11.955864 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0923 22:16:11.956083 (+ 219us) server_negotiation.cc:922] calling sasl_server_start()
0923 22:16:11.956127 (+ 44us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0923 22:16:11.956141 (+ 14us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0923 22:16:11.956202 (+ 61us) server_negotiation.cc:1035] Waiting for connection context
0923 22:16:11.956277 (+ 75us) server_negotiation.cc:299] Negotiation successful
0923 22:16:11.956351 (+ 74us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":346,"thread_start_us":97,"threads_started":1}
I20260923 22:16:11.962401 20192 catalog_manager.cc:1370] Generated new certificate authority record
I20260923 22:16:11.962875 20192 catalog_manager.cc:1504] Loading token signing keys...
I20260923 22:16:11.967855 20192 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 141301c78aa44ae1bf7a3b8ba5e1c9ed: Generated new TSK 0
I20260923 22:16:11.968022 20192 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260923 22:16:11.969440 1567 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskRiQe_a/build/release/bin/kudu
/tmp/dist-test-taskRiQe_a/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskRiQe_a/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790201539513534-1567-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskRiQe_a/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790201539513534-1567-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskRiQe_a/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790201539513534-1567-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskRiQe_a/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790201539513534-1567-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.1.135.193:0
--local_ip_for_outbound_sockets=127.1.135.193
--webserver_interface=127.1.135.193
--webserver_port=0
--tserver_master_addrs=127.1.135.254:46585
--builtin_ntp_servers=127.1.135.212:41107
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
W20260923 22:16:12.050067 20204 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260923 22:16:12.050263 20204 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260923 22:16:12.050285 20204 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260923 22:16:12.051571 20204 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260923 22:16:12.051613 20204 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260923 22:16:12.051640 20204 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.1.135.193
I20260923 22:16:12.053123 20204 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.1.135.212:41107
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskRiQe_a/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790201539513534-1567-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskRiQe_a/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790201539513534-1567-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.1.135.193:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskRiQe_a/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790201539513534-1567-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.1.135.193
--webserver_port=0
--tserver_master_addrs=127.1.135.254:46585
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.20204
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.1.135.193
--log_dir=/tmp/dist-test-taskRiQe_a/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790201539513534-1567-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 23 Sep 2026 20:53:37 UTC on ccycloud.quasar-rkviyx.root.comops.site
build id 4668083
I20260923 22:16:12.053356 20204 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260923 22:16:12.053642 20204 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260923 22:16:12.058367 20209 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260923 22:16:12.058514 20210 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260923 22:16:12.058745 20212 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260923 22:16:12.058919 20204 server_base.cc:1034] running on GCE node
I20260923 22:16:12.059128 20204 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260923 22:16:12.059389 20204 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260923 22:16:12.060542 20204 hybrid_clock.cc:648] HybridClock initialized: now 1790201772060531 us; error 27 us; skew 500 ppm
I20260923 22:16:12.061892 20204 webserver.cc:460] Webserver started at http://127.1.135.193:33547/ using document root <none> and password file <none>
I20260923 22:16:12.062122 20204 fs_manager.cc:359] Metadata directory not provided
I20260923 22:16:12.062191 20204 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260923 22:16:12.062357 20204 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260923 22:16:12.063768 20204 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskRiQe_a/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790201539513534-1567-0/minicluster-data/ts-0/data/instance:
uuid: "107ca9a428674dc68bc806bd62136d2d"
format_stamp: "Formatted at 2026-09-23 22:16:12 on dist-test-slave-s2fj"
server_key: "9be389f4d962215a718bc91423b1caac"
server_key_iv: "c8e02e5a7bbf4452836fecba1405d81a"
server_key_version: "encryptionkey@0"
I20260923 22:16:12.064200 20204 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskRiQe_a/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790201539513534-1567-0/minicluster-data/ts-0/wal/instance:
uuid: "107ca9a428674dc68bc806bd62136d2d"
format_stamp: "Formatted at 2026-09-23 22:16:12 on dist-test-slave-s2fj"
server_key: "9be389f4d962215a718bc91423b1caac"
server_key_iv: "c8e02e5a7bbf4452836fecba1405d81a"
server_key_version: "encryptionkey@0"
I20260923 22:16:12.065529 20204 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.001s
I20260923 22:16:12.066294 20218 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260923 22:16:12.066531 20204 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.001s
I20260923 22:16:12.066637 20204 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskRiQe_a/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790201539513534-1567-0/minicluster-data/ts-0/data,/tmp/dist-test-taskRiQe_a/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790201539513534-1567-0/minicluster-data/ts-0/wal
uuid: "107ca9a428674dc68bc806bd62136d2d"
format_stamp: "Formatted at 2026-09-23 22:16:12 on dist-test-slave-s2fj"
server_key: "9be389f4d962215a718bc91423b1caac"
server_key_iv: "c8e02e5a7bbf4452836fecba1405d81a"
server_key_version: "encryptionkey@0"
I20260923 22:16:12.066725 20204 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskRiQe_a/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790201539513534-1567-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskRiQe_a/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790201539513534-1567-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskRiQe_a/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790201539513534-1567-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260923 22:16:12.077569 20204 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260923 22:16:12.077925 20204 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260923 22:16:12.078068 20204 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260923 22:16:12.078308 20204 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260923 22:16:12.078656 20204 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260923 22:16:12.078706 20204 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260923 22:16:12.078747 20204 ts_tablet_manager.cc:610] Registered 0 tablets
I20260923 22:16:12.078773 20204 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260923 22:16:12.086407 20204 rpc_server.cc:307] RPC server started. Bound to: 127.1.135.193:35231
I20260923 22:16:12.086437 20331 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.1.135.193:35231 every 8 connection(s)
I20260923 22:16:12.086820 20204 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskRiQe_a/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790201539513534-1567-0/minicluster-data/ts-0/data/info.pb
I20260923 22:16:12.092350 20202 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 22:16:12.087515 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.135.193:35687 (local address 127.1.135.254:46585)
0923 22:16:12.087581 (+ 66us) server_negotiation.cc:206] Beginning negotiation
0923 22:16:12.087583 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0923 22:16:12.087814 (+ 231us) server_negotiation.cc:407] Connection header received
0923 22:16:12.088356 (+ 542us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 22:16:12.088357 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 22:16:12.088370 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 22:16:12.088450 (+ 80us) server_negotiation.cc:226] Negotiated authn=SASL
0923 22:16:12.089052 (+ 602us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 22:16:12.089908 (+ 856us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 22:16:12.090838 (+ 930us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 22:16:12.091117 (+ 279us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 22:16:12.091196 (+ 79us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 22:16:12.092027 (+ 831us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0923 22:16:12.092028 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0923 22:16:12.092029 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0923 22:16:12.092034 (+ 5us) server_negotiation.cc:922] calling sasl_server_start()
0923 22:16:12.092050 (+ 16us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0923 22:16:12.092055 (+ 5us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0923 22:16:12.092158 (+ 103us) server_negotiation.cc:1035] Waiting for connection context
0923 22:16:12.092245 (+ 87us) server_negotiation.cc:299] Negotiation successful
0923 22:16:12.092272 (+ 27us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":37}
I20260923 22:16:12.092568 20334 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 22:16:12.087485 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.135.254:46585 (local address 127.1.135.193:35687)
0923 22:16:12.087683 (+ 198us) negotiation.cc:107] Waiting for socket to connect
0923 22:16:12.087721 (+ 38us) client_negotiation.cc:174] Beginning negotiation
0923 22:16:12.088250 (+ 529us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0923 22:16:12.088513 (+ 263us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0923 22:16:12.088516 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0923 22:16:12.088524 (+ 8us) client_negotiation.cc:189] Negotiated authn=SASL
0923 22:16:12.088977 (+ 453us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 22:16:12.088981 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 22:16:12.089999 (+ 1018us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 22:16:12.090000 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 22:16:12.090753 (+ 753us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 22:16:12.090755 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 22:16:12.091208 (+ 453us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 22:16:12.091208 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 22:16:12.091308 (+ 100us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 22:16:12.091891 (+ 583us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0923 22:16:12.091891 (+ 0us) client_negotiation.cc:656] calling sasl_client_start()
0923 22:16:12.091915 (+ 24us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0923 22:16:12.091935 (+ 20us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0923 22:16:12.092163 (+ 228us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0923 22:16:12.092168 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0923 22:16:12.092170 (+ 2us) client_negotiation.cc:769] Sending connection context
0923 22:16:12.092222 (+ 52us) client_negotiation.cc:240] Negotiation successful
0923 22:16:12.092294 (+ 72us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":138,"thread_start_us":112,"threads_started":1}
I20260923 22:16:12.093065 20332 heartbeater.cc:347] Connected to a master server at 127.1.135.254:46585
I20260923 22:16:12.093148 20332 heartbeater.cc:464] Registering TS with master...
I20260923 22:16:12.093360 20332 heartbeater.cc:511] Master 127.1.135.254:46585 requested a full tablet report, sending...
I20260923 22:16:12.093890 20144 ts_manager.cc:194] Registered new tserver with Master: 107ca9a428674dc68bc806bd62136d2d (127.1.135.193:35231)
I20260923 22:16:12.094352 1567 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskRiQe_a/build/release/bin/kudu as pid 20204
I20260923 22:16:12.094460 1567 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskRiQe_a/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790201539513534-1567-0/minicluster-data/ts-0/wal/instance
I20260923 22:16:12.094674 1567 external_mini_cluster.cc:1428] Setting key b1c9a3def3480b705ba1e33e099be086
I20260923 22:16:12.094751 20144 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.1.135.193:35687
I20260923 22:16:12.095669 1567 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskRiQe_a/build/release/bin/kudu
/tmp/dist-test-taskRiQe_a/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskRiQe_a/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790201539513534-1567-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskRiQe_a/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790201539513534-1567-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskRiQe_a/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790201539513534-1567-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskRiQe_a/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790201539513534-1567-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.1.135.194:0
--local_ip_for_outbound_sockets=127.1.135.194
--webserver_interface=127.1.135.194
--webserver_port=0
--tserver_master_addrs=127.1.135.254:46585
--builtin_ntp_servers=127.1.135.212:41107
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
W20260923 22:16:12.174927 20335 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260923 22:16:12.175223 20335 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260923 22:16:12.175258 20335 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260923 22:16:12.176786 20335 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260923 22:16:12.176826 20335 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260923 22:16:12.176851 20335 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.1.135.194
I20260923 22:16:12.178188 20335 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.1.135.212:41107
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskRiQe_a/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790201539513534-1567-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskRiQe_a/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790201539513534-1567-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.1.135.194:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskRiQe_a/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790201539513534-1567-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.1.135.194
--webserver_port=0
--tserver_master_addrs=127.1.135.254:46585
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.20335
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.1.135.194
--log_dir=/tmp/dist-test-taskRiQe_a/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790201539513534-1567-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 23 Sep 2026 20:53:37 UTC on ccycloud.quasar-rkviyx.root.comops.site
build id 4668083
I20260923 22:16:12.178469 20335 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260923 22:16:12.178831 20335 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260923 22:16:12.183661 20341 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260923 22:16:12.183772 20335 server_base.cc:1034] running on GCE node
W20260923 22:16:12.183673 20343 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260923 22:16:12.183642 20340 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260923 22:16:12.184041 20335 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260923 22:16:12.184327 20335 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260923 22:16:12.185489 20335 hybrid_clock.cc:648] HybridClock initialized: now 1790201772185486 us; error 38 us; skew 500 ppm
I20260923 22:16:12.186712 20335 webserver.cc:460] Webserver started at http://127.1.135.194:43697/ using document root <none> and password file <none>
I20260923 22:16:12.186960 20335 fs_manager.cc:359] Metadata directory not provided
I20260923 22:16:12.187029 20335 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260923 22:16:12.187186 20335 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260923 22:16:12.188318 20335 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskRiQe_a/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790201539513534-1567-0/minicluster-data/ts-1/data/instance:
uuid: "347d81c7769b4d60a0b554924c203e9b"
format_stamp: "Formatted at 2026-09-23 22:16:12 on dist-test-slave-s2fj"
server_key: "f59a17a2e5a497b664f1a7e59bb37ff7"
server_key_iv: "c4d00d98673981ddd2d3694aeca7966e"
server_key_version: "encryptionkey@0"
I20260923 22:16:12.188714 20335 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskRiQe_a/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790201539513534-1567-0/minicluster-data/ts-1/wal/instance:
uuid: "347d81c7769b4d60a0b554924c203e9b"
format_stamp: "Formatted at 2026-09-23 22:16:12 on dist-test-slave-s2fj"
server_key: "f59a17a2e5a497b664f1a7e59bb37ff7"
server_key_iv: "c4d00d98673981ddd2d3694aeca7966e"
server_key_version: "encryptionkey@0"
I20260923 22:16:12.190011 20335 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.000s
I20260923 22:16:12.190768 20349 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260923 22:16:12.190948 20335 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260923 22:16:12.191038 20335 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskRiQe_a/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790201539513534-1567-0/minicluster-data/ts-1/data,/tmp/dist-test-taskRiQe_a/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790201539513534-1567-0/minicluster-data/ts-1/wal
uuid: "347d81c7769b4d60a0b554924c203e9b"
format_stamp: "Formatted at 2026-09-23 22:16:12 on dist-test-slave-s2fj"
server_key: "f59a17a2e5a497b664f1a7e59bb37ff7"
server_key_iv: "c4d00d98673981ddd2d3694aeca7966e"
server_key_version: "encryptionkey@0"
I20260923 22:16:12.191120 20335 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskRiQe_a/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790201539513534-1567-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskRiQe_a/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790201539513534-1567-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskRiQe_a/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790201539513534-1567-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260923 22:16:12.223711 20335 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260923 22:16:12.224031 20335 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260923 22:16:12.224211 20335 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260923 22:16:12.224450 20335 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260923 22:16:12.224774 20335 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260923 22:16:12.224830 20335 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260923 22:16:12.224877 20335 ts_tablet_manager.cc:610] Registered 0 tablets
I20260923 22:16:12.224907 20335 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260923 22:16:12.230504 20335 rpc_server.cc:307] RPC server started. Bound to: 127.1.135.194:34643
I20260923 22:16:12.230548 20462 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.1.135.194:34643 every 8 connection(s)
I20260923 22:16:12.230841 20335 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskRiQe_a/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790201539513534-1567-0/minicluster-data/ts-1/data/info.pb
I20260923 22:16:12.235649 20202 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 22:16:12.231488 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.135.194:42515 (local address 127.1.135.254:46585)
0923 22:16:12.231541 (+ 53us) server_negotiation.cc:206] Beginning negotiation
0923 22:16:12.231544 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0923 22:16:12.231835 (+ 291us) server_negotiation.cc:407] Connection header received
0923 22:16:12.232362 (+ 527us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 22:16:12.232364 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 22:16:12.232373 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 22:16:12.232427 (+ 54us) server_negotiation.cc:226] Negotiated authn=SASL
0923 22:16:12.233071 (+ 644us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 22:16:12.233544 (+ 473us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 22:16:12.234466 (+ 922us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 22:16:12.234644 (+ 178us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 22:16:12.234692 (+ 48us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 22:16:12.235436 (+ 744us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0923 22:16:12.235437 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0923 22:16:12.235437 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0923 22:16:12.235441 (+ 4us) server_negotiation.cc:922] calling sasl_server_start()
0923 22:16:12.235452 (+ 11us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0923 22:16:12.235456 (+ 4us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0923 22:16:12.235496 (+ 40us) server_negotiation.cc:1035] Waiting for connection context
0923 22:16:12.235577 (+ 81us) server_negotiation.cc:299] Negotiation successful
0923 22:16:12.235596 (+ 19us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":23}
I20260923 22:16:12.235814 20465 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 22:16:12.231495 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.135.254:46585 (local address 127.1.135.194:42515)
0923 22:16:12.231683 (+ 188us) negotiation.cc:107] Waiting for socket to connect
0923 22:16:12.231721 (+ 38us) client_negotiation.cc:174] Beginning negotiation
0923 22:16:12.232270 (+ 549us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0923 22:16:12.232509 (+ 239us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0923 22:16:12.232513 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0923 22:16:12.232521 (+ 8us) client_negotiation.cc:189] Negotiated authn=SASL
0923 22:16:12.232999 (+ 478us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 22:16:12.233002 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 22:16:12.233618 (+ 616us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 22:16:12.233619 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 22:16:12.234393 (+ 774us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 22:16:12.234395 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 22:16:12.234714 (+ 319us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 22:16:12.234715 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 22:16:12.234817 (+ 102us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 22:16:12.235313 (+ 496us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0923 22:16:12.235313 (+ 0us) client_negotiation.cc:656] calling sasl_client_start()
0923 22:16:12.235339 (+ 26us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0923 22:16:12.235359 (+ 20us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0923 22:16:12.235507 (+ 148us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0923 22:16:12.235511 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0923 22:16:12.235513 (+ 2us) client_negotiation.cc:769] Sending connection context
0923 22:16:12.235561 (+ 48us) client_negotiation.cc:240] Negotiation successful
0923 22:16:12.235633 (+ 72us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":133,"thread_start_us":88,"threads_started":1}
I20260923 22:16:12.236356 20463 heartbeater.cc:347] Connected to a master server at 127.1.135.254:46585
I20260923 22:16:12.236459 20463 heartbeater.cc:464] Registering TS with master...
I20260923 22:16:12.236624 20463 heartbeater.cc:511] Master 127.1.135.254:46585 requested a full tablet report, sending...
I20260923 22:16:12.236970 20144 ts_manager.cc:194] Registered new tserver with Master: 347d81c7769b4d60a0b554924c203e9b (127.1.135.194:34643)
I20260923 22:16:12.237295 20144 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.1.135.194:42515
I20260923 22:16:12.240471 1567 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskRiQe_a/build/release/bin/kudu as pid 20335
I20260923 22:16:12.240556 1567 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskRiQe_a/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790201539513534-1567-0/minicluster-data/ts-1/wal/instance
I20260923 22:16:12.240717 1567 external_mini_cluster.cc:1428] Setting key dfb03d88cf8ebd9c4edb8dcfb19955dd
I20260923 22:16:12.241513 1567 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskRiQe_a/build/release/bin/kudu
/tmp/dist-test-taskRiQe_a/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskRiQe_a/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790201539513534-1567-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskRiQe_a/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790201539513534-1567-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskRiQe_a/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790201539513534-1567-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskRiQe_a/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790201539513534-1567-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.1.135.195:0
--local_ip_for_outbound_sockets=127.1.135.195
--webserver_interface=127.1.135.195
--webserver_port=0
--tserver_master_addrs=127.1.135.254:46585
--builtin_ntp_servers=127.1.135.212:41107
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
W20260923 22:16:12.319592 20466 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260923 22:16:12.319770 20466 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260923 22:16:12.319790 20466 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260923 22:16:12.320901 20466 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260923 22:16:12.320940 20466 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260923 22:16:12.320963 20466 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.1.135.195
I20260923 22:16:12.322237 20466 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.1.135.212:41107
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskRiQe_a/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790201539513534-1567-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskRiQe_a/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790201539513534-1567-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.1.135.195:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskRiQe_a/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790201539513534-1567-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.1.135.195
--webserver_port=0
--tserver_master_addrs=127.1.135.254:46585
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.20466
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.1.135.195
--log_dir=/tmp/dist-test-taskRiQe_a/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790201539513534-1567-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 23 Sep 2026 20:53:37 UTC on ccycloud.quasar-rkviyx.root.comops.site
build id 4668083
I20260923 22:16:12.322484 20466 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260923 22:16:12.322750 20466 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260923 22:16:12.327390 20474 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260923 22:16:12.327459 20466 server_base.cc:1034] running on GCE node
W20260923 22:16:12.327399 20472 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260923 22:16:12.327399 20471 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260923 22:16:12.327806 20466 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260923 22:16:12.328034 20466 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260923 22:16:12.329205 20466 hybrid_clock.cc:648] HybridClock initialized: now 1790201772329196 us; error 27 us; skew 500 ppm
I20260923 22:16:12.330390 20466 webserver.cc:460] Webserver started at http://127.1.135.195:38709/ using document root <none> and password file <none>
I20260923 22:16:12.330622 20466 fs_manager.cc:359] Metadata directory not provided
I20260923 22:16:12.330691 20466 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260923 22:16:12.330822 20466 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260923 22:16:12.331848 20466 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskRiQe_a/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790201539513534-1567-0/minicluster-data/ts-2/data/instance:
uuid: "fed186bc78f24f0a88da24a2a86c9979"
format_stamp: "Formatted at 2026-09-23 22:16:12 on dist-test-slave-s2fj"
server_key: "5409fa1c0b4e8b226dba811836d25ae2"
server_key_iv: "6c086461b9cfb6753164174da61219d6"
server_key_version: "encryptionkey@0"
I20260923 22:16:12.332237 20466 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskRiQe_a/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790201539513534-1567-0/minicluster-data/ts-2/wal/instance:
uuid: "fed186bc78f24f0a88da24a2a86c9979"
format_stamp: "Formatted at 2026-09-23 22:16:12 on dist-test-slave-s2fj"
server_key: "5409fa1c0b4e8b226dba811836d25ae2"
server_key_iv: "6c086461b9cfb6753164174da61219d6"
server_key_version: "encryptionkey@0"
I20260923 22:16:12.333524 20466 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.001s sys 0.000s
I20260923 22:16:12.334215 20480 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260923 22:16:12.334374 20466 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260923 22:16:12.334470 20466 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskRiQe_a/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790201539513534-1567-0/minicluster-data/ts-2/data,/tmp/dist-test-taskRiQe_a/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790201539513534-1567-0/minicluster-data/ts-2/wal
uuid: "fed186bc78f24f0a88da24a2a86c9979"
format_stamp: "Formatted at 2026-09-23 22:16:12 on dist-test-slave-s2fj"
server_key: "5409fa1c0b4e8b226dba811836d25ae2"
server_key_iv: "6c086461b9cfb6753164174da61219d6"
server_key_version: "encryptionkey@0"
I20260923 22:16:12.334553 20466 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskRiQe_a/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790201539513534-1567-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskRiQe_a/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790201539513534-1567-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskRiQe_a/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790201539513534-1567-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260923 22:16:12.349848 20466 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260923 22:16:12.350167 20466 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260923 22:16:12.350374 20466 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260923 22:16:12.350597 20466 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260923 22:16:12.350992 20466 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260923 22:16:12.351063 20466 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260923 22:16:12.351106 20466 ts_tablet_manager.cc:610] Registered 0 tablets
I20260923 22:16:12.351128 20466 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260923 22:16:12.356889 20466 rpc_server.cc:307] RPC server started. Bound to: 127.1.135.195:37103
I20260923 22:16:12.356968 20593 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.1.135.195:37103 every 8 connection(s)
I20260923 22:16:12.357304 20466 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskRiQe_a/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790201539513534-1567-0/minicluster-data/ts-2/data/info.pb
I20260923 22:16:12.365538 20202 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 22:16:12.358049 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.135.195:38911 (local address 127.1.135.254:46585)
0923 22:16:12.358097 (+ 48us) server_negotiation.cc:206] Beginning negotiation
0923 22:16:12.358099 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0923 22:16:12.358335 (+ 236us) server_negotiation.cc:407] Connection header received
0923 22:16:12.358789 (+ 454us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 22:16:12.358789 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 22:16:12.358799 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 22:16:12.358856 (+ 57us) server_negotiation.cc:226] Negotiated authn=SASL
0923 22:16:12.359478 (+ 622us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 22:16:12.359966 (+ 488us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 22:16:12.360969 (+ 1003us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 22:16:12.361153 (+ 184us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 22:16:12.361206 (+ 53us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 22:16:12.365304 (+ 4098us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0923 22:16:12.365306 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0923 22:16:12.365306 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0923 22:16:12.365310 (+ 4us) server_negotiation.cc:922] calling sasl_server_start()
0923 22:16:12.365324 (+ 14us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0923 22:16:12.365329 (+ 5us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0923 22:16:12.365375 (+ 46us) server_negotiation.cc:1035] Waiting for connection context
0923 22:16:12.365454 (+ 79us) server_negotiation.cc:299] Negotiation successful
0923 22:16:12.365481 (+ 27us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":23}
I20260923 22:16:12.365737 20596 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 22:16:12.358013 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.135.254:46585 (local address 127.1.135.195:38911)
0923 22:16:12.358204 (+ 191us) negotiation.cc:107] Waiting for socket to connect
0923 22:16:12.358243 (+ 39us) client_negotiation.cc:174] Beginning negotiation
0923 22:16:12.358705 (+ 462us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0923 22:16:12.358906 (+ 201us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0923 22:16:12.358909 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0923 22:16:12.358916 (+ 7us) client_negotiation.cc:189] Negotiated authn=SASL
0923 22:16:12.359403 (+ 487us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 22:16:12.359408 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 22:16:12.360048 (+ 640us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 22:16:12.360049 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 22:16:12.360887 (+ 838us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 22:16:12.360889 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 22:16:12.361228 (+ 339us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 22:16:12.361229 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 22:16:12.361338 (+ 109us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 22:16:12.365177 (+ 3839us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0923 22:16:12.365177 (+ 0us) client_negotiation.cc:656] calling sasl_client_start()
0923 22:16:12.365204 (+ 27us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0923 22:16:12.365224 (+ 20us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0923 22:16:12.365386 (+ 162us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0923 22:16:12.365390 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0923 22:16:12.365392 (+ 2us) client_negotiation.cc:769] Sending connection context
0923 22:16:12.365442 (+ 50us) client_negotiation.cc:240] Negotiation successful
0923 22:16:12.365513 (+ 71us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":131,"thread_start_us":96,"threads_started":1}
I20260923 22:16:12.365846 1567 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskRiQe_a/build/release/bin/kudu as pid 20466
I20260923 22:16:12.365988 1567 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskRiQe_a/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790201539513534-1567-0/minicluster-data/ts-2/wal/instance
I20260923 22:16:12.366223 1567 external_mini_cluster.cc:1428] Setting key 7e23d0362164a1084790ab321cf870c8
I20260923 22:16:12.366271 20594 heartbeater.cc:347] Connected to a master server at 127.1.135.254:46585
I20260923 22:16:12.366360 20594 heartbeater.cc:464] Registering TS with master...
I20260923 22:16:12.366540 20594 heartbeater.cc:511] Master 127.1.135.254:46585 requested a full tablet report, sending...
I20260923 22:16:12.366850 20144 ts_manager.cc:194] Registered new tserver with Master: fed186bc78f24f0a88da24a2a86c9979 (127.1.135.195:37103)
I20260923 22:16:12.367209 1567 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260923 22:16:12.367251 20144 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.1.135.195:38911
I20260923 22:16:12.371273 20202 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 22:16:12.368634 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37614 (local address 127.1.135.254:46585)
0923 22:16:12.368669 (+ 35us) server_negotiation.cc:206] Beginning negotiation
0923 22:16:12.368670 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0923 22:16:12.368788 (+ 118us) server_negotiation.cc:407] Connection header received
0923 22:16:12.368858 (+ 70us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 22:16:12.368859 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 22:16:12.368868 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 22:16:12.368913 (+ 45us) server_negotiation.cc:226] Negotiated authn=SASL
0923 22:16:12.369019 (+ 106us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 22:16:12.369449 (+ 430us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 22:16:12.370156 (+ 707us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 22:16:12.370322 (+ 166us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 22:16:12.370369 (+ 47us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 22:16:12.371040 (+ 671us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0923 22:16:12.371041 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0923 22:16:12.371041 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0923 22:16:12.371044 (+ 3us) server_negotiation.cc:922] calling sasl_server_start()
0923 22:16:12.371053 (+ 9us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0923 22:16:12.371056 (+ 3us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0923 22:16:12.371109 (+ 53us) server_negotiation.cc:1035] Waiting for connection context
0923 22:16:12.371192 (+ 83us) server_negotiation.cc:299] Negotiation successful
0923 22:16:12.371201 (+ 9us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":17}
I20260923 22:16:12.372849 20144 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:37614:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260923 22:16:12.374339 20144 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260923 22:16:12.380050 20607 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 22:16:12.376589 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59494 (local address 127.1.135.195:37103)
0923 22:16:12.376732 (+ 143us) server_negotiation.cc:206] Beginning negotiation
0923 22:16:12.376736 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0923 22:16:12.376746 (+ 10us) server_negotiation.cc:407] Connection header received
0923 22:16:12.376775 (+ 29us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 22:16:12.376777 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 22:16:12.376791 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 22:16:12.376855 (+ 64us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0923 22:16:12.377087 (+ 232us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 22:16:12.378143 (+ 1056us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 22:16:12.379104 (+ 961us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 22:16:12.379670 (+ 566us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 22:16:12.379762 (+ 92us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 22:16:12.379851 (+ 89us) server_negotiation.cc:1035] Waiting for connection context
0923 22:16:12.379942 (+ 91us) server_negotiation.cc:299] Negotiation successful
0923 22:16:12.379974 (+ 32us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":114,"thread_start_us":70,"threads_started":1}
I20260923 22:16:12.380046 20603 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 22:16:12.376432 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.135.195:37103 (local address 127.0.0.1:59494)
0923 22:16:12.376567 (+ 135us) negotiation.cc:107] Waiting for socket to connect
0923 22:16:12.376578 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0923 22:16:12.376623 (+ 45us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0923 22:16:12.376907 (+ 284us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0923 22:16:12.376908 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0923 22:16:12.376911 (+ 3us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0923 22:16:12.376954 (+ 43us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 22:16:12.376959 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 22:16:12.378228 (+ 1269us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 22:16:12.378229 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 22:16:12.379006 (+ 777us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 22:16:12.379008 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 22:16:12.379764 (+ 756us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 22:16:12.379765 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 22:16:12.379842 (+ 77us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 22:16:12.379855 (+ 13us) client_negotiation.cc:769] Sending connection context
0923 22:16:12.379929 (+ 74us) client_negotiation.cc:240] Negotiation successful
0923 22:16:12.379980 (+ 51us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":120,"thread_start_us":67,"threads_started":1}
I20260923 22:16:12.380975 20528 tablet_service.cc:1467] Processing CreateTablet for tablet afbf4e4aa6ce4ab6ad0d06feaa65e7f1 (DEFAULT_TABLE table=test-table [id=c09836df255846f9b9b97127f4257906]), partition=RANGE (key) PARTITION UNBOUNDED
I20260923 22:16:12.381299 20528 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet afbf4e4aa6ce4ab6ad0d06feaa65e7f1. 1 dirs total, 0 dirs full, 0 dirs failed
I20260923 22:16:12.381371 20602 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 22:16:12.376235 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.135.194:34643 (local address 127.0.0.1:37404)
0923 22:16:12.376393 (+ 158us) negotiation.cc:107] Waiting for socket to connect
0923 22:16:12.376427 (+ 34us) client_negotiation.cc:174] Beginning negotiation
0923 22:16:12.376485 (+ 58us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0923 22:16:12.376921 (+ 436us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0923 22:16:12.376922 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0923 22:16:12.376923 (+ 1us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0923 22:16:12.376954 (+ 31us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 22:16:12.376956 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 22:16:12.379009 (+ 2053us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 22:16:12.379010 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 22:16:12.380181 (+ 1171us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 22:16:12.380183 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 22:16:12.381097 (+ 914us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 22:16:12.381098 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 22:16:12.381161 (+ 63us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 22:16:12.381165 (+ 4us) client_negotiation.cc:769] Sending connection context
0923 22:16:12.381251 (+ 86us) client_negotiation.cc:240] Negotiation successful
0923 22:16:12.381291 (+ 40us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":138,"mutex_wait_us":3,"thread_start_us":66,"threads_started":1}
I20260923 22:16:12.381407 20605 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 22:16:12.376336 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37404 (local address 127.1.135.194:34643)
0923 22:16:12.376738 (+ 402us) server_negotiation.cc:206] Beginning negotiation
0923 22:16:12.376742 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0923 22:16:12.376759 (+ 17us) server_negotiation.cc:407] Connection header received
0923 22:16:12.376814 (+ 55us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 22:16:12.376816 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 22:16:12.376839 (+ 23us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 22:16:12.376917 (+ 78us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0923 22:16:12.378251 (+ 1334us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 22:16:12.378903 (+ 652us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 22:16:12.380279 (+ 1376us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 22:16:12.381006 (+ 727us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 22:16:12.381081 (+ 75us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 22:16:12.381161 (+ 80us) server_negotiation.cc:1035] Waiting for connection context
0923 22:16:12.381272 (+ 111us) server_negotiation.cc:299] Negotiation successful
0923 22:16:12.381321 (+ 49us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":369,"thread_start_us":80,"threads_started":1}
I20260923 22:16:12.381680 20606 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 22:16:12.376235 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.135.193:35231 (local address 127.0.0.1:40244)
0923 22:16:12.376546 (+ 311us) negotiation.cc:107] Waiting for socket to connect
0923 22:16:12.376557 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0923 22:16:12.376623 (+ 66us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0923 22:16:12.376827 (+ 204us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0923 22:16:12.376829 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0923 22:16:12.376832 (+ 3us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0923 22:16:12.376894 (+ 62us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 22:16:12.376896 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 22:16:12.379090 (+ 2194us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 22:16:12.379091 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 22:16:12.380181 (+ 1090us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 22:16:12.380183 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 22:16:12.381451 (+ 1268us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 22:16:12.381452 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 22:16:12.381508 (+ 56us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 22:16:12.381512 (+ 4us) client_negotiation.cc:769] Sending connection context
0923 22:16:12.381566 (+ 54us) client_negotiation.cc:240] Negotiation successful
0923 22:16:12.381607 (+ 41us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":291,"mutex_wait_us":6,"spinlock_wait_cycles":384,"thread_start_us":82,"threads_started":1}
I20260923 22:16:12.381950 20604 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 22:16:12.376332 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40244 (local address 127.1.135.193:35231)
0923 22:16:12.376514 (+ 182us) server_negotiation.cc:206] Beginning negotiation
0923 22:16:12.376519 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0923 22:16:12.376626 (+ 107us) server_negotiation.cc:407] Connection header received
0923 22:16:12.376702 (+ 76us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 22:16:12.376704 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 22:16:12.376721 (+ 17us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 22:16:12.376809 (+ 88us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0923 22:16:12.377036 (+ 227us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 22:16:12.378144 (+ 1108us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 22:16:12.380279 (+ 2135us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 22:16:12.381006 (+ 727us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 22:16:12.381081 (+ 75us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 22:16:12.381160 (+ 79us) server_negotiation.cc:1035] Waiting for connection context
0923 22:16:12.381829 (+ 669us) server_negotiation.cc:299] Negotiation successful
0923 22:16:12.381874 (+ 45us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":148,"thread_start_us":80,"threads_started":1}
I20260923 22:16:12.382094 20397 tablet_service.cc:1467] Processing CreateTablet for tablet afbf4e4aa6ce4ab6ad0d06feaa65e7f1 (DEFAULT_TABLE table=test-table [id=c09836df255846f9b9b97127f4257906]), partition=RANGE (key) PARTITION UNBOUNDED
I20260923 22:16:12.382405 20397 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet afbf4e4aa6ce4ab6ad0d06feaa65e7f1. 1 dirs total, 0 dirs full, 0 dirs failed
I20260923 22:16:12.382624 20266 tablet_service.cc:1467] Processing CreateTablet for tablet afbf4e4aa6ce4ab6ad0d06feaa65e7f1 (DEFAULT_TABLE table=test-table [id=c09836df255846f9b9b97127f4257906]), partition=RANGE (key) PARTITION UNBOUNDED
I20260923 22:16:12.382913 20266 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet afbf4e4aa6ce4ab6ad0d06feaa65e7f1. 1 dirs total, 0 dirs full, 0 dirs failed
I20260923 22:16:12.383862 20608 tablet_bootstrap.cc:492] T afbf4e4aa6ce4ab6ad0d06feaa65e7f1 P fed186bc78f24f0a88da24a2a86c9979: Bootstrap starting.
I20260923 22:16:12.384559 20608 tablet_bootstrap.cc:654] T afbf4e4aa6ce4ab6ad0d06feaa65e7f1 P fed186bc78f24f0a88da24a2a86c9979: Neither blocks nor log segments found. Creating new log.
I20260923 22:16:12.384858 20608 log.cc:824] T afbf4e4aa6ce4ab6ad0d06feaa65e7f1 P fed186bc78f24f0a88da24a2a86c9979: Log is configured to *not* fsync() on all Append() calls
I20260923 22:16:12.385583 20608 tablet_bootstrap.cc:492] T afbf4e4aa6ce4ab6ad0d06feaa65e7f1 P fed186bc78f24f0a88da24a2a86c9979: No bootstrap required, opened a new log
I20260923 22:16:12.385591 20610 tablet_bootstrap.cc:492] T afbf4e4aa6ce4ab6ad0d06feaa65e7f1 P 347d81c7769b4d60a0b554924c203e9b: Bootstrap starting.
I20260923 22:16:12.385727 20608 ts_tablet_manager.cc:1397] T afbf4e4aa6ce4ab6ad0d06feaa65e7f1 P fed186bc78f24f0a88da24a2a86c9979: Time spent bootstrapping tablet: real 0.002s user 0.002s sys 0.000s
I20260923 22:16:12.386072 20611 tablet_bootstrap.cc:492] T afbf4e4aa6ce4ab6ad0d06feaa65e7f1 P 107ca9a428674dc68bc806bd62136d2d: Bootstrap starting.
I20260923 22:16:12.386198 20610 tablet_bootstrap.cc:654] T afbf4e4aa6ce4ab6ad0d06feaa65e7f1 P 347d81c7769b4d60a0b554924c203e9b: Neither blocks nor log segments found. Creating new log.
I20260923 22:16:12.386538 20610 log.cc:824] T afbf4e4aa6ce4ab6ad0d06feaa65e7f1 P 347d81c7769b4d60a0b554924c203e9b: Log is configured to *not* fsync() on all Append() calls
I20260923 22:16:12.386659 20611 tablet_bootstrap.cc:654] T afbf4e4aa6ce4ab6ad0d06feaa65e7f1 P 107ca9a428674dc68bc806bd62136d2d: Neither blocks nor log segments found. Creating new log.
I20260923 22:16:12.386966 20611 log.cc:824] T afbf4e4aa6ce4ab6ad0d06feaa65e7f1 P 107ca9a428674dc68bc806bd62136d2d: Log is configured to *not* fsync() on all Append() calls
I20260923 22:16:12.387274 20610 tablet_bootstrap.cc:492] T afbf4e4aa6ce4ab6ad0d06feaa65e7f1 P 347d81c7769b4d60a0b554924c203e9b: No bootstrap required, opened a new log
I20260923 22:16:12.387352 20610 ts_tablet_manager.cc:1397] T afbf4e4aa6ce4ab6ad0d06feaa65e7f1 P 347d81c7769b4d60a0b554924c203e9b: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20260923 22:16:12.387622 20611 tablet_bootstrap.cc:492] T afbf4e4aa6ce4ab6ad0d06feaa65e7f1 P 107ca9a428674dc68bc806bd62136d2d: No bootstrap required, opened a new log
I20260923 22:16:12.387699 20611 ts_tablet_manager.cc:1397] T afbf4e4aa6ce4ab6ad0d06feaa65e7f1 P 107ca9a428674dc68bc806bd62136d2d: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20260923 22:16:12.388378 20608 raft_consensus.cc:348] T afbf4e4aa6ce4ab6ad0d06feaa65e7f1 P fed186bc78f24f0a88da24a2a86c9979 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "107ca9a428674dc68bc806bd62136d2d" member_type: VOTER last_known_addr { host: "127.1.135.193" port: 35231 } } peers { permanent_uuid: "fed186bc78f24f0a88da24a2a86c9979" member_type: VOTER last_known_addr { host: "127.1.135.195" port: 37103 } } peers { permanent_uuid: "347d81c7769b4d60a0b554924c203e9b" member_type: VOTER last_known_addr { host: "127.1.135.194" port: 34643 } }
I20260923 22:16:12.388625 20608 raft_consensus.cc:374] T afbf4e4aa6ce4ab6ad0d06feaa65e7f1 P fed186bc78f24f0a88da24a2a86c9979 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260923 22:16:12.388716 20608 raft_consensus.cc:729] T afbf4e4aa6ce4ab6ad0d06feaa65e7f1 P fed186bc78f24f0a88da24a2a86c9979 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: fed186bc78f24f0a88da24a2a86c9979, State: Initialized, Role: FOLLOWER
I20260923 22:16:12.388924 20608 consensus_queue.cc:260] T afbf4e4aa6ce4ab6ad0d06feaa65e7f1 P fed186bc78f24f0a88da24a2a86c9979 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "107ca9a428674dc68bc806bd62136d2d" member_type: VOTER last_known_addr { host: "127.1.135.193" port: 35231 } } peers { permanent_uuid: "fed186bc78f24f0a88da24a2a86c9979" member_type: VOTER last_known_addr { host: "127.1.135.195" port: 37103 } } peers { permanent_uuid: "347d81c7769b4d60a0b554924c203e9b" member_type: VOTER last_known_addr { host: "127.1.135.194" port: 34643 } }
I20260923 22:16:12.389286 20608 ts_tablet_manager.cc:1428] T afbf4e4aa6ce4ab6ad0d06feaa65e7f1 P fed186bc78f24f0a88da24a2a86c9979: Time spent starting tablet: real 0.004s user 0.001s sys 0.002s
I20260923 22:16:12.389339 20594 heartbeater.cc:503] Master 127.1.135.254:46585 was elected leader, sending a full tablet report...
I20260923 22:16:12.389839 20610 raft_consensus.cc:348] T afbf4e4aa6ce4ab6ad0d06feaa65e7f1 P 347d81c7769b4d60a0b554924c203e9b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "107ca9a428674dc68bc806bd62136d2d" member_type: VOTER last_known_addr { host: "127.1.135.193" port: 35231 } } peers { permanent_uuid: "fed186bc78f24f0a88da24a2a86c9979" member_type: VOTER last_known_addr { host: "127.1.135.195" port: 37103 } } peers { permanent_uuid: "347d81c7769b4d60a0b554924c203e9b" member_type: VOTER last_known_addr { host: "127.1.135.194" port: 34643 } }
I20260923 22:16:12.389976 20610 raft_consensus.cc:374] T afbf4e4aa6ce4ab6ad0d06feaa65e7f1 P 347d81c7769b4d60a0b554924c203e9b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260923 22:16:12.390023 20610 raft_consensus.cc:729] T afbf4e4aa6ce4ab6ad0d06feaa65e7f1 P 347d81c7769b4d60a0b554924c203e9b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 347d81c7769b4d60a0b554924c203e9b, State: Initialized, Role: FOLLOWER
I20260923 22:16:12.390175 20610 consensus_queue.cc:260] T afbf4e4aa6ce4ab6ad0d06feaa65e7f1 P 347d81c7769b4d60a0b554924c203e9b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "107ca9a428674dc68bc806bd62136d2d" member_type: VOTER last_known_addr { host: "127.1.135.193" port: 35231 } } peers { permanent_uuid: "fed186bc78f24f0a88da24a2a86c9979" member_type: VOTER last_known_addr { host: "127.1.135.195" port: 37103 } } peers { permanent_uuid: "347d81c7769b4d60a0b554924c203e9b" member_type: VOTER last_known_addr { host: "127.1.135.194" port: 34643 } }
I20260923 22:16:12.390476 20610 ts_tablet_manager.cc:1428] T afbf4e4aa6ce4ab6ad0d06feaa65e7f1 P 347d81c7769b4d60a0b554924c203e9b: Time spent starting tablet: real 0.003s user 0.003s sys 0.000s
I20260923 22:16:12.390571 20463 heartbeater.cc:503] Master 127.1.135.254:46585 was elected leader, sending a full tablet report...
I20260923 22:16:12.390450 20611 raft_consensus.cc:348] T afbf4e4aa6ce4ab6ad0d06feaa65e7f1 P 107ca9a428674dc68bc806bd62136d2d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "107ca9a428674dc68bc806bd62136d2d" member_type: VOTER last_known_addr { host: "127.1.135.193" port: 35231 } } peers { permanent_uuid: "fed186bc78f24f0a88da24a2a86c9979" member_type: VOTER last_known_addr { host: "127.1.135.195" port: 37103 } } peers { permanent_uuid: "347d81c7769b4d60a0b554924c203e9b" member_type: VOTER last_known_addr { host: "127.1.135.194" port: 34643 } }
I20260923 22:16:12.390642 20611 raft_consensus.cc:374] T afbf4e4aa6ce4ab6ad0d06feaa65e7f1 P 107ca9a428674dc68bc806bd62136d2d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260923 22:16:12.390687 20611 raft_consensus.cc:729] T afbf4e4aa6ce4ab6ad0d06feaa65e7f1 P 107ca9a428674dc68bc806bd62136d2d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 107ca9a428674dc68bc806bd62136d2d, State: Initialized, Role: FOLLOWER
I20260923 22:16:12.390810 20611 consensus_queue.cc:260] T afbf4e4aa6ce4ab6ad0d06feaa65e7f1 P 107ca9a428674dc68bc806bd62136d2d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "107ca9a428674dc68bc806bd62136d2d" member_type: VOTER last_known_addr { host: "127.1.135.193" port: 35231 } } peers { permanent_uuid: "fed186bc78f24f0a88da24a2a86c9979" member_type: VOTER last_known_addr { host: "127.1.135.195" port: 37103 } } peers { permanent_uuid: "347d81c7769b4d60a0b554924c203e9b" member_type: VOTER last_known_addr { host: "127.1.135.194" port: 34643 } }
I20260923 22:16:12.391067 20611 ts_tablet_manager.cc:1428] T afbf4e4aa6ce4ab6ad0d06feaa65e7f1 P 107ca9a428674dc68bc806bd62136d2d: Time spent starting tablet: real 0.003s user 0.000s sys 0.003s
I20260923 22:16:12.391108 20332 heartbeater.cc:503] Master 127.1.135.254:46585 was elected leader, sending a full tablet report...
W20260923 22:16:12.481369 20464 tablet.cc:2369] T afbf4e4aa6ce4ab6ad0d06feaa65e7f1 P 347d81c7769b4d60a0b554924c203e9b: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260923 22:16:12.540977 20615 raft_consensus.cc:482] T afbf4e4aa6ce4ab6ad0d06feaa65e7f1 P 347d81c7769b4d60a0b554924c203e9b [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260923 22:16:12.541164 20615 raft_consensus.cc:504] T afbf4e4aa6ce4ab6ad0d06feaa65e7f1 P 347d81c7769b4d60a0b554924c203e9b [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "107ca9a428674dc68bc806bd62136d2d" member_type: VOTER last_known_addr { host: "127.1.135.193" port: 35231 } } peers { permanent_uuid: "fed186bc78f24f0a88da24a2a86c9979" member_type: VOTER last_known_addr { host: "127.1.135.195" port: 37103 } } peers { permanent_uuid: "347d81c7769b4d60a0b554924c203e9b" member_type: VOTER last_known_addr { host: "127.1.135.194" port: 34643 } }
I20260923 22:16:12.541605 20615 leader_election.cc:288] T afbf4e4aa6ce4ab6ad0d06feaa65e7f1 P 347d81c7769b4d60a0b554924c203e9b [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 107ca9a428674dc68bc806bd62136d2d (127.1.135.193:35231), fed186bc78f24f0a88da24a2a86c9979 (127.1.135.195:37103)
I20260923 22:16:12.542528 20614 raft_consensus.cc:482] T afbf4e4aa6ce4ab6ad0d06feaa65e7f1 P fed186bc78f24f0a88da24a2a86c9979 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260923 22:16:12.542668 20614 raft_consensus.cc:504] T afbf4e4aa6ce4ab6ad0d06feaa65e7f1 P fed186bc78f24f0a88da24a2a86c9979 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "107ca9a428674dc68bc806bd62136d2d" member_type: VOTER last_known_addr { host: "127.1.135.193" port: 35231 } } peers { permanent_uuid: "fed186bc78f24f0a88da24a2a86c9979" member_type: VOTER last_known_addr { host: "127.1.135.195" port: 37103 } } peers { permanent_uuid: "347d81c7769b4d60a0b554924c203e9b" member_type: VOTER last_known_addr { host: "127.1.135.194" port: 34643 } }
I20260923 22:16:12.543015 20614 leader_election.cc:288] T afbf4e4aa6ce4ab6ad0d06feaa65e7f1 P fed186bc78f24f0a88da24a2a86c9979 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 107ca9a428674dc68bc806bd62136d2d (127.1.135.193:35231), 347d81c7769b4d60a0b554924c203e9b (127.1.135.194:34643)
I20260923 22:16:12.545460 20604 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 22:16:12.541915 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.135.194:58899 (local address 127.1.135.193:35231)
0923 22:16:12.541963 (+ 48us) server_negotiation.cc:206] Beginning negotiation
0923 22:16:12.541965 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0923 22:16:12.541976 (+ 11us) server_negotiation.cc:407] Connection header received
0923 22:16:12.542061 (+ 85us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 22:16:12.542063 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 22:16:12.542076 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 22:16:12.542170 (+ 94us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0923 22:16:12.542363 (+ 193us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 22:16:12.543150 (+ 787us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 22:16:12.544668 (+ 1518us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 22:16:12.545084 (+ 416us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 22:16:12.545174 (+ 90us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 22:16:12.545184 (+ 10us) server_negotiation.cc:1035] Waiting for connection context
0923 22:16:12.545358 (+ 174us) server_negotiation.cc:299] Negotiation successful
0923 22:16:12.545398 (+ 40us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":25}
I20260923 22:16:12.545460 20465 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 22:16:12.541816 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.135.193:35231 (local address 127.1.135.194:58899)
0923 22:16:12.541891 (+ 75us) negotiation.cc:107] Waiting for socket to connect
0923 22:16:12.541907 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0923 22:16:12.541981 (+ 74us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0923 22:16:12.542175 (+ 194us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0923 22:16:12.542176 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0923 22:16:12.542177 (+ 1us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0923 22:16:12.542281 (+ 104us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 22:16:12.542283 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 22:16:12.543244 (+ 961us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 22:16:12.543245 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 22:16:12.544577 (+ 1332us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 22:16:12.544580 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 22:16:12.545182 (+ 602us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 22:16:12.545183 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 22:16:12.545250 (+ 67us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 22:16:12.545253 (+ 3us) client_negotiation.cc:769] Sending connection context
0923 22:16:12.545340 (+ 87us) client_negotiation.cc:240] Negotiation successful
0923 22:16:12.545386 (+ 46us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":47}
I20260923 22:16:12.545581 20607 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 22:16:12.542042 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.135.194:42165 (local address 127.1.135.195:37103)
0923 22:16:12.542095 (+ 53us) server_negotiation.cc:206] Beginning negotiation
0923 22:16:12.542097 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0923 22:16:12.542185 (+ 88us) server_negotiation.cc:407] Connection header received
0923 22:16:12.542291 (+ 106us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 22:16:12.542293 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 22:16:12.542301 (+ 8us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 22:16:12.542384 (+ 83us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0923 22:16:12.542544 (+ 160us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 22:16:12.543243 (+ 699us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 22:16:12.544700 (+ 1457us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 22:16:12.545210 (+ 510us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 22:16:12.545257 (+ 47us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 22:16:12.545264 (+ 7us) server_negotiation.cc:1035] Waiting for connection context
0923 22:16:12.545482 (+ 218us) server_negotiation.cc:299] Negotiation successful
0923 22:16:12.545519 (+ 37us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":32}
I20260923 22:16:12.545768 20617 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 22:16:12.541945 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.135.195:37103 (local address 127.1.135.194:42165)
0923 22:16:12.542114 (+ 169us) negotiation.cc:107] Waiting for socket to connect
0923 22:16:12.542122 (+ 8us) client_negotiation.cc:174] Beginning negotiation
0923 22:16:12.542195 (+ 73us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0923 22:16:12.542386 (+ 191us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0923 22:16:12.542403 (+ 17us) client_negotiation.cc:394] Received NEGOTIATE response from server
0923 22:16:12.542405 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0923 22:16:12.542477 (+ 72us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 22:16:12.542479 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 22:16:12.543344 (+ 865us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 22:16:12.543345 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 22:16:12.544609 (+ 1264us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 22:16:12.544611 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 22:16:12.545334 (+ 723us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 22:16:12.545335 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 22:16:12.545408 (+ 73us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 22:16:12.545410 (+ 2us) client_negotiation.cc:769] Sending connection context
0923 22:16:12.545656 (+ 246us) client_negotiation.cc:240] Negotiation successful
0923 22:16:12.545695 (+ 39us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":154,"thread_start_us":102,"threads_started":1}
I20260923 22:16:12.546028 20286 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "afbf4e4aa6ce4ab6ad0d06feaa65e7f1" candidate_uuid: "347d81c7769b4d60a0b554924c203e9b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "107ca9a428674dc68bc806bd62136d2d" is_pre_election: true
I20260923 22:16:12.546198 20286 raft_consensus.cc:2454] T afbf4e4aa6ce4ab6ad0d06feaa65e7f1 P 107ca9a428674dc68bc806bd62136d2d [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 347d81c7769b4d60a0b554924c203e9b in term 0.
I20260923 22:16:12.546423 20548 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "afbf4e4aa6ce4ab6ad0d06feaa65e7f1" candidate_uuid: "347d81c7769b4d60a0b554924c203e9b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "fed186bc78f24f0a88da24a2a86c9979" is_pre_election: true
I20260923 22:16:12.546458 20350 leader_election.cc:302] T afbf4e4aa6ce4ab6ad0d06feaa65e7f1 P 347d81c7769b4d60a0b554924c203e9b [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 107ca9a428674dc68bc806bd62136d2d, 347d81c7769b4d60a0b554924c203e9b; no voters:
I20260923 22:16:12.546525 20548 raft_consensus.cc:2454] T afbf4e4aa6ce4ab6ad0d06feaa65e7f1 P fed186bc78f24f0a88da24a2a86c9979 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 347d81c7769b4d60a0b554924c203e9b in term 0.
I20260923 22:16:12.546604 20615 raft_consensus.cc:2781] T afbf4e4aa6ce4ab6ad0d06feaa65e7f1 P 347d81c7769b4d60a0b554924c203e9b [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260923 22:16:12.546674 20615 raft_consensus.cc:482] T afbf4e4aa6ce4ab6ad0d06feaa65e7f1 P 347d81c7769b4d60a0b554924c203e9b [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260923 22:16:12.546717 20615 raft_consensus.cc:3037] T afbf4e4aa6ce4ab6ad0d06feaa65e7f1 P 347d81c7769b4d60a0b554924c203e9b [term 0 FOLLOWER]: Advancing to term 1
I20260923 22:16:12.546878 20596 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 22:16:12.543121 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.135.193:35231 (local address 127.1.135.195:60389)
0923 22:16:12.543168 (+ 47us) negotiation.cc:107] Waiting for socket to connect
0923 22:16:12.543177 (+ 9us) client_negotiation.cc:174] Beginning negotiation
0923 22:16:12.543214 (+ 37us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0923 22:16:12.543769 (+ 555us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0923 22:16:12.543770 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0923 22:16:12.543771 (+ 1us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0923 22:16:12.543842 (+ 71us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 22:16:12.543844 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 22:16:12.544803 (+ 959us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 22:16:12.544804 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 22:16:12.545899 (+ 1095us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 22:16:12.545902 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 22:16:12.546670 (+ 768us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 22:16:12.546671 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 22:16:12.546736 (+ 65us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 22:16:12.546739 (+ 3us) client_negotiation.cc:769] Sending connection context
0923 22:16:12.546765 (+ 26us) client_negotiation.cc:240] Negotiation successful
0923 22:16:12.546800 (+ 35us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":34}
I20260923 22:16:12.547120 20618 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 22:16:12.543347 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.135.195:60389 (local address 127.1.135.193:35231)
0923 22:16:12.543643 (+ 296us) server_negotiation.cc:206] Beginning negotiation
0923 22:16:12.543645 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0923 22:16:12.543652 (+ 7us) server_negotiation.cc:407] Connection header received
0923 22:16:12.543673 (+ 21us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 22:16:12.543673 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 22:16:12.543681 (+ 8us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 22:16:12.543900 (+ 219us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0923 22:16:12.543932 (+ 32us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 22:16:12.544709 (+ 777us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 22:16:12.545994 (+ 1285us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 22:16:12.546572 (+ 578us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 22:16:12.546989 (+ 417us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 22:16:12.546997 (+ 8us) server_negotiation.cc:1035] Waiting for connection context
0923 22:16:12.547014 (+ 17us) server_negotiation.cc:299] Negotiation successful
0923 22:16:12.547048 (+ 34us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":28,"server-negotiator.queue_time_us":274,"thread_start_us":80,"threads_started":1}
I20260923 22:16:12.547375 20286 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "afbf4e4aa6ce4ab6ad0d06feaa65e7f1" candidate_uuid: "fed186bc78f24f0a88da24a2a86c9979" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "107ca9a428674dc68bc806bd62136d2d" is_pre_election: true
I20260923 22:16:12.547459 20286 raft_consensus.cc:2454] T afbf4e4aa6ce4ab6ad0d06feaa65e7f1 P 107ca9a428674dc68bc806bd62136d2d [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate fed186bc78f24f0a88da24a2a86c9979 in term 0.
I20260923 22:16:12.547640 20481 leader_election.cc:302] T afbf4e4aa6ce4ab6ad0d06feaa65e7f1 P fed186bc78f24f0a88da24a2a86c9979 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 107ca9a428674dc68bc806bd62136d2d, fed186bc78f24f0a88da24a2a86c9979; no voters:
I20260923 22:16:12.547761 20614 raft_consensus.cc:2781] T afbf4e4aa6ce4ab6ad0d06feaa65e7f1 P fed186bc78f24f0a88da24a2a86c9979 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260923 22:16:12.547816 20614 raft_consensus.cc:482] T afbf4e4aa6ce4ab6ad0d06feaa65e7f1 P fed186bc78f24f0a88da24a2a86c9979 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260923 22:16:12.547847 20605 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 22:16:12.543522 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.135.195:59503 (local address 127.1.135.194:34643)
0923 22:16:12.543556 (+ 34us) server_negotiation.cc:206] Beginning negotiation
0923 22:16:12.543558 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0923 22:16:12.543564 (+ 6us) server_negotiation.cc:407] Connection header received
0923 22:16:12.543584 (+ 20us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 22:16:12.543585 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 22:16:12.543593 (+ 8us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 22:16:12.543645 (+ 52us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0923 22:16:12.543837 (+ 192us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 22:16:12.544550 (+ 713us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 22:16:12.546994 (+ 2444us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 22:16:12.547441 (+ 447us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 22:16:12.547516 (+ 75us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 22:16:12.547524 (+ 8us) server_negotiation.cc:1035] Waiting for connection context
0923 22:16:12.547731 (+ 207us) server_negotiation.cc:299] Negotiation successful
0923 22:16:12.547775 (+ 44us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":18}
I20260923 22:16:12.547858 20614 raft_consensus.cc:3037] T afbf4e4aa6ce4ab6ad0d06feaa65e7f1 P fed186bc78f24f0a88da24a2a86c9979 [term 0 FOLLOWER]: Advancing to term 1
I20260923 22:16:12.547940 20619 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 22:16:12.543353 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.135.194:34643 (local address 127.1.135.195:59503)
0923 22:16:12.543489 (+ 136us) negotiation.cc:107] Waiting for socket to connect
0923 22:16:12.543499 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0923 22:16:12.543543 (+ 44us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0923 22:16:12.543681 (+ 138us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0923 22:16:12.543682 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0923 22:16:12.543684 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0923 22:16:12.543749 (+ 65us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 22:16:12.543750 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 22:16:12.544642 (+ 892us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 22:16:12.544643 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 22:16:12.546899 (+ 2256us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 22:16:12.546901 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 22:16:12.547582 (+ 681us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 22:16:12.547583 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 22:16:12.547657 (+ 74us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 22:16:12.547659 (+ 2us) client_negotiation.cc:769] Sending connection context
0923 22:16:12.547724 (+ 65us) client_negotiation.cc:240] Negotiation successful
0923 22:16:12.547760 (+ 36us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":120,"mutex_wait_us":1089,"thread_start_us":83,"threads_started":1}
I20260923 22:16:12.547976 20615 raft_consensus.cc:504] T afbf4e4aa6ce4ab6ad0d06feaa65e7f1 P 347d81c7769b4d60a0b554924c203e9b [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "107ca9a428674dc68bc806bd62136d2d" member_type: VOTER last_known_addr { host: "127.1.135.193" port: 35231 } } peers { permanent_uuid: "fed186bc78f24f0a88da24a2a86c9979" member_type: VOTER last_known_addr { host: "127.1.135.195" port: 37103 } } peers { permanent_uuid: "347d81c7769b4d60a0b554924c203e9b" member_type: VOTER last_known_addr { host: "127.1.135.194" port: 34643 } }
I20260923 22:16:12.548115 20615 leader_election.cc:288] T afbf4e4aa6ce4ab6ad0d06feaa65e7f1 P 347d81c7769b4d60a0b554924c203e9b [CANDIDATE]: Term 1 election: Requested vote from peers 107ca9a428674dc68bc806bd62136d2d (127.1.135.193:35231), fed186bc78f24f0a88da24a2a86c9979 (127.1.135.195:37103)
I20260923 22:16:12.548357 20417 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "afbf4e4aa6ce4ab6ad0d06feaa65e7f1" candidate_uuid: "fed186bc78f24f0a88da24a2a86c9979" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "347d81c7769b4d60a0b554924c203e9b" is_pre_election: true
I20260923 22:16:12.548470 20417 raft_consensus.cc:2379] T afbf4e4aa6ce4ab6ad0d06feaa65e7f1 P 347d81c7769b4d60a0b554924c203e9b [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate fed186bc78f24f0a88da24a2a86c9979 in current term 1: Already voted for candidate 347d81c7769b4d60a0b554924c203e9b in this term.
I20260923 22:16:12.548753 20614 raft_consensus.cc:504] T afbf4e4aa6ce4ab6ad0d06feaa65e7f1 P fed186bc78f24f0a88da24a2a86c9979 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "107ca9a428674dc68bc806bd62136d2d" member_type: VOTER last_known_addr { host: "127.1.135.193" port: 35231 } } peers { permanent_uuid: "fed186bc78f24f0a88da24a2a86c9979" member_type: VOTER last_known_addr { host: "127.1.135.195" port: 37103 } } peers { permanent_uuid: "347d81c7769b4d60a0b554924c203e9b" member_type: VOTER last_known_addr { host: "127.1.135.194" port: 34643 } }
I20260923 22:16:12.548893 20614 leader_election.cc:288] T afbf4e4aa6ce4ab6ad0d06feaa65e7f1 P fed186bc78f24f0a88da24a2a86c9979 [CANDIDATE]: Term 1 election: Requested vote from peers 107ca9a428674dc68bc806bd62136d2d (127.1.135.193:35231), 347d81c7769b4d60a0b554924c203e9b (127.1.135.194:34643)
I20260923 22:16:12.549059 20417 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "afbf4e4aa6ce4ab6ad0d06feaa65e7f1" candidate_uuid: "fed186bc78f24f0a88da24a2a86c9979" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "347d81c7769b4d60a0b554924c203e9b"
I20260923 22:16:12.549060 20286 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "afbf4e4aa6ce4ab6ad0d06feaa65e7f1" candidate_uuid: "fed186bc78f24f0a88da24a2a86c9979" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "107ca9a428674dc68bc806bd62136d2d"
I20260923 22:16:12.549141 20417 raft_consensus.cc:2379] T afbf4e4aa6ce4ab6ad0d06feaa65e7f1 P 347d81c7769b4d60a0b554924c203e9b [term 1 FOLLOWER]: Leader election vote request: Denying vote to candidate fed186bc78f24f0a88da24a2a86c9979 in current term 1: Already voted for candidate 347d81c7769b4d60a0b554924c203e9b in this term.
I20260923 22:16:12.549074 20285 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "afbf4e4aa6ce4ab6ad0d06feaa65e7f1" candidate_uuid: "347d81c7769b4d60a0b554924c203e9b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "107ca9a428674dc68bc806bd62136d2d"
I20260923 22:16:12.549182 20285 raft_consensus.cc:3037] T afbf4e4aa6ce4ab6ad0d06feaa65e7f1 P 107ca9a428674dc68bc806bd62136d2d [term 0 FOLLOWER]: Advancing to term 1
I20260923 22:16:12.549347 20548 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "afbf4e4aa6ce4ab6ad0d06feaa65e7f1" candidate_uuid: "347d81c7769b4d60a0b554924c203e9b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "fed186bc78f24f0a88da24a2a86c9979"
I20260923 22:16:12.549345 20481 leader_election.cc:302] T afbf4e4aa6ce4ab6ad0d06feaa65e7f1 P fed186bc78f24f0a88da24a2a86c9979 [CANDIDATE]: Term 1 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: fed186bc78f24f0a88da24a2a86c9979; no voters: 107ca9a428674dc68bc806bd62136d2d, 347d81c7769b4d60a0b554924c203e9b
I20260923 22:16:12.549476 20548 raft_consensus.cc:2379] T afbf4e4aa6ce4ab6ad0d06feaa65e7f1 P fed186bc78f24f0a88da24a2a86c9979 [term 1 FOLLOWER]: Leader election vote request: Denying vote to candidate 347d81c7769b4d60a0b554924c203e9b in current term 1: Already voted for candidate fed186bc78f24f0a88da24a2a86c9979 in this term.
I20260923 22:16:12.549605 20614 raft_consensus.cc:2726] T afbf4e4aa6ce4ab6ad0d06feaa65e7f1 P fed186bc78f24f0a88da24a2a86c9979 [term 1 FOLLOWER]: Leader election lost for term 1. Reason: could not achieve majority
I20260923 22:16:12.550020 20285 raft_consensus.cc:2454] T afbf4e4aa6ce4ab6ad0d06feaa65e7f1 P 107ca9a428674dc68bc806bd62136d2d [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 347d81c7769b4d60a0b554924c203e9b in term 1.
I20260923 22:16:12.550211 20350 leader_election.cc:302] T afbf4e4aa6ce4ab6ad0d06feaa65e7f1 P 347d81c7769b4d60a0b554924c203e9b [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 107ca9a428674dc68bc806bd62136d2d, 347d81c7769b4d60a0b554924c203e9b; no voters: fed186bc78f24f0a88da24a2a86c9979
I20260923 22:16:12.550335 20615 raft_consensus.cc:2781] T afbf4e4aa6ce4ab6ad0d06feaa65e7f1 P 347d81c7769b4d60a0b554924c203e9b [term 1 FOLLOWER]: Leader election won for term 1
I20260923 22:16:12.550489 20615 raft_consensus.cc:686] T afbf4e4aa6ce4ab6ad0d06feaa65e7f1 P 347d81c7769b4d60a0b554924c203e9b [term 1 LEADER]: Becoming Leader. State: Replica: 347d81c7769b4d60a0b554924c203e9b, State: Running, Role: LEADER
I20260923 22:16:12.550583 20615 consensus_queue.cc:237] T afbf4e4aa6ce4ab6ad0d06feaa65e7f1 P 347d81c7769b4d60a0b554924c203e9b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "107ca9a428674dc68bc806bd62136d2d" member_type: VOTER last_known_addr { host: "127.1.135.193" port: 35231 } } peers { permanent_uuid: "fed186bc78f24f0a88da24a2a86c9979" member_type: VOTER last_known_addr { host: "127.1.135.195" port: 37103 } } peers { permanent_uuid: "347d81c7769b4d60a0b554924c203e9b" member_type: VOTER last_known_addr { host: "127.1.135.194" port: 34643 } }
I20260923 22:16:12.551435 20144 catalog_manager.cc:5495] T afbf4e4aa6ce4ab6ad0d06feaa65e7f1 P 347d81c7769b4d60a0b554924c203e9b reported cstate change: term changed from 0 to 1, leader changed from <none> to 347d81c7769b4d60a0b554924c203e9b (127.1.135.194). New cstate: current_term: 1 leader_uuid: "347d81c7769b4d60a0b554924c203e9b" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "107ca9a428674dc68bc806bd62136d2d" member_type: VOTER last_known_addr { host: "127.1.135.193" port: 35231 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "fed186bc78f24f0a88da24a2a86c9979" member_type: VOTER last_known_addr { host: "127.1.135.195" port: 37103 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "347d81c7769b4d60a0b554924c203e9b" member_type: VOTER last_known_addr { host: "127.1.135.194" port: 34643 } health_report { overall_health: HEALTHY } } }
I20260923 22:16:12.562186 20605 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 22:16:12.559609 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37418 (local address 127.1.135.194:34643)
0923 22:16:12.559667 (+ 58us) server_negotiation.cc:206] Beginning negotiation
0923 22:16:12.559669 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0923 22:16:12.559679 (+ 10us) server_negotiation.cc:407] Connection header received
0923 22:16:12.559722 (+ 43us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 22:16:12.559723 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 22:16:12.559743 (+ 20us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 22:16:12.559851 (+ 108us) server_negotiation.cc:226] Negotiated authn=TOKEN
0923 22:16:12.559998 (+ 147us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 22:16:12.560547 (+ 549us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 22:16:12.561383 (+ 836us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 22:16:12.561586 (+ 203us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 22:16:12.561662 (+ 76us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 22:16:12.561799 (+ 137us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0923 22:16:12.561908 (+ 109us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0923 22:16:12.562016 (+ 108us) server_negotiation.cc:1035] Waiting for connection context
0923 22:16:12.562069 (+ 53us) server_negotiation.cc:299] Negotiation successful
0923 22:16:12.562080 (+ 11us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":34}
I20260923 22:16:12.564258 20285 raft_consensus.cc:1261] T afbf4e4aa6ce4ab6ad0d06feaa65e7f1 P 107ca9a428674dc68bc806bd62136d2d [term 1 FOLLOWER]: Refusing update from remote peer 347d81c7769b4d60a0b554924c203e9b: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260923 22:16:12.564258 20548 raft_consensus.cc:1261] T afbf4e4aa6ce4ab6ad0d06feaa65e7f1 P fed186bc78f24f0a88da24a2a86c9979 [term 1 FOLLOWER]: Refusing update from remote peer 347d81c7769b4d60a0b554924c203e9b: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260923 22:16:12.564558 20615 consensus_queue.cc:1035] T afbf4e4aa6ce4ab6ad0d06feaa65e7f1 P 347d81c7769b4d60a0b554924c203e9b [LEADER]: Connected to new peer: Peer: permanent_uuid: "fed186bc78f24f0a88da24a2a86c9979" member_type: VOTER last_known_addr { host: "127.1.135.195" port: 37103 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260923 22:16:12.564742 20620 consensus_queue.cc:1035] T afbf4e4aa6ce4ab6ad0d06feaa65e7f1 P 347d81c7769b4d60a0b554924c203e9b [LEADER]: Connected to new peer: Peer: permanent_uuid: "107ca9a428674dc68bc806bd62136d2d" member_type: VOTER last_known_addr { host: "127.1.135.193" port: 35231 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260923 22:16:12.568156 20624 mvcc.cc:205] Tried to move back new op lower bound from 7332666460419678208 to 7332666460367790080. Current Snapshot: MvccSnapshot[applied={T|T < 7332666460419678208}]
I20260923 22:16:12.570034 20143 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:37614:
table { table_name: "test-table" } modify_external_catalogs: true
I20260923 22:16:12.570142 20143 catalog_manager.cc:2645] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:37614:
table { table_name: "test-table" } modify_external_catalogs: true
I20260923 22:16:12.571002 20143 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 141301c78aa44ae1bf7a3b8ba5e1c9ed: Sending DeleteTablet for 3 replicas of tablet afbf4e4aa6ce4ab6ad0d06feaa65e7f1
I20260923 22:16:12.571426 20397 tablet_service.cc:1514] Processing DeleteTablet for tablet afbf4e4aa6ce4ab6ad0d06feaa65e7f1 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-23 22:16:12 UTC) from {username='slave'} at 127.0.0.1:37404
I20260923 22:16:12.571445 20528 tablet_service.cc:1514] Processing DeleteTablet for tablet afbf4e4aa6ce4ab6ad0d06feaa65e7f1 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-23 22:16:12 UTC) from {username='slave'} at 127.0.0.1:59494
I20260923 22:16:12.571688 20633 tablet_replica.cc:330] T afbf4e4aa6ce4ab6ad0d06feaa65e7f1 P 347d81c7769b4d60a0b554924c203e9b: stopping tablet replica
I20260923 22:16:12.571700 20266 tablet_service.cc:1514] Processing DeleteTablet for tablet afbf4e4aa6ce4ab6ad0d06feaa65e7f1 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-23 22:16:12 UTC) from {username='slave'} at 127.0.0.1:40244
I20260923 22:16:12.571784 20633 raft_consensus.cc:2229] T afbf4e4aa6ce4ab6ad0d06feaa65e7f1 P 347d81c7769b4d60a0b554924c203e9b [term 1 LEADER]: Raft consensus shutting down.
I20260923 22:16:12.571841 20634 tablet_replica.cc:330] T afbf4e4aa6ce4ab6ad0d06feaa65e7f1 P fed186bc78f24f0a88da24a2a86c9979: stopping tablet replica
I20260923 22:16:12.571906 20633 raft_consensus.cc:2258] T afbf4e4aa6ce4ab6ad0d06feaa65e7f1 P 347d81c7769b4d60a0b554924c203e9b [term 1 FOLLOWER]: Raft consensus is shut down!
I20260923 22:16:12.571943 20634 raft_consensus.cc:2229] T afbf4e4aa6ce4ab6ad0d06feaa65e7f1 P fed186bc78f24f0a88da24a2a86c9979 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260923 22:16:12.572047 20634 raft_consensus.cc:2258] T afbf4e4aa6ce4ab6ad0d06feaa65e7f1 P fed186bc78f24f0a88da24a2a86c9979 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260923 22:16:12.572111 1567 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskRiQe_a/build/release/bin/kudu with pid 20204
I20260923 22:16:12.572417 20633 ts_tablet_manager.cc:1905] T afbf4e4aa6ce4ab6ad0d06feaa65e7f1 P 347d81c7769b4d60a0b554924c203e9b: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260923 22:16:12.574340 20633 ts_tablet_manager.cc:1918] T afbf4e4aa6ce4ab6ad0d06feaa65e7f1 P 347d81c7769b4d60a0b554924c203e9b: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260923 22:16:12.574432 20633 log.cc:1196] T afbf4e4aa6ce4ab6ad0d06feaa65e7f1 P 347d81c7769b4d60a0b554924c203e9b: Deleting WAL directory at /tmp/dist-test-taskRiQe_a/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790201539513534-1567-0/minicluster-data/ts-1/wal/wals/afbf4e4aa6ce4ab6ad0d06feaa65e7f1
I20260923 22:16:12.574677 20633 ts_tablet_manager.cc:1939] T afbf4e4aa6ce4ab6ad0d06feaa65e7f1 P 347d81c7769b4d60a0b554924c203e9b: Deleting consensus metadata
I20260923 22:16:12.575078 20130 catalog_manager.cc:4841] TS 347d81c7769b4d60a0b554924c203e9b (127.1.135.194:34643): tablet afbf4e4aa6ce4ab6ad0d06feaa65e7f1 (table test-table [id=c09836df255846f9b9b97127f4257906]) successfully deleted
I20260923 22:16:12.575515 20634 ts_tablet_manager.cc:1905] T afbf4e4aa6ce4ab6ad0d06feaa65e7f1 P fed186bc78f24f0a88da24a2a86c9979: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260923 22:16:12.577193 20634 ts_tablet_manager.cc:1918] T afbf4e4aa6ce4ab6ad0d06feaa65e7f1 P fed186bc78f24f0a88da24a2a86c9979: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260923 22:16:12.577271 20634 log.cc:1196] T afbf4e4aa6ce4ab6ad0d06feaa65e7f1 P fed186bc78f24f0a88da24a2a86c9979: Deleting WAL directory at /tmp/dist-test-taskRiQe_a/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790201539513534-1567-0/minicluster-data/ts-2/wal/wals/afbf4e4aa6ce4ab6ad0d06feaa65e7f1
I20260923 22:16:12.577530 20634 ts_tablet_manager.cc:1939] T afbf4e4aa6ce4ab6ad0d06feaa65e7f1 P fed186bc78f24f0a88da24a2a86c9979: Deleting consensus metadata
I20260923 22:16:12.578006 20128 catalog_manager.cc:4841] TS fed186bc78f24f0a88da24a2a86c9979 (127.1.135.195:37103): tablet afbf4e4aa6ce4ab6ad0d06feaa65e7f1 (table test-table [id=c09836df255846f9b9b97127f4257906]) successfully deleted
W20260923 22:16:12.579357 20128 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.1.135.193:35231 (error 108)
I20260923 22:16:12.579874 20606 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 22:16:12.579722 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.135.193:35231 (local address 127.0.0.1:40256)
0923 22:16:12.579787 (+ 65us) negotiation.cc:107] Waiting for socket to connect
0923 22:16:12.579841 (+ 54us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.135.193:35231: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":44}
I20260923 22:16:12.579932 1567 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskRiQe_a/build/release/bin/kudu with pid 20335
W20260923 22:16:12.579994 20128 catalog_manager.cc:4584] TS 107ca9a428674dc68bc806bd62136d2d (127.1.135.193:35231): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet afbf4e4aa6ce4ab6ad0d06feaa65e7f1: Network error: Client connection negotiation failed: client connection to 127.1.135.193:35231: connect: Connection refused (error 111)
I20260923 22:16:12.584684 1567 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskRiQe_a/build/release/bin/kudu with pid 20466
I20260923 22:16:12.589038 1567 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskRiQe_a/build/release/bin/kudu with pid 20113
2026-09-23T22:16:12Z chronyd exiting
[ OK ] EncryptionPolicyTest.TestEncryptionPolicy/3 (798 ms)
[ RUN ] EncryptionPolicyTest.TestEncryptionPolicy/4
Loading random data
Initializing database '/tmp/dist-test-taskRiQe_a/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790201539513534-1567-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Sep 23 22:16:12 dist-test-slave-s2fj krb5kdc[20639](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Sep 23 22:16:12 dist-test-slave-s2fj krb5kdc[20639](info): set up 2 sockets
Sep 23 22:16:12 dist-test-slave-s2fj krb5kdc[20639](info): commencing operation
krb5kdc: starting...
W20260923 22:16:13.615904 1567 mini_kdc.cc:121] Time spent starting KDC: real 1.007s user 0.003s sys 0.002s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskRiQe_a/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790201539513534-1567-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskRiQe_a/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790201539513534-1567-0/krb5kdc/test-user.keytab.
Sep 23 22:16:13 dist-test-slave-s2fj krb5kdc[20639](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790201773, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-09-23T22:16:13Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-09-23T22:16:13Z Disabled control of system clock
WARNING: no policy specified for kudu/127.1.135.254@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.1.135.254@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.1.135.254 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskRiQe_a/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790201539513534-1567-0/krb5kdc/kudu_127.1.135.254.keytab.
Entry for principal kudu/127.1.135.254 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskRiQe_a/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790201539513534-1567-0/krb5kdc/kudu_127.1.135.254.keytab.
I20260923 22:16:13.726421 1567 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskRiQe_a/build/release/bin/kudu
/tmp/dist-test-taskRiQe_a/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskRiQe_a/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790201539513534-1567-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskRiQe_a/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790201539513534-1567-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskRiQe_a/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790201539513534-1567-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskRiQe_a/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790201539513534-1567-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.1.135.254:38733
--webserver_interface=127.1.135.254
--webserver_port=0
--keytab_file=/tmp/dist-test-taskRiQe_a/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790201539513534-1567-0/krb5kdc/kudu_127.1.135.254.keytab
--principal=kudu/127.1.135.254
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.1.135.212:34457
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.1.135.254:38733
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskRiQe_a/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790201539513534-1567-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskRiQe_a/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790201539513534-1567-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskRiQe_a/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790201539513534-1567-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260923 22:16:13.802982 20652 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260923 22:16:13.803165 20652 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260923 22:16:13.803187 20652 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260923 22:16:13.804474 20652 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260923 22:16:13.804512 20652 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260923 22:16:13.804527 20652 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260923 22:16:13.804575 20652 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260923 22:16:13.804602 20652 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260923 22:16:13.806022 20652 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.1.135.212:34457
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskRiQe_a/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790201539513534-1567-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskRiQe_a/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790201539513534-1567-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.1.135.254:38733
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskRiQe_a/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790201539513534-1567-0/krb5kdc/kudu_127.1.135.254.keytab
--principal=kudu/127.1.135.254
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.1.135.254:38733
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskRiQe_a/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790201539513534-1567-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.1.135.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.20652
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskRiQe_a/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790201539513534-1567-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 23 Sep 2026 20:53:37 UTC on ccycloud.quasar-rkviyx.root.comops.site
build id 4668083
I20260923 22:16:13.806257 20652 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260923 22:16:13.806475 20652 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260923 22:16:13.811302 20660 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260923 22:16:13.811347 20658 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260923 22:16:13.811352 20657 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260923 22:16:13.811383 20652 server_base.cc:1034] running on GCE node
I20260923 22:16:13.811764 20652 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260923 22:16:13.812036 20652 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260923 22:16:13.813212 20652 hybrid_clock.cc:648] HybridClock initialized: now 1790201773813192 us; error 35 us; skew 500 ppm
Sep 23 22:16:13 dist-test-slave-s2fj krb5kdc[20639](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790201773, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.135.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260923 22:16:13.815470 20652 init.cc:377] Logged in from keytab as kudu/127.1.135.254@KRBTEST.COM (short username kudu)
I20260923 22:16:13.815891 20652 webserver.cc:460] Webserver started at http://127.1.135.254:41325/ using document root <none> and password file <none>
I20260923 22:16:13.816154 20652 fs_manager.cc:359] Metadata directory not provided
I20260923 22:16:13.816217 20652 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260923 22:16:13.816375 20652 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260923 22:16:13.817406 20652 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskRiQe_a/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790201539513534-1567-0/minicluster-data/master-0/data/instance:
uuid: "ca9eb5d833034e81bb733303438ed92b"
format_stamp: "Formatted at 2026-09-23 22:16:13 on dist-test-slave-s2fj"
server_key: "59d6595df080d8a3d090a2d149860517"
server_key_iv: "8be499a87f074ed2d2402750930c63f9"
server_key_version: "encryptionkey@0"
I20260923 22:16:13.817736 20652 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskRiQe_a/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790201539513534-1567-0/minicluster-data/master-0/wal/instance:
uuid: "ca9eb5d833034e81bb733303438ed92b"
format_stamp: "Formatted at 2026-09-23 22:16:13 on dist-test-slave-s2fj"
server_key: "59d6595df080d8a3d090a2d149860517"
server_key_iv: "8be499a87f074ed2d2402750930c63f9"
server_key_version: "encryptionkey@0"
I20260923 22:16:13.818938 20652 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260923 22:16:13.819624 20667 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260923 22:16:13.819784 20652 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260923 22:16:13.819900 20652 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskRiQe_a/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790201539513534-1567-0/minicluster-data/master-0/data,/tmp/dist-test-taskRiQe_a/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790201539513534-1567-0/minicluster-data/master-0/wal
uuid: "ca9eb5d833034e81bb733303438ed92b"
format_stamp: "Formatted at 2026-09-23 22:16:13 on dist-test-slave-s2fj"
server_key: "59d6595df080d8a3d090a2d149860517"
server_key_iv: "8be499a87f074ed2d2402750930c63f9"
server_key_version: "encryptionkey@0"
I20260923 22:16:13.819953 20652 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskRiQe_a/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790201539513534-1567-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskRiQe_a/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790201539513534-1567-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskRiQe_a/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790201539513534-1567-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260923 22:16:13.843408 20652 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260923 22:16:13.845089 20652 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260923 22:16:13.845189 20652 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260923 22:16:13.849061 20652 rpc_server.cc:307] RPC server started. Bound to: 127.1.135.254:38733
I20260923 22:16:13.849093 20719 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.1.135.254:38733 every 8 connection(s)
I20260923 22:16:13.849418 20652 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskRiQe_a/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790201539513534-1567-0/minicluster-data/master-0/data/info.pb
I20260923 22:16:13.850193 20720 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260923 22:16:13.850324 1567 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskRiQe_a/build/release/bin/kudu as pid 20652
I20260923 22:16:13.850462 1567 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskRiQe_a/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790201539513534-1567-0/minicluster-data/master-0/wal/instance
I20260923 22:16:13.850672 1567 external_mini_cluster.cc:1428] Setting key 73fc7377daaaf289faba88fb63ac2f3d
I20260923 22:16:13.853416 20720 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P ca9eb5d833034e81bb733303438ed92b: Bootstrap starting.
I20260923 22:16:13.854025 20720 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P ca9eb5d833034e81bb733303438ed92b: Neither blocks nor log segments found. Creating new log.
I20260923 22:16:13.854287 20720 log.cc:824] T 00000000000000000000000000000000 P ca9eb5d833034e81bb733303438ed92b: Log is configured to *not* fsync() on all Append() calls
I20260923 22:16:13.854920 20720 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P ca9eb5d833034e81bb733303438ed92b: No bootstrap required, opened a new log
I20260923 22:16:13.856884 20720 raft_consensus.cc:348] T 00000000000000000000000000000000 P ca9eb5d833034e81bb733303438ed92b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ca9eb5d833034e81bb733303438ed92b" member_type: VOTER last_known_addr { host: "127.1.135.254" port: 38733 } }
I20260923 22:16:13.857034 20720 raft_consensus.cc:374] T 00000000000000000000000000000000 P ca9eb5d833034e81bb733303438ed92b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260923 22:16:13.857065 20720 raft_consensus.cc:729] T 00000000000000000000000000000000 P ca9eb5d833034e81bb733303438ed92b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: ca9eb5d833034e81bb733303438ed92b, State: Initialized, Role: FOLLOWER
Sep 23 22:16:13 dist-test-slave-s2fj krb5kdc[20639](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790201773, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.1.135.254@KRBTEST.COM
I20260923 22:16:13.857203 20720 consensus_queue.cc:260] T 00000000000000000000000000000000 P ca9eb5d833034e81bb733303438ed92b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ca9eb5d833034e81bb733303438ed92b" member_type: VOTER last_known_addr { host: "127.1.135.254" port: 38733 } }
I20260923 22:16:13.857291 20720 raft_consensus.cc:388] T 00000000000000000000000000000000 P ca9eb5d833034e81bb733303438ed92b [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260923 22:16:13.857338 20720 raft_consensus.cc:482] T 00000000000000000000000000000000 P ca9eb5d833034e81bb733303438ed92b [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260923 22:16:13.857401 20720 raft_consensus.cc:3037] T 00000000000000000000000000000000 P ca9eb5d833034e81bb733303438ed92b [term 0 FOLLOWER]: Advancing to term 1
I20260923 22:16:13.857901 20720 raft_consensus.cc:504] T 00000000000000000000000000000000 P ca9eb5d833034e81bb733303438ed92b [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ca9eb5d833034e81bb733303438ed92b" member_type: VOTER last_known_addr { host: "127.1.135.254" port: 38733 } }
I20260923 22:16:13.858003 20720 leader_election.cc:302] T 00000000000000000000000000000000 P ca9eb5d833034e81bb733303438ed92b [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: ca9eb5d833034e81bb733303438ed92b; no voters:
I20260923 22:16:13.858168 20720 leader_election.cc:288] T 00000000000000000000000000000000 P ca9eb5d833034e81bb733303438ed92b [CANDIDATE]: Term 1 election: Requested vote from peers
I20260923 22:16:13.858196 20725 raft_consensus.cc:2781] T 00000000000000000000000000000000 P ca9eb5d833034e81bb733303438ed92b [term 1 FOLLOWER]: Leader election won for term 1
I20260923 22:16:13.858294 20725 raft_consensus.cc:686] T 00000000000000000000000000000000 P ca9eb5d833034e81bb733303438ed92b [term 1 LEADER]: Becoming Leader. State: Replica: ca9eb5d833034e81bb733303438ed92b, State: Running, Role: LEADER
I20260923 22:16:13.858448 20720 sys_catalog.cc:564] T 00000000000000000000000000000000 P ca9eb5d833034e81bb733303438ed92b [sys.catalog]: configured and running, proceeding with master startup.
I20260923 22:16:13.858690 20725 consensus_queue.cc:237] T 00000000000000000000000000000000 P ca9eb5d833034e81bb733303438ed92b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ca9eb5d833034e81bb733303438ed92b" member_type: VOTER last_known_addr { host: "127.1.135.254" port: 38733 } }
I20260923 22:16:13.859123 20726 sys_catalog.cc:455] T 00000000000000000000000000000000 P ca9eb5d833034e81bb733303438ed92b [sys.catalog]: SysCatalogTable state changed. Reason: New leader ca9eb5d833034e81bb733303438ed92b. Latest consensus state: current_term: 1 leader_uuid: "ca9eb5d833034e81bb733303438ed92b" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ca9eb5d833034e81bb733303438ed92b" member_type: VOTER last_known_addr { host: "127.1.135.254" port: 38733 } } }
I20260923 22:16:13.859211 20726 sys_catalog.cc:458] T 00000000000000000000000000000000 P ca9eb5d833034e81bb733303438ed92b [sys.catalog]: This master's current role is: LEADER
I20260923 22:16:13.859429 20727 sys_catalog.cc:455] T 00000000000000000000000000000000 P ca9eb5d833034e81bb733303438ed92b [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "ca9eb5d833034e81bb733303438ed92b" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ca9eb5d833034e81bb733303438ed92b" member_type: VOTER last_known_addr { host: "127.1.135.254" port: 38733 } } }
I20260923 22:16:13.859675 20734 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260923 22:16:13.860374 20734 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260923 22:16:13.859776 20727 sys_catalog.cc:458] T 00000000000000000000000000000000 P ca9eb5d833034e81bb733303438ed92b [sys.catalog]: This master's current role is: LEADER
I20260923 22:16:13.861145 20723 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 22:16:13.851152 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37020 (local address 127.1.135.254:38733)
0923 22:16:13.851337 (+ 185us) server_negotiation.cc:206] Beginning negotiation
0923 22:16:13.851346 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0923 22:16:13.851375 (+ 29us) server_negotiation.cc:407] Connection header received
0923 22:16:13.851817 (+ 442us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 22:16:13.851824 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 22:16:13.851882 (+ 58us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 22:16:13.852027 (+ 145us) server_negotiation.cc:226] Negotiated authn=SASL
0923 22:16:13.852643 (+ 616us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 22:16:13.853931 (+ 1288us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 22:16:13.854727 (+ 796us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 22:16:13.855156 (+ 429us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 22:16:13.855286 (+ 130us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 22:16:13.857632 (+ 2346us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0923 22:16:13.857635 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0923 22:16:13.857636 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0923 22:16:13.857646 (+ 10us) server_negotiation.cc:922] calling sasl_server_start()
0923 22:16:13.859522 (+ 1876us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0923 22:16:13.859942 (+ 420us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0923 22:16:13.859945 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0923 22:16:13.859946 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0923 22:16:13.860041 (+ 95us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0923 22:16:13.860288 (+ 247us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0923 22:16:13.860288 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0923 22:16:13.860289 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0923 22:16:13.860546 (+ 257us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0923 22:16:13.860660 (+ 114us) server_negotiation.cc:1035] Waiting for connection context
0923 22:16:13.860819 (+ 159us) server_negotiation.cc:299] Negotiation successful
0923 22:16:13.860916 (+ 97us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":138,"thread_start_us":75,"threads_started":1}
I20260923 22:16:13.862103 20734 catalog_manager.cc:1347] Generated new cluster ID: a1b51f261f474a94ae07ca55b35b3b36
I20260923 22:16:13.862157 20734 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260923 22:16:13.874382 20734 catalog_manager.cc:1370] Generated new certificate authority record
I20260923 22:16:13.874866 20734 catalog_manager.cc:1504] Loading token signing keys...
I20260923 22:16:13.879830 20734 catalog_manager.cc:5868] T 00000000000000000000000000000000 P ca9eb5d833034e81bb733303438ed92b: Generated new TSK 0
I20260923 22:16:13.879988 20734 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.1.135.193@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.1.135.193@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.1.135.193 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskRiQe_a/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790201539513534-1567-0/krb5kdc/kudu_127.1.135.193.keytab.
Entry for principal kudu/127.1.135.193 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskRiQe_a/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790201539513534-1567-0/krb5kdc/kudu_127.1.135.193.keytab.
I20260923 22:16:13.908265 1567 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskRiQe_a/build/release/bin/kudu
/tmp/dist-test-taskRiQe_a/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskRiQe_a/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790201539513534-1567-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskRiQe_a/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790201539513534-1567-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskRiQe_a/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790201539513534-1567-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskRiQe_a/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790201539513534-1567-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.1.135.193:0
--local_ip_for_outbound_sockets=127.1.135.193
--webserver_interface=127.1.135.193
--webserver_port=0
--tserver_master_addrs=127.1.135.254:38733
--keytab_file=/tmp/dist-test-taskRiQe_a/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790201539513534-1567-0/krb5kdc/kudu_127.1.135.193.keytab
--principal=kudu/127.1.135.193
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.1.135.212:34457
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskRiQe_a/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790201539513534-1567-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskRiQe_a/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790201539513534-1567-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskRiQe_a/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790201539513534-1567-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260923 22:16:13.989858 20746 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260923 22:16:13.990079 20746 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260923 22:16:13.990111 20746 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260923 22:16:13.991235 20746 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260923 22:16:13.991272 20746 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260923 22:16:13.991298 20746 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.1.135.193
I20260923 22:16:13.992726 20746 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.1.135.212:34457
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskRiQe_a/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790201539513534-1567-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskRiQe_a/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790201539513534-1567-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskRiQe_a/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790201539513534-1567-0/krb5kdc/kudu_127.1.135.193.keytab
--principal=kudu/127.1.135.193
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.1.135.193:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskRiQe_a/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790201539513534-1567-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.1.135.193
--webserver_port=0
--tserver_master_addrs=127.1.135.254:38733
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.20746
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.1.135.193
--log_dir=/tmp/dist-test-taskRiQe_a/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790201539513534-1567-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 23 Sep 2026 20:53:37 UTC on ccycloud.quasar-rkviyx.root.comops.site
build id 4668083
I20260923 22:16:13.992980 20746 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260923 22:16:13.993167 20746 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260923 22:16:13.998147 20752 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260923 22:16:13.998162 20751 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260923 22:16:13.998248 20746 server_base.cc:1034] running on GCE node
W20260923 22:16:13.998140 20754 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260923 22:16:13.998558 20746 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260923 22:16:13.998845 20746 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260923 22:16:13.999987 20746 hybrid_clock.cc:648] HybridClock initialized: now 1790201773999967 us; error 38 us; skew 500 ppm
Sep 23 22:16:13 dist-test-slave-s2fj krb5kdc[20639](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790201773, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.135.193@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260923 22:16:14.002624 20746 init.cc:377] Logged in from keytab as kudu/127.1.135.193@KRBTEST.COM (short username kudu)
I20260923 22:16:14.003022 20746 webserver.cc:460] Webserver started at http://127.1.135.193:43297/ using document root <none> and password file <none>
I20260923 22:16:14.003216 20746 fs_manager.cc:359] Metadata directory not provided
I20260923 22:16:14.003255 20746 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260923 22:16:14.003350 20746 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260923 22:16:14.004473 20746 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskRiQe_a/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790201539513534-1567-0/minicluster-data/ts-0/data/instance:
uuid: "217a705b595a4a60be43f06a9f4a0479"
format_stamp: "Formatted at 2026-09-23 22:16:13 on dist-test-slave-s2fj"
server_key: "6e2ed2d71abe9c1584dc332cec2f536a"
server_key_iv: "bf4807b5f36854cb0b4399a2336176e8"
server_key_version: "encryptionkey@0"
I20260923 22:16:14.004770 20746 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskRiQe_a/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790201539513534-1567-0/minicluster-data/ts-0/wal/instance:
uuid: "217a705b595a4a60be43f06a9f4a0479"
format_stamp: "Formatted at 2026-09-23 22:16:13 on dist-test-slave-s2fj"
server_key: "6e2ed2d71abe9c1584dc332cec2f536a"
server_key_iv: "bf4807b5f36854cb0b4399a2336176e8"
server_key_version: "encryptionkey@0"
I20260923 22:16:14.006054 20746 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.000s
I20260923 22:16:14.006749 20761 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260923 22:16:14.006920 20746 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260923 22:16:14.006999 20746 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskRiQe_a/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790201539513534-1567-0/minicluster-data/ts-0/data,/tmp/dist-test-taskRiQe_a/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790201539513534-1567-0/minicluster-data/ts-0/wal
uuid: "217a705b595a4a60be43f06a9f4a0479"
format_stamp: "Formatted at 2026-09-23 22:16:13 on dist-test-slave-s2fj"
server_key: "6e2ed2d71abe9c1584dc332cec2f536a"
server_key_iv: "bf4807b5f36854cb0b4399a2336176e8"
server_key_version: "encryptionkey@0"
I20260923 22:16:14.007054 20746 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskRiQe_a/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790201539513534-1567-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskRiQe_a/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790201539513534-1567-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskRiQe_a/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790201539513534-1567-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260923 22:16:14.024382 20746 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260923 22:16:14.026387 20746 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260923 22:16:14.026501 20746 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260923 22:16:14.026688 20746 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260923 22:16:14.027035 20746 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260923 22:16:14.027076 20746 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260923 22:16:14.027098 20746 ts_tablet_manager.cc:610] Registered 0 tablets
I20260923 22:16:14.027143 20746 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260923 22:16:14.032904 20746 rpc_server.cc:307] RPC server started. Bound to: 127.1.135.193:33649
I20260923 22:16:14.032991 20874 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.1.135.193:33649 every 8 connection(s)
I20260923 22:16:14.033310 20746 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskRiQe_a/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790201539513534-1567-0/minicluster-data/ts-0/data/info.pb
I20260923 22:16:14.033509 1567 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskRiQe_a/build/release/bin/kudu as pid 20746
I20260923 22:16:14.033619 1567 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskRiQe_a/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790201539513534-1567-0/minicluster-data/ts-0/wal/instance
I20260923 22:16:14.033834 1567 external_mini_cluster.cc:1428] Setting key 4404f8fd3094b63faef61906c6057940
Sep 23 22:16:14 dist-test-slave-s2fj krb5kdc[20639](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790201773, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.135.193@KRBTEST.COM for kudu/127.1.135.254@KRBTEST.COM
WARNING: no policy specified for kudu/127.1.135.194@KRBTEST.COM; defaulting to no policy
I20260923 22:16:14.044301 20723 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 22:16:14.034056 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.135.193:37183 (local address 127.1.135.254:38733)
0923 22:16:14.034103 (+ 47us) server_negotiation.cc:206] Beginning negotiation
0923 22:16:14.034105 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0923 22:16:14.034449 (+ 344us) server_negotiation.cc:407] Connection header received
0923 22:16:14.035184 (+ 735us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 22:16:14.035185 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 22:16:14.035194 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 22:16:14.035239 (+ 45us) server_negotiation.cc:226] Negotiated authn=SASL
0923 22:16:14.036546 (+ 1307us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 22:16:14.037370 (+ 824us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 22:16:14.038390 (+ 1020us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 22:16:14.038752 (+ 362us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 22:16:14.038831 (+ 79us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 22:16:14.041245 (+ 2414us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0923 22:16:14.041247 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0923 22:16:14.041247 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0923 22:16:14.041256 (+ 9us) server_negotiation.cc:922] calling sasl_server_start()
0923 22:16:14.042948 (+ 1692us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0923 22:16:14.043324 (+ 376us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0923 22:16:14.043325 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0923 22:16:14.043325 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0923 22:16:14.043378 (+ 53us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0923 22:16:14.043594 (+ 216us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0923 22:16:14.043594 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0923 22:16:14.043595 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0923 22:16:14.043696 (+ 101us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0923 22:16:14.043764 (+ 68us) server_negotiation.cc:1035] Waiting for connection context
0923 22:16:14.044137 (+ 373us) server_negotiation.cc:299] Negotiation successful
0923 22:16:14.044230 (+ 93us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":21}
I20260923 22:16:14.044505 20877 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 22:16:14.034074 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.135.254:38733 (local address 127.1.135.193:37183)
0923 22:16:14.034323 (+ 249us) negotiation.cc:107] Waiting for socket to connect
0923 22:16:14.034370 (+ 47us) client_negotiation.cc:174] Beginning negotiation
0923 22:16:14.035070 (+ 700us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0923 22:16:14.035326 (+ 256us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0923 22:16:14.035330 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0923 22:16:14.036176 (+ 846us) client_negotiation.cc:189] Negotiated authn=SASL
0923 22:16:14.036447 (+ 271us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 22:16:14.036453 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 22:16:14.037464 (+ 1011us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 22:16:14.037465 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 22:16:14.038274 (+ 809us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 22:16:14.038276 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 22:16:14.038842 (+ 566us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 22:16:14.038843 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 22:16:14.038954 (+ 111us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 22:16:14.039544 (+ 590us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0923 22:16:14.039552 (+ 8us) client_negotiation.cc:656] calling sasl_client_start()
0923 22:16:14.041147 (+ 1595us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0923 22:16:14.043025 (+ 1878us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0923 22:16:14.043031 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0923 22:16:14.043033 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0923 22:16:14.043253 (+ 220us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0923 22:16:14.043441 (+ 188us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0923 22:16:14.043442 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0923 22:16:14.043442 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0923 22:16:14.043539 (+ 97us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0923 22:16:14.043767 (+ 228us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0923 22:16:14.043771 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0923 22:16:14.043957 (+ 186us) client_negotiation.cc:769] Sending connection context
0923 22:16:14.044114 (+ 157us) client_negotiation.cc:240] Negotiation successful
0923 22:16:14.044253 (+ 139us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":148,"thread_start_us":92,"threads_started":1}
I20260923 22:16:14.045013 20875 heartbeater.cc:347] Connected to a master server at 127.1.135.254:38733
I20260923 22:16:14.045118 20875 heartbeater.cc:464] Registering TS with master...
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.1.135.194@KRBTEST.COM" created.
I20260923 22:16:14.045331 20875 heartbeater.cc:511] Master 127.1.135.254:38733 requested a full tablet report, sending...
I20260923 22:16:14.045925 20684 ts_manager.cc:194] Registered new tserver with Master: 217a705b595a4a60be43f06a9f4a0479 (127.1.135.193:33649)
I20260923 22:16:14.046617 20684 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.1.135.193@KRBTEST.COM'} at 127.1.135.193:37183
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.1.135.194 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskRiQe_a/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790201539513534-1567-0/krb5kdc/kudu_127.1.135.194.keytab.
Entry for principal kudu/127.1.135.194 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskRiQe_a/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790201539513534-1567-0/krb5kdc/kudu_127.1.135.194.keytab.
I20260923 22:16:14.062252 1567 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskRiQe_a/build/release/bin/kudu
/tmp/dist-test-taskRiQe_a/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskRiQe_a/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790201539513534-1567-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskRiQe_a/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790201539513534-1567-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskRiQe_a/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790201539513534-1567-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskRiQe_a/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790201539513534-1567-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.1.135.194:0
--local_ip_for_outbound_sockets=127.1.135.194
--webserver_interface=127.1.135.194
--webserver_port=0
--tserver_master_addrs=127.1.135.254:38733
--keytab_file=/tmp/dist-test-taskRiQe_a/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790201539513534-1567-0/krb5kdc/kudu_127.1.135.194.keytab
--principal=kudu/127.1.135.194
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.1.135.212:34457
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskRiQe_a/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790201539513534-1567-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskRiQe_a/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790201539513534-1567-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskRiQe_a/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790201539513534-1567-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260923 22:16:14.141991 20880 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260923 22:16:14.142299 20880 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260923 22:16:14.142352 20880 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260923 22:16:14.143560 20880 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260923 22:16:14.143641 20880 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260923 22:16:14.143702 20880 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.1.135.194
I20260923 22:16:14.145229 20880 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.1.135.212:34457
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskRiQe_a/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790201539513534-1567-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskRiQe_a/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790201539513534-1567-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskRiQe_a/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790201539513534-1567-0/krb5kdc/kudu_127.1.135.194.keytab
--principal=kudu/127.1.135.194
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.1.135.194:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskRiQe_a/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790201539513534-1567-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.1.135.194
--webserver_port=0
--tserver_master_addrs=127.1.135.254:38733
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.20880
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.1.135.194
--log_dir=/tmp/dist-test-taskRiQe_a/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790201539513534-1567-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 23 Sep 2026 20:53:37 UTC on ccycloud.quasar-rkviyx.root.comops.site
build id 4668083
I20260923 22:16:14.145537 20880 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260923 22:16:14.145777 20880 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260923 22:16:14.150782 20885 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260923 22:16:14.150823 20886 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260923 22:16:14.150923 20880 server_base.cc:1034] running on GCE node
W20260923 22:16:14.150816 20888 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260923 22:16:14.151248 20880 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260923 22:16:14.151500 20880 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260923 22:16:14.152640 20880 hybrid_clock.cc:648] HybridClock initialized: now 1790201774152625 us; error 36 us; skew 500 ppm
Sep 23 22:16:14 dist-test-slave-s2fj krb5kdc[20639](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790201774, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.135.194@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260923 22:16:14.155143 20880 init.cc:377] Logged in from keytab as kudu/127.1.135.194@KRBTEST.COM (short username kudu)
I20260923 22:16:14.155627 20880 webserver.cc:460] Webserver started at http://127.1.135.194:37337/ using document root <none> and password file <none>
I20260923 22:16:14.155855 20880 fs_manager.cc:359] Metadata directory not provided
I20260923 22:16:14.155900 20880 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260923 22:16:14.156101 20880 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260923 22:16:14.157153 20880 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskRiQe_a/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790201539513534-1567-0/minicluster-data/ts-1/data/instance:
uuid: "418f2d5cb24a43df99129b44c0a797dd"
format_stamp: "Formatted at 2026-09-23 22:16:14 on dist-test-slave-s2fj"
server_key: "953fac212f8ec0ec42501b4567aa7f1a"
server_key_iv: "e1bcdff7904907184a14d604f45424fd"
server_key_version: "encryptionkey@0"
I20260923 22:16:14.157518 20880 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskRiQe_a/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790201539513534-1567-0/minicluster-data/ts-1/wal/instance:
uuid: "418f2d5cb24a43df99129b44c0a797dd"
format_stamp: "Formatted at 2026-09-23 22:16:14 on dist-test-slave-s2fj"
server_key: "953fac212f8ec0ec42501b4567aa7f1a"
server_key_iv: "e1bcdff7904907184a14d604f45424fd"
server_key_version: "encryptionkey@0"
I20260923 22:16:14.158928 20880 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260923 22:16:14.159766 20895 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260923 22:16:14.159979 20880 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260923 22:16:14.160095 20880 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskRiQe_a/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790201539513534-1567-0/minicluster-data/ts-1/data,/tmp/dist-test-taskRiQe_a/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790201539513534-1567-0/minicluster-data/ts-1/wal
uuid: "418f2d5cb24a43df99129b44c0a797dd"
format_stamp: "Formatted at 2026-09-23 22:16:14 on dist-test-slave-s2fj"
server_key: "953fac212f8ec0ec42501b4567aa7f1a"
server_key_iv: "e1bcdff7904907184a14d604f45424fd"
server_key_version: "encryptionkey@0"
I20260923 22:16:14.160192 20880 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskRiQe_a/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790201539513534-1567-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskRiQe_a/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790201539513534-1567-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskRiQe_a/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790201539513534-1567-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260923 22:16:14.177611 20880 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260923 22:16:14.179324 20880 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260923 22:16:14.179433 20880 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260923 22:16:14.179605 20880 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260923 22:16:14.179862 20880 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260923 22:16:14.179893 20880 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260923 22:16:14.179913 20880 ts_tablet_manager.cc:610] Registered 0 tablets
I20260923 22:16:14.179926 20880 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260923 22:16:14.185540 20880 rpc_server.cc:307] RPC server started. Bound to: 127.1.135.194:39767
I20260923 22:16:14.185585 21008 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.1.135.194:39767 every 8 connection(s)
I20260923 22:16:14.185873 20880 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskRiQe_a/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790201539513534-1567-0/minicluster-data/ts-1/data/info.pb
I20260923 22:16:14.187670 1567 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskRiQe_a/build/release/bin/kudu as pid 20880
I20260923 22:16:14.187810 1567 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskRiQe_a/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790201539513534-1567-0/minicluster-data/ts-1/wal/instance
I20260923 22:16:14.187994 1567 external_mini_cluster.cc:1428] Setting key bf15860b05a4eac6687a316f4d805530
Sep 23 22:16:14 dist-test-slave-s2fj krb5kdc[20639](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790201774, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.135.194@KRBTEST.COM for kudu/127.1.135.254@KRBTEST.COM
I20260923 22:16:14.195953 20723 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 22:16:14.186528 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.135.194:54773 (local address 127.1.135.254:38733)
0923 22:16:14.186569 (+ 41us) server_negotiation.cc:206] Beginning negotiation
0923 22:16:14.186571 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0923 22:16:14.186766 (+ 195us) server_negotiation.cc:407] Connection header received
0923 22:16:14.187277 (+ 511us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 22:16:14.187278 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 22:16:14.187290 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 22:16:14.187356 (+ 66us) server_negotiation.cc:226] Negotiated authn=SASL
0923 22:16:14.188628 (+ 1272us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 22:16:14.189158 (+ 530us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 22:16:14.190530 (+ 1372us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 22:16:14.190721 (+ 191us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 22:16:14.190803 (+ 82us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 22:16:14.193012 (+ 2209us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0923 22:16:14.193013 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0923 22:16:14.193013 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0923 22:16:14.193021 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0923 22:16:14.194636 (+ 1615us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0923 22:16:14.195008 (+ 372us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0923 22:16:14.195009 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0923 22:16:14.195010 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0923 22:16:14.195070 (+ 60us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0923 22:16:14.195289 (+ 219us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0923 22:16:14.195290 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0923 22:16:14.195291 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0923 22:16:14.195438 (+ 147us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0923 22:16:14.195512 (+ 74us) server_negotiation.cc:1035] Waiting for connection context
0923 22:16:14.195736 (+ 224us) server_negotiation.cc:299] Negotiation successful
0923 22:16:14.195845 (+ 109us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":20}
I20260923 22:16:14.195973 21011 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 22:16:14.186500 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.135.254:38733 (local address 127.1.135.194:54773)
0923 22:16:14.186661 (+ 161us) negotiation.cc:107] Waiting for socket to connect
0923 22:16:14.186691 (+ 30us) client_negotiation.cc:174] Beginning negotiation
0923 22:16:14.187178 (+ 487us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0923 22:16:14.187423 (+ 245us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0923 22:16:14.187426 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0923 22:16:14.188295 (+ 869us) client_negotiation.cc:189] Negotiated authn=SASL
0923 22:16:14.188522 (+ 227us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 22:16:14.188526 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 22:16:14.189262 (+ 736us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 22:16:14.189264 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 22:16:14.190429 (+ 1165us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 22:16:14.190431 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 22:16:14.190819 (+ 388us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 22:16:14.190820 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 22:16:14.190946 (+ 126us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 22:16:14.191370 (+ 424us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0923 22:16:14.191380 (+ 10us) client_negotiation.cc:656] calling sasl_client_start()
0923 22:16:14.192922 (+ 1542us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0923 22:16:14.194722 (+ 1800us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0923 22:16:14.194727 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0923 22:16:14.194728 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0923 22:16:14.194935 (+ 207us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0923 22:16:14.195139 (+ 204us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0923 22:16:14.195139 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0923 22:16:14.195140 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0923 22:16:14.195227 (+ 87us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0923 22:16:14.195504 (+ 277us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0923 22:16:14.195508 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0923 22:16:14.195606 (+ 98us) client_negotiation.cc:769] Sending connection context
0923 22:16:14.195683 (+ 77us) client_negotiation.cc:240] Negotiation successful
0923 22:16:14.195820 (+ 137us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":122,"thread_start_us":76,"threads_started":1}
I20260923 22:16:14.196514 21009 heartbeater.cc:347] Connected to a master server at 127.1.135.254:38733
I20260923 22:16:14.196601 21009 heartbeater.cc:464] Registering TS with master...
I20260923 22:16:14.196769 21009 heartbeater.cc:511] Master 127.1.135.254:38733 requested a full tablet report, sending...
WARNING: no policy specified for kudu/127.1.135.195@KRBTEST.COM; defaulting to no policy
I20260923 22:16:14.197152 20684 ts_manager.cc:194] Registered new tserver with Master: 418f2d5cb24a43df99129b44c0a797dd (127.1.135.194:39767)
I20260923 22:16:14.197512 20684 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.1.135.194@KRBTEST.COM'} at 127.1.135.194:54773
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.1.135.195@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.1.135.195 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskRiQe_a/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790201539513534-1567-0/krb5kdc/kudu_127.1.135.195.keytab.
Entry for principal kudu/127.1.135.195 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskRiQe_a/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790201539513534-1567-0/krb5kdc/kudu_127.1.135.195.keytab.
I20260923 22:16:14.214129 1567 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskRiQe_a/build/release/bin/kudu
/tmp/dist-test-taskRiQe_a/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskRiQe_a/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790201539513534-1567-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskRiQe_a/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790201539513534-1567-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskRiQe_a/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790201539513534-1567-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskRiQe_a/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790201539513534-1567-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.1.135.195:0
--local_ip_for_outbound_sockets=127.1.135.195
--webserver_interface=127.1.135.195
--webserver_port=0
--tserver_master_addrs=127.1.135.254:38733
--keytab_file=/tmp/dist-test-taskRiQe_a/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790201539513534-1567-0/krb5kdc/kudu_127.1.135.195.keytab
--principal=kudu/127.1.135.195
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.1.135.212:34457
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskRiQe_a/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790201539513534-1567-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskRiQe_a/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790201539513534-1567-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskRiQe_a/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790201539513534-1567-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260923 22:16:14.290555 21014 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260923 22:16:14.290740 21014 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260923 22:16:14.290760 21014 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260923 22:16:14.291872 21014 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260923 22:16:14.291913 21014 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260923 22:16:14.291937 21014 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.1.135.195
I20260923 22:16:14.293408 21014 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.1.135.212:34457
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskRiQe_a/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790201539513534-1567-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskRiQe_a/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790201539513534-1567-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskRiQe_a/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790201539513534-1567-0/krb5kdc/kudu_127.1.135.195.keytab
--principal=kudu/127.1.135.195
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.1.135.195:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskRiQe_a/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790201539513534-1567-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.1.135.195
--webserver_port=0
--tserver_master_addrs=127.1.135.254:38733
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.21014
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.1.135.195
--log_dir=/tmp/dist-test-taskRiQe_a/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790201539513534-1567-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 23 Sep 2026 20:53:37 UTC on ccycloud.quasar-rkviyx.root.comops.site
build id 4668083
I20260923 22:16:14.293571 21014 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260923 22:16:14.293790 21014 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260923 22:16:14.298435 21020 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260923 22:16:14.298522 21014 server_base.cc:1034] running on GCE node
W20260923 22:16:14.298436 21022 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260923 22:16:14.298460 21019 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260923 22:16:14.298825 21014 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260923 22:16:14.299054 21014 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260923 22:16:14.300215 21014 hybrid_clock.cc:648] HybridClock initialized: now 1790201774300203 us; error 42 us; skew 500 ppm
Sep 23 22:16:14 dist-test-slave-s2fj krb5kdc[20639](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790201774, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.135.195@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260923 22:16:14.302659 21014 init.cc:377] Logged in from keytab as kudu/127.1.135.195@KRBTEST.COM (short username kudu)
I20260923 22:16:14.303048 21014 webserver.cc:460] Webserver started at http://127.1.135.195:40845/ using document root <none> and password file <none>
I20260923 22:16:14.303270 21014 fs_manager.cc:359] Metadata directory not provided
I20260923 22:16:14.303344 21014 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260923 22:16:14.303498 21014 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260923 22:16:14.304522 21014 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskRiQe_a/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790201539513534-1567-0/minicluster-data/ts-2/data/instance:
uuid: "670b827f9c034305b3e9cb2e335d9761"
format_stamp: "Formatted at 2026-09-23 22:16:14 on dist-test-slave-s2fj"
server_key: "7d45c9bb6550b7568cb8f109f259d618"
server_key_iv: "554641d98cbd81c31e2c7134290dcc00"
server_key_version: "encryptionkey@0"
I20260923 22:16:14.304860 21014 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskRiQe_a/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790201539513534-1567-0/minicluster-data/ts-2/wal/instance:
uuid: "670b827f9c034305b3e9cb2e335d9761"
format_stamp: "Formatted at 2026-09-23 22:16:14 on dist-test-slave-s2fj"
server_key: "7d45c9bb6550b7568cb8f109f259d618"
server_key_iv: "554641d98cbd81c31e2c7134290dcc00"
server_key_version: "encryptionkey@0"
I20260923 22:16:14.306154 21014 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.000s
I20260923 22:16:14.306800 21029 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260923 22:16:14.306959 21014 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260923 22:16:14.307052 21014 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskRiQe_a/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790201539513534-1567-0/minicluster-data/ts-2/data,/tmp/dist-test-taskRiQe_a/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790201539513534-1567-0/minicluster-data/ts-2/wal
uuid: "670b827f9c034305b3e9cb2e335d9761"
format_stamp: "Formatted at 2026-09-23 22:16:14 on dist-test-slave-s2fj"
server_key: "7d45c9bb6550b7568cb8f109f259d618"
server_key_iv: "554641d98cbd81c31e2c7134290dcc00"
server_key_version: "encryptionkey@0"
I20260923 22:16:14.307132 21014 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskRiQe_a/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790201539513534-1567-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskRiQe_a/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790201539513534-1567-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskRiQe_a/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790201539513534-1567-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260923 22:16:14.324723 21014 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260923 22:16:14.326887 21014 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260923 22:16:14.327042 21014 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260923 22:16:14.327245 21014 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260923 22:16:14.327531 21014 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260923 22:16:14.327585 21014 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260923 22:16:14.327625 21014 ts_tablet_manager.cc:610] Registered 0 tablets
I20260923 22:16:14.327652 21014 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260923 22:16:14.333212 21014 rpc_server.cc:307] RPC server started. Bound to: 127.1.135.195:41699
I20260923 22:16:14.333246 21142 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.1.135.195:41699 every 8 connection(s)
I20260923 22:16:14.333532 21014 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskRiQe_a/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790201539513534-1567-0/minicluster-data/ts-2/data/info.pb
I20260923 22:16:14.338249 1567 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskRiQe_a/build/release/bin/kudu as pid 21014
I20260923 22:16:14.338346 1567 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskRiQe_a/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790201539513534-1567-0/minicluster-data/ts-2/wal/instance
I20260923 22:16:14.338557 1567 external_mini_cluster.cc:1428] Setting key 576fe3914f7a9d7ca692db23d873fc32
Sep 23 22:16:14 dist-test-slave-s2fj krb5kdc[20639](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790201774, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.135.195@KRBTEST.COM for kudu/127.1.135.254@KRBTEST.COM
I20260923 22:16:14.342535 20723 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 22:16:14.334189 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.135.195:47481 (local address 127.1.135.254:38733)
0923 22:16:14.334253 (+ 64us) server_negotiation.cc:206] Beginning negotiation
0923 22:16:14.334256 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0923 22:16:14.334472 (+ 216us) server_negotiation.cc:407] Connection header received
0923 22:16:14.334908 (+ 436us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 22:16:14.334909 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 22:16:14.334930 (+ 21us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 22:16:14.334981 (+ 51us) server_negotiation.cc:226] Negotiated authn=SASL
0923 22:16:14.335786 (+ 805us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 22:16:14.336330 (+ 544us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 22:16:14.337230 (+ 900us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 22:16:14.337412 (+ 182us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 22:16:14.337475 (+ 63us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 22:16:14.339720 (+ 2245us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0923 22:16:14.339721 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0923 22:16:14.339721 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0923 22:16:14.339728 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0923 22:16:14.341169 (+ 1441us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0923 22:16:14.341606 (+ 437us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0923 22:16:14.341608 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0923 22:16:14.341608 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0923 22:16:14.341666 (+ 58us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0923 22:16:14.341916 (+ 250us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0923 22:16:14.341917 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0923 22:16:14.341917 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0923 22:16:14.342024 (+ 107us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0923 22:16:14.342099 (+ 75us) server_negotiation.cc:1035] Waiting for connection context
0923 22:16:14.342372 (+ 273us) server_negotiation.cc:299] Negotiation successful
0923 22:16:14.342446 (+ 74us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":37}
I20260923 22:16:14.342725 21145 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 22:16:14.334186 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.135.254:38733 (local address 127.1.135.195:47481)
0923 22:16:14.334362 (+ 176us) negotiation.cc:107] Waiting for socket to connect
0923 22:16:14.334392 (+ 30us) client_negotiation.cc:174] Beginning negotiation
0923 22:16:14.334828 (+ 436us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0923 22:16:14.335040 (+ 212us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0923 22:16:14.335043 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0923 22:16:14.335547 (+ 504us) client_negotiation.cc:189] Negotiated authn=SASL
0923 22:16:14.335712 (+ 165us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 22:16:14.335717 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 22:16:14.336402 (+ 685us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 22:16:14.336403 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 22:16:14.337161 (+ 758us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 22:16:14.337163 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 22:16:14.337495 (+ 332us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 22:16:14.337495 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 22:16:14.337591 (+ 96us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 22:16:14.337976 (+ 385us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0923 22:16:14.337984 (+ 8us) client_negotiation.cc:656] calling sasl_client_start()
0923 22:16:14.339612 (+ 1628us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0923 22:16:14.341266 (+ 1654us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0923 22:16:14.341273 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0923 22:16:14.341275 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0923 22:16:14.341517 (+ 242us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0923 22:16:14.341734 (+ 217us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0923 22:16:14.341735 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0923 22:16:14.341736 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0923 22:16:14.341846 (+ 110us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0923 22:16:14.342092 (+ 246us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0923 22:16:14.342097 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0923 22:16:14.342244 (+ 147us) client_negotiation.cc:769] Sending connection context
0923 22:16:14.342351 (+ 107us) client_negotiation.cc:240] Negotiation successful
0923 22:16:14.342498 (+ 147us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":142,"thread_start_us":81,"threads_started":1}
I20260923 22:16:14.343127 21143 heartbeater.cc:347] Connected to a master server at 127.1.135.254:38733
I20260923 22:16:14.343194 21143 heartbeater.cc:464] Registering TS with master...
I20260923 22:16:14.343318 21143 heartbeater.cc:511] Master 127.1.135.254:38733 requested a full tablet report, sending...
I20260923 22:16:14.343608 20684 ts_manager.cc:194] Registered new tserver with Master: 670b827f9c034305b3e9cb2e335d9761 (127.1.135.195:41699)
I20260923 22:16:14.343914 20684 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.1.135.195@KRBTEST.COM'} at 127.1.135.195:47481
I20260923 22:16:14.350246 1567 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260923 22:16:14.357349 20723 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 22:16:14.351476 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37036 (local address 127.1.135.254:38733)
0923 22:16:14.351511 (+ 35us) server_negotiation.cc:206] Beginning negotiation
0923 22:16:14.351513 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0923 22:16:14.351645 (+ 132us) server_negotiation.cc:407] Connection header received
0923 22:16:14.351719 (+ 74us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 22:16:14.351720 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 22:16:14.351729 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 22:16:14.351797 (+ 68us) server_negotiation.cc:226] Negotiated authn=SASL
0923 22:16:14.352420 (+ 623us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 22:16:14.352877 (+ 457us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 22:16:14.353621 (+ 744us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 22:16:14.353809 (+ 188us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 22:16:14.353892 (+ 83us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 22:16:14.355021 (+ 1129us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0923 22:16:14.355022 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0923 22:16:14.355023 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0923 22:16:14.355029 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0923 22:16:14.356178 (+ 1149us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0923 22:16:14.356526 (+ 348us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0923 22:16:14.356527 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0923 22:16:14.356528 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0923 22:16:14.356586 (+ 58us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0923 22:16:14.356782 (+ 196us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0923 22:16:14.356783 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0923 22:16:14.356783 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0923 22:16:14.356928 (+ 145us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0923 22:16:14.357044 (+ 116us) server_negotiation.cc:1035] Waiting for connection context
0923 22:16:14.357164 (+ 120us) server_negotiation.cc:299] Negotiation successful
0923 22:16:14.357242 (+ 78us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":15}
I20260923 22:16:14.358847 20684 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:37036:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260923 22:16:14.360829 20684 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260923 22:16:14.366950 21152 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 22:16:14.362975 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.135.193:33649 (local address 127.0.0.1:48046)
0923 22:16:14.363275 (+ 300us) negotiation.cc:107] Waiting for socket to connect
0923 22:16:14.363286 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0923 22:16:14.363337 (+ 51us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0923 22:16:14.363591 (+ 254us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0923 22:16:14.363594 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0923 22:16:14.363596 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0923 22:16:14.363681 (+ 85us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 22:16:14.363682 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 22:16:14.364715 (+ 1033us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 22:16:14.364716 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 22:16:14.365742 (+ 1026us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 22:16:14.365744 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 22:16:14.366634 (+ 890us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 22:16:14.366635 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 22:16:14.366716 (+ 81us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 22:16:14.366729 (+ 13us) client_negotiation.cc:769] Sending connection context
0923 22:16:14.366826 (+ 97us) client_negotiation.cc:240] Negotiation successful
0923 22:16:14.366879 (+ 53us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":284,"thread_start_us":56,"threads_started":1}
I20260923 22:16:14.366959 21154 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 22:16:14.363126 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48046 (local address 127.1.135.193:33649)
0923 22:16:14.363442 (+ 316us) server_negotiation.cc:206] Beginning negotiation
0923 22:16:14.363446 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0923 22:16:14.363453 (+ 7us) server_negotiation.cc:407] Connection header received
0923 22:16:14.363485 (+ 32us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 22:16:14.363487 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 22:16:14.363499 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 22:16:14.363587 (+ 88us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0923 22:16:14.363768 (+ 181us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 22:16:14.364610 (+ 842us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 22:16:14.365849 (+ 1239us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 22:16:14.366540 (+ 691us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 22:16:14.366630 (+ 90us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 22:16:14.366690 (+ 60us) server_negotiation.cc:1035] Waiting for connection context
0923 22:16:14.366844 (+ 154us) server_negotiation.cc:299] Negotiation successful
0923 22:16:14.366886 (+ 42us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":290,"thread_start_us":85,"threads_started":1}
I20260923 22:16:14.367225 21155 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 22:16:14.363182 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.135.195:41699 (local address 127.0.0.1:34950)
0923 22:16:14.363293 (+ 111us) negotiation.cc:107] Waiting for socket to connect
0923 22:16:14.363303 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0923 22:16:14.363345 (+ 42us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0923 22:16:14.363774 (+ 429us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0923 22:16:14.363775 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0923 22:16:14.363777 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0923 22:16:14.363877 (+ 100us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 22:16:14.363879 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 22:16:14.365200 (+ 1321us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 22:16:14.365201 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 22:16:14.366102 (+ 901us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 22:16:14.366105 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 22:16:14.366986 (+ 881us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 22:16:14.366987 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 22:16:14.367060 (+ 73us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 22:16:14.367065 (+ 5us) client_negotiation.cc:769] Sending connection context
0923 22:16:14.367120 (+ 55us) client_negotiation.cc:240] Negotiation successful
0923 22:16:14.367157 (+ 37us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":99,"thread_start_us":53,"threads_started":1}
I20260923 22:16:14.367484 21156 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 22:16:14.363371 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34950 (local address 127.1.135.195:41699)
0923 22:16:14.363533 (+ 162us) server_negotiation.cc:206] Beginning negotiation
0923 22:16:14.363537 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0923 22:16:14.363546 (+ 9us) server_negotiation.cc:407] Connection header received
0923 22:16:14.363589 (+ 43us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 22:16:14.363591 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 22:16:14.363602 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 22:16:14.363697 (+ 95us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0923 22:16:14.363955 (+ 258us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 22:16:14.365086 (+ 1131us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 22:16:14.366186 (+ 1100us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 22:16:14.366836 (+ 650us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 22:16:14.366904 (+ 68us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 22:16:14.366951 (+ 47us) server_negotiation.cc:1035] Waiting for connection context
0923 22:16:14.367355 (+ 404us) server_negotiation.cc:299] Negotiation successful
0923 22:16:14.367407 (+ 52us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":133,"thread_start_us":79,"threads_started":1}
I20260923 22:16:14.367838 21151 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 22:16:14.362972 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.135.194:39767 (local address 127.0.0.1:60896)
0923 22:16:14.363134 (+ 162us) negotiation.cc:107] Waiting for socket to connect
0923 22:16:14.363177 (+ 43us) client_negotiation.cc:174] Beginning negotiation
0923 22:16:14.363242 (+ 65us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0923 22:16:14.363951 (+ 709us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0923 22:16:14.363952 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0923 22:16:14.363954 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0923 22:16:14.363997 (+ 43us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 22:16:14.363999 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 22:16:14.365261 (+ 1262us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 22:16:14.365262 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 22:16:14.366334 (+ 1072us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 22:16:14.366337 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 22:16:14.367501 (+ 1164us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 22:16:14.367502 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 22:16:14.367647 (+ 145us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 22:16:14.367650 (+ 3us) client_negotiation.cc:769] Sending connection context
0923 22:16:14.367733 (+ 83us) client_negotiation.cc:240] Negotiation successful
0923 22:16:14.367770 (+ 37us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":139,"thread_start_us":55,"threads_started":1}
I20260923 22:16:14.367781 20809 tablet_service.cc:1467] Processing CreateTablet for tablet c247fea8afd943339528cf33079a1ea4 (DEFAULT_TABLE table=test-table [id=00f1c0012de24709a612065333b278a4]), partition=RANGE (key) PARTITION UNBOUNDED
I20260923 22:16:14.367992 21153 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 22:16:14.363101 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60896 (local address 127.1.135.194:39767)
0923 22:16:14.363719 (+ 618us) server_negotiation.cc:206] Beginning negotiation
0923 22:16:14.363724 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0923 22:16:14.363733 (+ 9us) server_negotiation.cc:407] Connection header received
0923 22:16:14.363771 (+ 38us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 22:16:14.363773 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 22:16:14.363787 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 22:16:14.363872 (+ 85us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0923 22:16:14.364102 (+ 230us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 22:16:14.365156 (+ 1054us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 22:16:14.366421 (+ 1265us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 22:16:14.367351 (+ 930us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 22:16:14.367415 (+ 64us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 22:16:14.367469 (+ 54us) server_negotiation.cc:1035] Waiting for connection context
0923 22:16:14.367754 (+ 285us) server_negotiation.cc:299] Negotiation successful
0923 22:16:14.367794 (+ 40us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":584,"thread_start_us":80,"threads_started":1}
I20260923 22:16:14.368171 20809 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet c247fea8afd943339528cf33079a1ea4. 1 dirs total, 0 dirs full, 0 dirs failed
I20260923 22:16:14.368088 21077 tablet_service.cc:1467] Processing CreateTablet for tablet c247fea8afd943339528cf33079a1ea4 (DEFAULT_TABLE table=test-table [id=00f1c0012de24709a612065333b278a4]), partition=RANGE (key) PARTITION UNBOUNDED
I20260923 22:16:14.368410 21077 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet c247fea8afd943339528cf33079a1ea4. 1 dirs total, 0 dirs full, 0 dirs failed
I20260923 22:16:14.371536 21157 tablet_bootstrap.cc:492] T c247fea8afd943339528cf33079a1ea4 P 670b827f9c034305b3e9cb2e335d9761: Bootstrap starting.
I20260923 22:16:14.372219 21158 tablet_bootstrap.cc:492] T c247fea8afd943339528cf33079a1ea4 P 217a705b595a4a60be43f06a9f4a0479: Bootstrap starting.
I20260923 22:16:14.372265 21157 tablet_bootstrap.cc:654] T c247fea8afd943339528cf33079a1ea4 P 670b827f9c034305b3e9cb2e335d9761: Neither blocks nor log segments found. Creating new log.
I20260923 22:16:14.372318 20943 tablet_service.cc:1467] Processing CreateTablet for tablet c247fea8afd943339528cf33079a1ea4 (DEFAULT_TABLE table=test-table [id=00f1c0012de24709a612065333b278a4]), partition=RANGE (key) PARTITION UNBOUNDED
I20260923 22:16:14.372658 20943 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet c247fea8afd943339528cf33079a1ea4. 1 dirs total, 0 dirs full, 0 dirs failed
I20260923 22:16:14.372701 21158 tablet_bootstrap.cc:654] T c247fea8afd943339528cf33079a1ea4 P 217a705b595a4a60be43f06a9f4a0479: Neither blocks nor log segments found. Creating new log.
I20260923 22:16:14.372848 21157 log.cc:824] T c247fea8afd943339528cf33079a1ea4 P 670b827f9c034305b3e9cb2e335d9761: Log is configured to *not* fsync() on all Append() calls
I20260923 22:16:14.373152 21158 log.cc:824] T c247fea8afd943339528cf33079a1ea4 P 217a705b595a4a60be43f06a9f4a0479: Log is configured to *not* fsync() on all Append() calls
I20260923 22:16:14.373626 21157 tablet_bootstrap.cc:492] T c247fea8afd943339528cf33079a1ea4 P 670b827f9c034305b3e9cb2e335d9761: No bootstrap required, opened a new log
I20260923 22:16:14.373687 21157 ts_tablet_manager.cc:1397] T c247fea8afd943339528cf33079a1ea4 P 670b827f9c034305b3e9cb2e335d9761: Time spent bootstrapping tablet: real 0.002s user 0.002s sys 0.000s
I20260923 22:16:14.373795 21158 tablet_bootstrap.cc:492] T c247fea8afd943339528cf33079a1ea4 P 217a705b595a4a60be43f06a9f4a0479: No bootstrap required, opened a new log
I20260923 22:16:14.373844 21158 ts_tablet_manager.cc:1397] T c247fea8afd943339528cf33079a1ea4 P 217a705b595a4a60be43f06a9f4a0479: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20260923 22:16:14.375270 21158 raft_consensus.cc:348] T c247fea8afd943339528cf33079a1ea4 P 217a705b595a4a60be43f06a9f4a0479 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "217a705b595a4a60be43f06a9f4a0479" member_type: VOTER last_known_addr { host: "127.1.135.193" port: 33649 } } peers { permanent_uuid: "418f2d5cb24a43df99129b44c0a797dd" member_type: VOTER last_known_addr { host: "127.1.135.194" port: 39767 } } peers { permanent_uuid: "670b827f9c034305b3e9cb2e335d9761" member_type: VOTER last_known_addr { host: "127.1.135.195" port: 41699 } }
I20260923 22:16:14.375393 21158 raft_consensus.cc:374] T c247fea8afd943339528cf33079a1ea4 P 217a705b595a4a60be43f06a9f4a0479 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260923 22:16:14.375414 21158 raft_consensus.cc:729] T c247fea8afd943339528cf33079a1ea4 P 217a705b595a4a60be43f06a9f4a0479 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 217a705b595a4a60be43f06a9f4a0479, State: Initialized, Role: FOLLOWER
I20260923 22:16:14.375530 21158 consensus_queue.cc:260] T c247fea8afd943339528cf33079a1ea4 P 217a705b595a4a60be43f06a9f4a0479 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "217a705b595a4a60be43f06a9f4a0479" member_type: VOTER last_known_addr { host: "127.1.135.193" port: 33649 } } peers { permanent_uuid: "418f2d5cb24a43df99129b44c0a797dd" member_type: VOTER last_known_addr { host: "127.1.135.194" port: 39767 } } peers { permanent_uuid: "670b827f9c034305b3e9cb2e335d9761" member_type: VOTER last_known_addr { host: "127.1.135.195" port: 41699 } }
I20260923 22:16:14.375622 21157 raft_consensus.cc:348] T c247fea8afd943339528cf33079a1ea4 P 670b827f9c034305b3e9cb2e335d9761 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "217a705b595a4a60be43f06a9f4a0479" member_type: VOTER last_known_addr { host: "127.1.135.193" port: 33649 } } peers { permanent_uuid: "418f2d5cb24a43df99129b44c0a797dd" member_type: VOTER last_known_addr { host: "127.1.135.194" port: 39767 } } peers { permanent_uuid: "670b827f9c034305b3e9cb2e335d9761" member_type: VOTER last_known_addr { host: "127.1.135.195" port: 41699 } }
I20260923 22:16:14.375788 21157 raft_consensus.cc:374] T c247fea8afd943339528cf33079a1ea4 P 670b827f9c034305b3e9cb2e335d9761 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260923 22:16:14.375787 21158 ts_tablet_manager.cc:1428] T c247fea8afd943339528cf33079a1ea4 P 217a705b595a4a60be43f06a9f4a0479: Time spent starting tablet: real 0.002s user 0.002s sys 0.000s
I20260923 22:16:14.375797 20875 heartbeater.cc:503] Master 127.1.135.254:38733 was elected leader, sending a full tablet report...
I20260923 22:16:14.375833 21157 raft_consensus.cc:729] T c247fea8afd943339528cf33079a1ea4 P 670b827f9c034305b3e9cb2e335d9761 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 670b827f9c034305b3e9cb2e335d9761, State: Initialized, Role: FOLLOWER
I20260923 22:16:14.375962 21157 consensus_queue.cc:260] T c247fea8afd943339528cf33079a1ea4 P 670b827f9c034305b3e9cb2e335d9761 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "217a705b595a4a60be43f06a9f4a0479" member_type: VOTER last_known_addr { host: "127.1.135.193" port: 33649 } } peers { permanent_uuid: "418f2d5cb24a43df99129b44c0a797dd" member_type: VOTER last_known_addr { host: "127.1.135.194" port: 39767 } } peers { permanent_uuid: "670b827f9c034305b3e9cb2e335d9761" member_type: VOTER last_known_addr { host: "127.1.135.195" port: 41699 } }
I20260923 22:16:14.376187 21162 tablet_bootstrap.cc:492] T c247fea8afd943339528cf33079a1ea4 P 418f2d5cb24a43df99129b44c0a797dd: Bootstrap starting.
I20260923 22:16:14.376257 21157 ts_tablet_manager.cc:1428] T c247fea8afd943339528cf33079a1ea4 P 670b827f9c034305b3e9cb2e335d9761: Time spent starting tablet: real 0.003s user 0.003s sys 0.000s
I20260923 22:16:14.376284 21143 heartbeater.cc:503] Master 127.1.135.254:38733 was elected leader, sending a full tablet report...
I20260923 22:16:14.376953 21162 tablet_bootstrap.cc:654] T c247fea8afd943339528cf33079a1ea4 P 418f2d5cb24a43df99129b44c0a797dd: Neither blocks nor log segments found. Creating new log.
I20260923 22:16:14.377434 21162 log.cc:824] T c247fea8afd943339528cf33079a1ea4 P 418f2d5cb24a43df99129b44c0a797dd: Log is configured to *not* fsync() on all Append() calls
I20260923 22:16:14.378173 21162 tablet_bootstrap.cc:492] T c247fea8afd943339528cf33079a1ea4 P 418f2d5cb24a43df99129b44c0a797dd: No bootstrap required, opened a new log
I20260923 22:16:14.378232 21162 ts_tablet_manager.cc:1397] T c247fea8afd943339528cf33079a1ea4 P 418f2d5cb24a43df99129b44c0a797dd: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20260923 22:16:14.379480 21162 raft_consensus.cc:348] T c247fea8afd943339528cf33079a1ea4 P 418f2d5cb24a43df99129b44c0a797dd [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "217a705b595a4a60be43f06a9f4a0479" member_type: VOTER last_known_addr { host: "127.1.135.193" port: 33649 } } peers { permanent_uuid: "418f2d5cb24a43df99129b44c0a797dd" member_type: VOTER last_known_addr { host: "127.1.135.194" port: 39767 } } peers { permanent_uuid: "670b827f9c034305b3e9cb2e335d9761" member_type: VOTER last_known_addr { host: "127.1.135.195" port: 41699 } }
I20260923 22:16:14.379601 21162 raft_consensus.cc:374] T c247fea8afd943339528cf33079a1ea4 P 418f2d5cb24a43df99129b44c0a797dd [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260923 22:16:14.379647 21162 raft_consensus.cc:729] T c247fea8afd943339528cf33079a1ea4 P 418f2d5cb24a43df99129b44c0a797dd [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 418f2d5cb24a43df99129b44c0a797dd, State: Initialized, Role: FOLLOWER
I20260923 22:16:14.379750 21162 consensus_queue.cc:260] T c247fea8afd943339528cf33079a1ea4 P 418f2d5cb24a43df99129b44c0a797dd [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "217a705b595a4a60be43f06a9f4a0479" member_type: VOTER last_known_addr { host: "127.1.135.193" port: 33649 } } peers { permanent_uuid: "418f2d5cb24a43df99129b44c0a797dd" member_type: VOTER last_known_addr { host: "127.1.135.194" port: 39767 } } peers { permanent_uuid: "670b827f9c034305b3e9cb2e335d9761" member_type: VOTER last_known_addr { host: "127.1.135.195" port: 41699 } }
I20260923 22:16:14.379998 21162 ts_tablet_manager.cc:1428] T c247fea8afd943339528cf33079a1ea4 P 418f2d5cb24a43df99129b44c0a797dd: Time spent starting tablet: real 0.002s user 0.002s sys 0.000s
I20260923 22:16:14.380246 21009 heartbeater.cc:503] Master 127.1.135.254:38733 was elected leader, sending a full tablet report...
W20260923 22:16:14.436467 21010 tablet.cc:2369] T c247fea8afd943339528cf33079a1ea4 P 418f2d5cb24a43df99129b44c0a797dd: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260923 22:16:14.534109 20876 tablet.cc:2369] T c247fea8afd943339528cf33079a1ea4 P 217a705b595a4a60be43f06a9f4a0479: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260923 22:16:14.543782 21163 raft_consensus.cc:482] T c247fea8afd943339528cf33079a1ea4 P 670b827f9c034305b3e9cb2e335d9761 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260923 22:16:14.543888 21163 raft_consensus.cc:504] T c247fea8afd943339528cf33079a1ea4 P 670b827f9c034305b3e9cb2e335d9761 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "217a705b595a4a60be43f06a9f4a0479" member_type: VOTER last_known_addr { host: "127.1.135.193" port: 33649 } } peers { permanent_uuid: "418f2d5cb24a43df99129b44c0a797dd" member_type: VOTER last_known_addr { host: "127.1.135.194" port: 39767 } } peers { permanent_uuid: "670b827f9c034305b3e9cb2e335d9761" member_type: VOTER last_known_addr { host: "127.1.135.195" port: 41699 } }
I20260923 22:16:14.544297 21163 leader_election.cc:288] T c247fea8afd943339528cf33079a1ea4 P 670b827f9c034305b3e9cb2e335d9761 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 217a705b595a4a60be43f06a9f4a0479 (127.1.135.193:33649), 418f2d5cb24a43df99129b44c0a797dd (127.1.135.194:39767)
I20260923 22:16:14.548025 21154 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 22:16:14.544513 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.135.195:57193 (local address 127.1.135.193:33649)
0923 22:16:14.544568 (+ 55us) server_negotiation.cc:206] Beginning negotiation
0923 22:16:14.544571 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0923 22:16:14.544582 (+ 11us) server_negotiation.cc:407] Connection header received
0923 22:16:14.544637 (+ 55us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 22:16:14.544638 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 22:16:14.544647 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 22:16:14.544702 (+ 55us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0923 22:16:14.544890 (+ 188us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 22:16:14.545663 (+ 773us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 22:16:14.547097 (+ 1434us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 22:16:14.547638 (+ 541us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 22:16:14.547735 (+ 97us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 22:16:14.547743 (+ 8us) server_negotiation.cc:1035] Waiting for connection context
0923 22:16:14.547921 (+ 178us) server_negotiation.cc:299] Negotiation successful
0923 22:16:14.547958 (+ 37us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":30}
I20260923 22:16:14.548026 21153 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 22:16:14.544586 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.135.195:51407 (local address 127.1.135.194:39767)
0923 22:16:14.544641 (+ 55us) server_negotiation.cc:206] Beginning negotiation
0923 22:16:14.544644 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0923 22:16:14.544761 (+ 117us) server_negotiation.cc:407] Connection header received
0923 22:16:14.544864 (+ 103us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 22:16:14.544865 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 22:16:14.544875 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 22:16:14.544930 (+ 55us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0923 22:16:14.545064 (+ 134us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 22:16:14.545779 (+ 715us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 22:16:14.547099 (+ 1320us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 22:16:14.547649 (+ 550us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 22:16:14.547735 (+ 86us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 22:16:14.547743 (+ 8us) server_negotiation.cc:1035] Waiting for connection context
0923 22:16:14.547922 (+ 179us) server_negotiation.cc:299] Negotiation successful
0923 22:16:14.547961 (+ 39us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":31}
I20260923 22:16:14.548101 21166 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 22:16:14.544485 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.135.194:39767 (local address 127.1.135.195:51407)
0923 22:16:14.544699 (+ 214us) negotiation.cc:107] Waiting for socket to connect
0923 22:16:14.544708 (+ 9us) client_negotiation.cc:174] Beginning negotiation
0923 22:16:14.544765 (+ 57us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0923 22:16:14.544945 (+ 180us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0923 22:16:14.544946 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0923 22:16:14.544947 (+ 1us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0923 22:16:14.544994 (+ 47us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 22:16:14.544996 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 22:16:14.545848 (+ 852us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 22:16:14.545849 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 22:16:14.546990 (+ 1141us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 22:16:14.546992 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 22:16:14.547751 (+ 759us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 22:16:14.547752 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 22:16:14.547836 (+ 84us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 22:16:14.547843 (+ 7us) client_negotiation.cc:769] Sending connection context
0923 22:16:14.547902 (+ 59us) client_negotiation.cc:240] Negotiation successful
0923 22:16:14.547955 (+ 53us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":197,"mutex_wait_us":12,"thread_start_us":117,"threads_started":1}
I20260923 22:16:14.548326 21145 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 22:16:14.544469 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.135.193:33649 (local address 127.1.135.195:57193)
0923 22:16:14.544524 (+ 55us) negotiation.cc:107] Waiting for socket to connect
0923 22:16:14.544534 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0923 22:16:14.544580 (+ 46us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0923 22:16:14.544731 (+ 151us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0923 22:16:14.544732 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0923 22:16:14.544734 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0923 22:16:14.544809 (+ 75us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 22:16:14.544811 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 22:16:14.545748 (+ 937us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 22:16:14.545749 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 22:16:14.546988 (+ 1239us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 22:16:14.546991 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 22:16:14.547750 (+ 759us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 22:16:14.547751 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 22:16:14.547836 (+ 85us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 22:16:14.547842 (+ 6us) client_negotiation.cc:769] Sending connection context
0923 22:16:14.547902 (+ 60us) client_negotiation.cc:240] Negotiation successful
0923 22:16:14.547955 (+ 53us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":37,"mutex_wait_us":27}
I20260923 22:16:14.548482 20963 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "c247fea8afd943339528cf33079a1ea4" candidate_uuid: "670b827f9c034305b3e9cb2e335d9761" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "418f2d5cb24a43df99129b44c0a797dd" is_pre_election: true
I20260923 22:16:14.548645 20963 raft_consensus.cc:2454] T c247fea8afd943339528cf33079a1ea4 P 418f2d5cb24a43df99129b44c0a797dd [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 670b827f9c034305b3e9cb2e335d9761 in term 0.
I20260923 22:16:14.548633 20829 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "c247fea8afd943339528cf33079a1ea4" candidate_uuid: "670b827f9c034305b3e9cb2e335d9761" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "217a705b595a4a60be43f06a9f4a0479" is_pre_election: true
I20260923 22:16:14.548802 20829 raft_consensus.cc:2454] T c247fea8afd943339528cf33079a1ea4 P 217a705b595a4a60be43f06a9f4a0479 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 670b827f9c034305b3e9cb2e335d9761 in term 0.
I20260923 22:16:14.548873 21033 leader_election.cc:302] T c247fea8afd943339528cf33079a1ea4 P 670b827f9c034305b3e9cb2e335d9761 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 418f2d5cb24a43df99129b44c0a797dd, 670b827f9c034305b3e9cb2e335d9761; no voters:
I20260923 22:16:14.549000 21163 raft_consensus.cc:2781] T c247fea8afd943339528cf33079a1ea4 P 670b827f9c034305b3e9cb2e335d9761 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260923 22:16:14.549070 21163 raft_consensus.cc:482] T c247fea8afd943339528cf33079a1ea4 P 670b827f9c034305b3e9cb2e335d9761 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260923 22:16:14.549109 21163 raft_consensus.cc:3037] T c247fea8afd943339528cf33079a1ea4 P 670b827f9c034305b3e9cb2e335d9761 [term 0 FOLLOWER]: Advancing to term 1
I20260923 22:16:14.549822 21163 raft_consensus.cc:504] T c247fea8afd943339528cf33079a1ea4 P 670b827f9c034305b3e9cb2e335d9761 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "217a705b595a4a60be43f06a9f4a0479" member_type: VOTER last_known_addr { host: "127.1.135.193" port: 33649 } } peers { permanent_uuid: "418f2d5cb24a43df99129b44c0a797dd" member_type: VOTER last_known_addr { host: "127.1.135.194" port: 39767 } } peers { permanent_uuid: "670b827f9c034305b3e9cb2e335d9761" member_type: VOTER last_known_addr { host: "127.1.135.195" port: 41699 } }
I20260923 22:16:14.549963 21163 leader_election.cc:288] T c247fea8afd943339528cf33079a1ea4 P 670b827f9c034305b3e9cb2e335d9761 [CANDIDATE]: Term 1 election: Requested vote from peers 217a705b595a4a60be43f06a9f4a0479 (127.1.135.193:33649), 418f2d5cb24a43df99129b44c0a797dd (127.1.135.194:39767)
I20260923 22:16:14.550134 20963 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "c247fea8afd943339528cf33079a1ea4" candidate_uuid: "670b827f9c034305b3e9cb2e335d9761" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "418f2d5cb24a43df99129b44c0a797dd"
I20260923 22:16:14.550227 20963 raft_consensus.cc:3037] T c247fea8afd943339528cf33079a1ea4 P 418f2d5cb24a43df99129b44c0a797dd [term 0 FOLLOWER]: Advancing to term 1
I20260923 22:16:14.550985 20963 raft_consensus.cc:2454] T c247fea8afd943339528cf33079a1ea4 P 418f2d5cb24a43df99129b44c0a797dd [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 670b827f9c034305b3e9cb2e335d9761 in term 1.
I20260923 22:16:14.551129 20829 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "c247fea8afd943339528cf33079a1ea4" candidate_uuid: "670b827f9c034305b3e9cb2e335d9761" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "217a705b595a4a60be43f06a9f4a0479"
I20260923 22:16:14.551182 21033 leader_election.cc:302] T c247fea8afd943339528cf33079a1ea4 P 670b827f9c034305b3e9cb2e335d9761 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 418f2d5cb24a43df99129b44c0a797dd, 670b827f9c034305b3e9cb2e335d9761; no voters:
I20260923 22:16:14.551210 20829 raft_consensus.cc:3037] T c247fea8afd943339528cf33079a1ea4 P 217a705b595a4a60be43f06a9f4a0479 [term 0 FOLLOWER]: Advancing to term 1
I20260923 22:16:14.551291 21163 raft_consensus.cc:2781] T c247fea8afd943339528cf33079a1ea4 P 670b827f9c034305b3e9cb2e335d9761 [term 1 FOLLOWER]: Leader election won for term 1
I20260923 22:16:14.551471 21163 raft_consensus.cc:686] T c247fea8afd943339528cf33079a1ea4 P 670b827f9c034305b3e9cb2e335d9761 [term 1 LEADER]: Becoming Leader. State: Replica: 670b827f9c034305b3e9cb2e335d9761, State: Running, Role: LEADER
I20260923 22:16:14.551712 21163 consensus_queue.cc:237] T c247fea8afd943339528cf33079a1ea4 P 670b827f9c034305b3e9cb2e335d9761 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "217a705b595a4a60be43f06a9f4a0479" member_type: VOTER last_known_addr { host: "127.1.135.193" port: 33649 } } peers { permanent_uuid: "418f2d5cb24a43df99129b44c0a797dd" member_type: VOTER last_known_addr { host: "127.1.135.194" port: 39767 } } peers { permanent_uuid: "670b827f9c034305b3e9cb2e335d9761" member_type: VOTER last_known_addr { host: "127.1.135.195" port: 41699 } }
I20260923 22:16:14.552160 20829 raft_consensus.cc:2454] T c247fea8afd943339528cf33079a1ea4 P 217a705b595a4a60be43f06a9f4a0479 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 670b827f9c034305b3e9cb2e335d9761 in term 1.
I20260923 22:16:14.552603 20684 catalog_manager.cc:5495] T c247fea8afd943339528cf33079a1ea4 P 670b827f9c034305b3e9cb2e335d9761 reported cstate change: term changed from 0 to 1, leader changed from <none> to 670b827f9c034305b3e9cb2e335d9761 (127.1.135.195). New cstate: current_term: 1 leader_uuid: "670b827f9c034305b3e9cb2e335d9761" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "217a705b595a4a60be43f06a9f4a0479" member_type: VOTER last_known_addr { host: "127.1.135.193" port: 33649 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "418f2d5cb24a43df99129b44c0a797dd" member_type: VOTER last_known_addr { host: "127.1.135.194" port: 39767 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "670b827f9c034305b3e9cb2e335d9761" member_type: VOTER last_known_addr { host: "127.1.135.195" port: 41699 } health_report { overall_health: HEALTHY } } }
W20260923 22:16:14.584221 21144 tablet.cc:2369] T c247fea8afd943339528cf33079a1ea4 P 670b827f9c034305b3e9cb2e335d9761: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260923 22:16:14.595134 21156 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 22:16:14.592851 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34962 (local address 127.1.135.195:41699)
0923 22:16:14.592906 (+ 55us) server_negotiation.cc:206] Beginning negotiation
0923 22:16:14.592908 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0923 22:16:14.592917 (+ 9us) server_negotiation.cc:407] Connection header received
0923 22:16:14.592988 (+ 71us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 22:16:14.592989 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 22:16:14.593019 (+ 30us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 22:16:14.593094 (+ 75us) server_negotiation.cc:226] Negotiated authn=TOKEN
0923 22:16:14.593224 (+ 130us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 22:16:14.593662 (+ 438us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 22:16:14.594404 (+ 742us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 22:16:14.594591 (+ 187us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 22:16:14.594665 (+ 74us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 22:16:14.594834 (+ 169us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0923 22:16:14.594927 (+ 93us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0923 22:16:14.595019 (+ 92us) server_negotiation.cc:1035] Waiting for connection context
0923 22:16:14.595038 (+ 19us) server_negotiation.cc:299] Negotiation successful
0923 22:16:14.595057 (+ 19us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":33}
I20260923 22:16:14.596879 20829 raft_consensus.cc:1261] T c247fea8afd943339528cf33079a1ea4 P 217a705b595a4a60be43f06a9f4a0479 [term 1 FOLLOWER]: Refusing update from remote peer 670b827f9c034305b3e9cb2e335d9761: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260923 22:16:14.597172 20963 raft_consensus.cc:1261] T c247fea8afd943339528cf33079a1ea4 P 418f2d5cb24a43df99129b44c0a797dd [term 1 FOLLOWER]: Refusing update from remote peer 670b827f9c034305b3e9cb2e335d9761: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260923 22:16:14.597254 21167 consensus_queue.cc:1035] T c247fea8afd943339528cf33079a1ea4 P 670b827f9c034305b3e9cb2e335d9761 [LEADER]: Connected to new peer: Peer: permanent_uuid: "217a705b595a4a60be43f06a9f4a0479" member_type: VOTER last_known_addr { host: "127.1.135.193" port: 33649 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260923 22:16:14.597419 21167 consensus_queue.cc:1035] T c247fea8afd943339528cf33079a1ea4 P 670b827f9c034305b3e9cb2e335d9761 [LEADER]: Connected to new peer: Peer: permanent_uuid: "418f2d5cb24a43df99129b44c0a797dd" member_type: VOTER last_known_addr { host: "127.1.135.194" port: 39767 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260923 22:16:14.599402 21170 mvcc.cc:205] Tried to move back new op lower bound from 7332666468746072064 to 7332666468564508672. Current Snapshot: MvccSnapshot[applied={T|T < 7332666468746072064}]
I20260923 22:16:14.599993 21171 mvcc.cc:205] Tried to move back new op lower bound from 7332666468746072064 to 7332666468564508672. Current Snapshot: MvccSnapshot[applied={T|T < 7332666468746072064}]
I20260923 22:16:14.600186 21172 mvcc.cc:205] Tried to move back new op lower bound from 7332666468746072064 to 7332666468564508672. Current Snapshot: MvccSnapshot[applied={T|T < 7332666468746072064}]
I20260923 22:16:14.601820 20684 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:37036:
table { table_name: "test-table" } modify_external_catalogs: true
I20260923 22:16:14.601918 20684 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:37036:
table { table_name: "test-table" } modify_external_catalogs: true
I20260923 22:16:14.602675 20684 catalog_manager.cc:5782] T 00000000000000000000000000000000 P ca9eb5d833034e81bb733303438ed92b: Sending DeleteTablet for 3 replicas of tablet c247fea8afd943339528cf33079a1ea4
I20260923 22:16:14.603065 20943 tablet_service.cc:1514] Processing DeleteTablet for tablet c247fea8afd943339528cf33079a1ea4 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-23 22:16:14 UTC) from {username='kudu', principal='kudu/127.1.135.254@KRBTEST.COM'} at 127.0.0.1:60896
I20260923 22:16:14.603096 21077 tablet_service.cc:1514] Processing DeleteTablet for tablet c247fea8afd943339528cf33079a1ea4 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-23 22:16:14 UTC) from {username='kudu', principal='kudu/127.1.135.254@KRBTEST.COM'} at 127.0.0.1:34950
I20260923 22:16:14.603118 20809 tablet_service.cc:1514] Processing DeleteTablet for tablet c247fea8afd943339528cf33079a1ea4 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-23 22:16:14 UTC) from {username='kudu', principal='kudu/127.1.135.254@KRBTEST.COM'} at 127.0.0.1:48046
I20260923 22:16:14.603332 21182 tablet_replica.cc:330] T c247fea8afd943339528cf33079a1ea4 P 217a705b595a4a60be43f06a9f4a0479: stopping tablet replica
I20260923 22:16:14.603531 21181 tablet_replica.cc:330] T c247fea8afd943339528cf33079a1ea4 P 418f2d5cb24a43df99129b44c0a797dd: stopping tablet replica
I20260923 22:16:14.603539 21182 raft_consensus.cc:2229] T c247fea8afd943339528cf33079a1ea4 P 217a705b595a4a60be43f06a9f4a0479 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260923 22:16:14.603619 1567 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskRiQe_a/build/release/bin/kudu with pid 20746
I20260923 22:16:14.603649 21182 raft_consensus.cc:2258] T c247fea8afd943339528cf33079a1ea4 P 217a705b595a4a60be43f06a9f4a0479 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260923 22:16:14.604550 21180 tablet_replica.cc:330] T c247fea8afd943339528cf33079a1ea4 P 670b827f9c034305b3e9cb2e335d9761: stopping tablet replica
I20260923 22:16:14.604704 21181 raft_consensus.cc:2229] T c247fea8afd943339528cf33079a1ea4 P 418f2d5cb24a43df99129b44c0a797dd [term 1 FOLLOWER]: Raft consensus shutting down.
I20260923 22:16:14.604826 21181 raft_consensus.cc:2258] T c247fea8afd943339528cf33079a1ea4 P 418f2d5cb24a43df99129b44c0a797dd [term 1 FOLLOWER]: Raft consensus is shut down!
I20260923 22:16:14.604972 21180 raft_consensus.cc:2229] T c247fea8afd943339528cf33079a1ea4 P 670b827f9c034305b3e9cb2e335d9761 [term 1 LEADER]: Raft consensus shutting down.
I20260923 22:16:14.605120 21180 raft_consensus.cc:2258] T c247fea8afd943339528cf33079a1ea4 P 670b827f9c034305b3e9cb2e335d9761 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260923 22:16:14.605597 21180 ts_tablet_manager.cc:1905] T c247fea8afd943339528cf33079a1ea4 P 670b827f9c034305b3e9cb2e335d9761: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260923 22:16:14.607313 21180 ts_tablet_manager.cc:1918] T c247fea8afd943339528cf33079a1ea4 P 670b827f9c034305b3e9cb2e335d9761: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260923 22:16:14.607388 21180 log.cc:1196] T c247fea8afd943339528cf33079a1ea4 P 670b827f9c034305b3e9cb2e335d9761: Deleting WAL directory at /tmp/dist-test-taskRiQe_a/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790201539513534-1567-0/minicluster-data/ts-2/wal/wals/c247fea8afd943339528cf33079a1ea4
I20260923 22:16:14.607651 21180 ts_tablet_manager.cc:1939] T c247fea8afd943339528cf33079a1ea4 P 670b827f9c034305b3e9cb2e335d9761: Deleting consensus metadata
I20260923 22:16:14.608022 20671 catalog_manager.cc:4841] TS 670b827f9c034305b3e9cb2e335d9761 (127.1.135.195:41699): tablet c247fea8afd943339528cf33079a1ea4 (table test-table [id=00f1c0012de24709a612065333b278a4]) successfully deleted
W20260923 22:16:14.609633 20668 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.1.135.193:33649 (error 108)
I20260923 22:16:14.609844 1567 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskRiQe_a/build/release/bin/kudu with pid 20880
I20260923 22:16:14.611188 21151 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 22:16:14.611044 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.135.193:33649 (local address 127.0.0.1:48062)
0923 22:16:14.611112 (+ 68us) negotiation.cc:107] Waiting for socket to connect
0923 22:16:14.611156 (+ 44us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.135.193:33649: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":47}
W20260923 22:16:14.611310 20668 catalog_manager.cc:4584] TS 217a705b595a4a60be43f06a9f4a0479 (127.1.135.193:33649): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet c247fea8afd943339528cf33079a1ea4: Network error: Client connection negotiation failed: client connection to 127.1.135.193:33649: connect: Connection refused (error 111)
I20260923 22:16:14.614997 1567 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskRiQe_a/build/release/bin/kudu with pid 21014
I20260923 22:16:14.615079 21151 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 22:16:14.614983 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.135.194:39767 (local address 127.0.0.1:60906)
0923 22:16:14.615023 (+ 40us) negotiation.cc:107] Waiting for socket to connect
0923 22:16:14.615047 (+ 24us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.135.194:39767: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":21}
I20260923 22:16:14.620049 1567 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskRiQe_a/build/release/bin/kudu with pid 20652
2026-09-23T22:16:14Z chronyd exiting
[ OK ] EncryptionPolicyTest.TestEncryptionPolicy/4 (2031 ms)
[ RUN ] EncryptionPolicyTest.TestEncryptionPolicy/5
2026-09-23T22:16:14Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-09-23T22:16:14Z Disabled control of system clock
I20260923 22:16:14.647377 1567 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskRiQe_a/build/release/bin/kudu
/tmp/dist-test-taskRiQe_a/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskRiQe_a/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790201539513534-1567-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskRiQe_a/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790201539513534-1567-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskRiQe_a/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790201539513534-1567-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskRiQe_a/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790201539513534-1567-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.1.135.254:46849
--webserver_interface=127.1.135.254
--webserver_port=0
--builtin_ntp_servers=127.1.135.212:41113
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.1.135.254:46849
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
W20260923 22:16:14.725163 21187 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260923 22:16:14.725351 21187 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260923 22:16:14.725370 21187 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260923 22:16:14.726410 21187 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260923 22:16:14.726449 21187 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260923 22:16:14.726459 21187 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260923 22:16:14.726471 21187 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260923 22:16:14.726519 21187 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260923 22:16:14.727711 21187 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.1.135.212:41113
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskRiQe_a/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790201539513534-1567-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskRiQe_a/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790201539513534-1567-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.1.135.254:46849
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.1.135.254:46849
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskRiQe_a/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790201539513534-1567-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.1.135.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.21187
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskRiQe_a/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790201539513534-1567-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 23 Sep 2026 20:53:37 UTC on ccycloud.quasar-rkviyx.root.comops.site
build id 4668083
I20260923 22:16:14.727936 21187 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260923 22:16:14.728250 21187 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260923 22:16:14.732811 21192 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260923 22:16:14.732968 21187 server_base.cc:1034] running on GCE node
W20260923 22:16:14.732792 21195 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260923 22:16:14.732831 21193 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260923 22:16:14.733307 21187 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260923 22:16:14.733574 21187 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260923 22:16:14.734712 21187 hybrid_clock.cc:648] HybridClock initialized: now 1790201774734702 us; error 26 us; skew 500 ppm
I20260923 22:16:14.735788 21187 webserver.cc:460] Webserver started at http://127.1.135.254:35153/ using document root <none> and password file <none>
I20260923 22:16:14.736037 21187 fs_manager.cc:359] Metadata directory not provided
I20260923 22:16:14.736133 21187 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260923 22:16:14.736227 21187 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260923 22:16:14.737191 21187 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskRiQe_a/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790201539513534-1567-0/minicluster-data/master-0/data/instance:
uuid: "e18deb6b029b4f0b8dbd7f106df83cc8"
format_stamp: "Formatted at 2026-09-23 22:16:14 on dist-test-slave-s2fj"
server_key: "8bcf62edf2969576370b5cf5f653cbca"
server_key_iv: "da3dd6ddc838d91c91b49fbda81a1caa"
server_key_version: "encryptionkey@0"
I20260923 22:16:14.737463 21187 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskRiQe_a/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790201539513534-1567-0/minicluster-data/master-0/wal/instance:
uuid: "e18deb6b029b4f0b8dbd7f106df83cc8"
format_stamp: "Formatted at 2026-09-23 22:16:14 on dist-test-slave-s2fj"
server_key: "8bcf62edf2969576370b5cf5f653cbca"
server_key_iv: "da3dd6ddc838d91c91b49fbda81a1caa"
server_key_version: "encryptionkey@0"
I20260923 22:16:14.738600 21187 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.000s
I20260923 22:16:14.739256 21201 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260923 22:16:14.739408 21187 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260923 22:16:14.739488 21187 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskRiQe_a/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790201539513534-1567-0/minicluster-data/master-0/data,/tmp/dist-test-taskRiQe_a/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790201539513534-1567-0/minicluster-data/master-0/wal
uuid: "e18deb6b029b4f0b8dbd7f106df83cc8"
format_stamp: "Formatted at 2026-09-23 22:16:14 on dist-test-slave-s2fj"
server_key: "8bcf62edf2969576370b5cf5f653cbca"
server_key_iv: "da3dd6ddc838d91c91b49fbda81a1caa"
server_key_version: "encryptionkey@0"
I20260923 22:16:14.739564 21187 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskRiQe_a/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790201539513534-1567-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskRiQe_a/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790201539513534-1567-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskRiQe_a/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790201539513534-1567-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260923 22:16:14.747169 21187 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260923 22:16:14.747373 21187 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260923 22:16:14.747465 21187 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260923 22:16:14.751089 21187 rpc_server.cc:307] RPC server started. Bound to: 127.1.135.254:46849
I20260923 22:16:14.751124 21253 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.1.135.254:46849 every 8 connection(s)
I20260923 22:16:14.751430 21187 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskRiQe_a/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790201539513534-1567-0/minicluster-data/master-0/data/info.pb
I20260923 22:16:14.752386 21254 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260923 22:16:14.754395 21254 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P e18deb6b029b4f0b8dbd7f106df83cc8: Bootstrap starting.
I20260923 22:16:14.754922 21254 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P e18deb6b029b4f0b8dbd7f106df83cc8: Neither blocks nor log segments found. Creating new log.
I20260923 22:16:14.755201 21254 log.cc:824] T 00000000000000000000000000000000 P e18deb6b029b4f0b8dbd7f106df83cc8: Log is configured to *not* fsync() on all Append() calls
I20260923 22:16:14.755820 21254 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P e18deb6b029b4f0b8dbd7f106df83cc8: No bootstrap required, opened a new log
I20260923 22:16:14.757330 21254 raft_consensus.cc:348] T 00000000000000000000000000000000 P e18deb6b029b4f0b8dbd7f106df83cc8 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e18deb6b029b4f0b8dbd7f106df83cc8" member_type: VOTER last_known_addr { host: "127.1.135.254" port: 46849 } }
I20260923 22:16:14.757467 21254 raft_consensus.cc:374] T 00000000000000000000000000000000 P e18deb6b029b4f0b8dbd7f106df83cc8 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260923 22:16:14.757511 21254 raft_consensus.cc:729] T 00000000000000000000000000000000 P e18deb6b029b4f0b8dbd7f106df83cc8 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e18deb6b029b4f0b8dbd7f106df83cc8, State: Initialized, Role: FOLLOWER
I20260923 22:16:14.757655 21254 consensus_queue.cc:260] T 00000000000000000000000000000000 P e18deb6b029b4f0b8dbd7f106df83cc8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e18deb6b029b4f0b8dbd7f106df83cc8" member_type: VOTER last_known_addr { host: "127.1.135.254" port: 46849 } }
I20260923 22:16:14.757727 21254 raft_consensus.cc:388] T 00000000000000000000000000000000 P e18deb6b029b4f0b8dbd7f106df83cc8 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260923 22:16:14.757767 21254 raft_consensus.cc:482] T 00000000000000000000000000000000 P e18deb6b029b4f0b8dbd7f106df83cc8 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260923 22:16:14.757815 21254 raft_consensus.cc:3037] T 00000000000000000000000000000000 P e18deb6b029b4f0b8dbd7f106df83cc8 [term 0 FOLLOWER]: Advancing to term 1
I20260923 22:16:14.758267 21254 raft_consensus.cc:504] T 00000000000000000000000000000000 P e18deb6b029b4f0b8dbd7f106df83cc8 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e18deb6b029b4f0b8dbd7f106df83cc8" member_type: VOTER last_known_addr { host: "127.1.135.254" port: 46849 } }
I20260923 22:16:14.758378 21254 leader_election.cc:302] T 00000000000000000000000000000000 P e18deb6b029b4f0b8dbd7f106df83cc8 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: e18deb6b029b4f0b8dbd7f106df83cc8; no voters:
I20260923 22:16:14.758550 21254 leader_election.cc:288] T 00000000000000000000000000000000 P e18deb6b029b4f0b8dbd7f106df83cc8 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260923 22:16:14.758594 21257 raft_consensus.cc:2781] T 00000000000000000000000000000000 P e18deb6b029b4f0b8dbd7f106df83cc8 [term 1 FOLLOWER]: Leader election won for term 1
I20260923 22:16:14.758776 21254 sys_catalog.cc:564] T 00000000000000000000000000000000 P e18deb6b029b4f0b8dbd7f106df83cc8 [sys.catalog]: configured and running, proceeding with master startup.
I20260923 22:16:14.758796 21257 raft_consensus.cc:686] T 00000000000000000000000000000000 P e18deb6b029b4f0b8dbd7f106df83cc8 [term 1 LEADER]: Becoming Leader. State: Replica: e18deb6b029b4f0b8dbd7f106df83cc8, State: Running, Role: LEADER
I20260923 22:16:14.758909 21257 consensus_queue.cc:237] T 00000000000000000000000000000000 P e18deb6b029b4f0b8dbd7f106df83cc8 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e18deb6b029b4f0b8dbd7f106df83cc8" member_type: VOTER last_known_addr { host: "127.1.135.254" port: 46849 } }
I20260923 22:16:14.759265 21259 sys_catalog.cc:455] T 00000000000000000000000000000000 P e18deb6b029b4f0b8dbd7f106df83cc8 [sys.catalog]: SysCatalogTable state changed. Reason: New leader e18deb6b029b4f0b8dbd7f106df83cc8. Latest consensus state: current_term: 1 leader_uuid: "e18deb6b029b4f0b8dbd7f106df83cc8" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e18deb6b029b4f0b8dbd7f106df83cc8" member_type: VOTER last_known_addr { host: "127.1.135.254" port: 46849 } } }
I20260923 22:16:14.759366 21259 sys_catalog.cc:458] T 00000000000000000000000000000000 P e18deb6b029b4f0b8dbd7f106df83cc8 [sys.catalog]: This master's current role is: LEADER
I20260923 22:16:14.759341 21258 sys_catalog.cc:455] T 00000000000000000000000000000000 P e18deb6b029b4f0b8dbd7f106df83cc8 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "e18deb6b029b4f0b8dbd7f106df83cc8" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e18deb6b029b4f0b8dbd7f106df83cc8" member_type: VOTER last_known_addr { host: "127.1.135.254" port: 46849 } } }
I20260923 22:16:14.759570 21258 sys_catalog.cc:458] T 00000000000000000000000000000000 P e18deb6b029b4f0b8dbd7f106df83cc8 [sys.catalog]: This master's current role is: LEADER
I20260923 22:16:14.759585 21266 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260923 22:16:14.760321 21266 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260923 22:16:14.761358 1567 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskRiQe_a/build/release/bin/kudu as pid 21187
I20260923 22:16:14.761466 1567 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskRiQe_a/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790201539513534-1567-0/minicluster-data/master-0/wal/instance
I20260923 22:16:14.761654 1567 external_mini_cluster.cc:1428] Setting key a1e548c7d8bcbf5c1d2176dfdc79e1e0
I20260923 22:16:14.762275 21266 catalog_manager.cc:1347] Generated new cluster ID: 9585ade86ced4d46a8c27360c1a98bd3
I20260923 22:16:14.762332 21266 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260923 22:16:14.766492 21277 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 22:16:14.762092 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43662 (local address 127.1.135.254:46849)
0923 22:16:14.762265 (+ 173us) server_negotiation.cc:206] Beginning negotiation
0923 22:16:14.762273 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0923 22:16:14.762445 (+ 172us) server_negotiation.cc:407] Connection header received
0923 22:16:14.762591 (+ 146us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 22:16:14.762597 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 22:16:14.762645 (+ 48us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 22:16:14.762774 (+ 129us) server_negotiation.cc:226] Negotiated authn=SASL
0923 22:16:14.762968 (+ 194us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 22:16:14.763809 (+ 841us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 22:16:14.764660 (+ 851us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 22:16:14.765081 (+ 421us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 22:16:14.765218 (+ 137us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 22:16:14.765847 (+ 629us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0923 22:16:14.765850 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0923 22:16:14.765850 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0923 22:16:14.766021 (+ 171us) server_negotiation.cc:922] calling sasl_server_start()
0923 22:16:14.766051 (+ 30us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0923 22:16:14.766061 (+ 10us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0923 22:16:14.766131 (+ 70us) server_negotiation.cc:1035] Waiting for connection context
0923 22:16:14.766201 (+ 70us) server_negotiation.cc:299] Negotiation successful
0923 22:16:14.766272 (+ 71us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":106,"thread_start_us":66,"threads_started":1}
I20260923 22:16:14.766616 21266 catalog_manager.cc:1370] Generated new certificate authority record
I20260923 22:16:14.767155 21266 catalog_manager.cc:1504] Loading token signing keys...
I20260923 22:16:14.774503 21266 catalog_manager.cc:5868] T 00000000000000000000000000000000 P e18deb6b029b4f0b8dbd7f106df83cc8: Generated new TSK 0
I20260923 22:16:14.774655 21266 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260923 22:16:14.780715 1567 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskRiQe_a/build/release/bin/kudu
/tmp/dist-test-taskRiQe_a/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskRiQe_a/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790201539513534-1567-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskRiQe_a/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790201539513534-1567-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskRiQe_a/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790201539513534-1567-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskRiQe_a/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790201539513534-1567-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.1.135.193:0
--local_ip_for_outbound_sockets=127.1.135.193
--webserver_interface=127.1.135.193
--webserver_port=0
--tserver_master_addrs=127.1.135.254:46849
--builtin_ntp_servers=127.1.135.212:41113
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
W20260923 22:16:14.864027 21278 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260923 22:16:14.864272 21278 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260923 22:16:14.864295 21278 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260923 22:16:14.865459 21278 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260923 22:16:14.865509 21278 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260923 22:16:14.865532 21278 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.1.135.193
I20260923 22:16:14.866825 21278 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.1.135.212:41113
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskRiQe_a/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790201539513534-1567-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskRiQe_a/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790201539513534-1567-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.1.135.193:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskRiQe_a/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790201539513534-1567-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.1.135.193
--webserver_port=0
--tserver_master_addrs=127.1.135.254:46849
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.21278
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.1.135.193
--log_dir=/tmp/dist-test-taskRiQe_a/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790201539513534-1567-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 23 Sep 2026 20:53:37 UTC on ccycloud.quasar-rkviyx.root.comops.site
build id 4668083
I20260923 22:16:14.867046 21278 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260923 22:16:14.867334 21278 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260923 22:16:14.872843 21284 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260923 22:16:14.872866 21286 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260923 22:16:14.872876 21283 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260923 22:16:14.873179 21278 server_base.cc:1034] running on GCE node
I20260923 22:16:14.873397 21278 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260923 22:16:14.873631 21278 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260923 22:16:14.874822 21278 hybrid_clock.cc:648] HybridClock initialized: now 1790201774874803 us; error 35 us; skew 500 ppm
I20260923 22:16:14.876044 21278 webserver.cc:460] Webserver started at http://127.1.135.193:33299/ using document root <none> and password file <none>
I20260923 22:16:14.876331 21278 fs_manager.cc:359] Metadata directory not provided
I20260923 22:16:14.876376 21278 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260923 22:16:14.876502 21278 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260923 22:16:14.877558 21278 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskRiQe_a/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790201539513534-1567-0/minicluster-data/ts-0/data/instance:
uuid: "a55563674f284fd7b34c511b095befbb"
format_stamp: "Formatted at 2026-09-23 22:16:14 on dist-test-slave-s2fj"
server_key: "af8a40013bd6150117be365464ced359"
server_key_iv: "c895d42a84e9e4749084971128314d96"
server_key_version: "encryptionkey@0"
I20260923 22:16:14.877902 21278 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskRiQe_a/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790201539513534-1567-0/minicluster-data/ts-0/wal/instance:
uuid: "a55563674f284fd7b34c511b095befbb"
format_stamp: "Formatted at 2026-09-23 22:16:14 on dist-test-slave-s2fj"
server_key: "af8a40013bd6150117be365464ced359"
server_key_iv: "c895d42a84e9e4749084971128314d96"
server_key_version: "encryptionkey@0"
I20260923 22:16:14.879185 21278 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.003s
I20260923 22:16:14.879936 21292 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260923 22:16:14.880149 21278 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.001s
I20260923 22:16:14.880244 21278 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskRiQe_a/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790201539513534-1567-0/minicluster-data/ts-0/data,/tmp/dist-test-taskRiQe_a/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790201539513534-1567-0/minicluster-data/ts-0/wal
uuid: "a55563674f284fd7b34c511b095befbb"
format_stamp: "Formatted at 2026-09-23 22:16:14 on dist-test-slave-s2fj"
server_key: "af8a40013bd6150117be365464ced359"
server_key_iv: "c895d42a84e9e4749084971128314d96"
server_key_version: "encryptionkey@0"
I20260923 22:16:14.880319 21278 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskRiQe_a/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790201539513534-1567-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskRiQe_a/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790201539513534-1567-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskRiQe_a/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790201539513534-1567-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260923 22:16:14.897931 21278 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260923 22:16:14.898216 21278 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260923 22:16:14.898362 21278 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260923 22:16:14.898581 21278 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260923 22:16:14.898882 21278 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260923 22:16:14.898933 21278 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260923 22:16:14.898962 21278 ts_tablet_manager.cc:610] Registered 0 tablets
I20260923 22:16:14.898990 21278 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260923 22:16:14.904345 21278 rpc_server.cc:307] RPC server started. Bound to: 127.1.135.193:35715
I20260923 22:16:14.904450 21405 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.1.135.193:35715 every 8 connection(s)
I20260923 22:16:14.904701 21278 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskRiQe_a/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790201539513534-1567-0/minicluster-data/ts-0/data/info.pb
I20260923 22:16:14.906637 1567 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskRiQe_a/build/release/bin/kudu as pid 21278
I20260923 22:16:14.906765 1567 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskRiQe_a/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790201539513534-1567-0/minicluster-data/ts-0/wal/instance
I20260923 22:16:14.906982 1567 external_mini_cluster.cc:1428] Setting key 85a06a2b11fc3f2b3d941c7e4ee4f973
I20260923 22:16:14.907994 1567 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskRiQe_a/build/release/bin/kudu
/tmp/dist-test-taskRiQe_a/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskRiQe_a/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790201539513534-1567-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskRiQe_a/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790201539513534-1567-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskRiQe_a/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790201539513534-1567-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskRiQe_a/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790201539513534-1567-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.1.135.194:0
--local_ip_for_outbound_sockets=127.1.135.194
--webserver_interface=127.1.135.194
--webserver_port=0
--tserver_master_addrs=127.1.135.254:46849
--builtin_ntp_servers=127.1.135.212:41113
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
I20260923 22:16:14.910598 21277 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 22:16:14.905403 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.135.193:45915 (local address 127.1.135.254:46849)
0923 22:16:14.905453 (+ 50us) server_negotiation.cc:206] Beginning negotiation
0923 22:16:14.905456 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0923 22:16:14.905699 (+ 243us) server_negotiation.cc:407] Connection header received
0923 22:16:14.906158 (+ 459us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 22:16:14.906159 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 22:16:14.906170 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 22:16:14.906217 (+ 47us) server_negotiation.cc:226] Negotiated authn=SASL
0923 22:16:14.907060 (+ 843us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 22:16:14.907673 (+ 613us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 22:16:14.909166 (+ 1493us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 22:16:14.909487 (+ 321us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 22:16:14.909561 (+ 74us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 22:16:14.910339 (+ 778us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0923 22:16:14.910341 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0923 22:16:14.910341 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0923 22:16:14.910347 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0923 22:16:14.910361 (+ 14us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0923 22:16:14.910365 (+ 4us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0923 22:16:14.910422 (+ 57us) server_negotiation.cc:1035] Waiting for connection context
0923 22:16:14.910496 (+ 74us) server_negotiation.cc:299] Negotiation successful
0923 22:16:14.910524 (+ 28us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":24}
I20260923 22:16:14.910828 21408 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 22:16:14.905397 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.135.254:46849 (local address 127.1.135.193:45915)
0923 22:16:14.905586 (+ 189us) negotiation.cc:107] Waiting for socket to connect
0923 22:16:14.905617 (+ 31us) client_negotiation.cc:174] Beginning negotiation
0923 22:16:14.906066 (+ 449us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0923 22:16:14.906273 (+ 207us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0923 22:16:14.906276 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0923 22:16:14.906284 (+ 8us) client_negotiation.cc:189] Negotiated authn=SASL
0923 22:16:14.906945 (+ 661us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 22:16:14.906952 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 22:16:14.907767 (+ 815us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 22:16:14.907768 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 22:16:14.909079 (+ 1311us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 22:16:14.909083 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 22:16:14.909585 (+ 502us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 22:16:14.909586 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 22:16:14.909690 (+ 104us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 22:16:14.910203 (+ 513us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0923 22:16:14.910203 (+ 0us) client_negotiation.cc:656] calling sasl_client_start()
0923 22:16:14.910228 (+ 25us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0923 22:16:14.910249 (+ 21us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0923 22:16:14.910424 (+ 175us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0923 22:16:14.910429 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0923 22:16:14.910431 (+ 2us) client_negotiation.cc:769] Sending connection context
0923 22:16:14.910489 (+ 58us) client_negotiation.cc:240] Negotiation successful
0923 22:16:14.910583 (+ 94us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":134,"thread_start_us":97,"threads_started":1}
I20260923 22:16:14.911257 21406 heartbeater.cc:347] Connected to a master server at 127.1.135.254:46849
I20260923 22:16:14.911358 21406 heartbeater.cc:464] Registering TS with master...
I20260923 22:16:14.911541 21406 heartbeater.cc:511] Master 127.1.135.254:46849 requested a full tablet report, sending...
I20260923 22:16:14.912038 21218 ts_manager.cc:194] Registered new tserver with Master: a55563674f284fd7b34c511b095befbb (127.1.135.193:35715)
I20260923 22:16:14.912636 21218 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.1.135.193:45915
W20260923 22:16:14.987033 21409 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260923 22:16:14.987216 21409 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260923 22:16:14.987237 21409 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260923 22:16:14.988430 21409 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260923 22:16:14.988467 21409 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260923 22:16:14.988492 21409 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.1.135.194
I20260923 22:16:14.989806 21409 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.1.135.212:41113
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskRiQe_a/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790201539513534-1567-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskRiQe_a/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790201539513534-1567-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.1.135.194:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskRiQe_a/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790201539513534-1567-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.1.135.194
--webserver_port=0
--tserver_master_addrs=127.1.135.254:46849
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.21409
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.1.135.194
--log_dir=/tmp/dist-test-taskRiQe_a/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790201539513534-1567-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 23 Sep 2026 20:53:37 UTC on ccycloud.quasar-rkviyx.root.comops.site
build id 4668083
I20260923 22:16:14.990025 21409 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260923 22:16:14.990301 21409 file_cache.cc:493] Constructed file cache file cache with capacity 419430
I20260923 22:16:14.995064 21409 server_base.cc:1034] running on GCE node
W20260923 22:16:14.995038 21415 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260923 22:16:14.995045 21417 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260923 22:16:14.995016 21414 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260923 22:16:14.995481 21409 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260923 22:16:14.995712 21409 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260923 22:16:14.996865 21409 hybrid_clock.cc:648] HybridClock initialized: now 1790201774996856 us; error 37 us; skew 500 ppm
I20260923 22:16:14.998065 21409 webserver.cc:460] Webserver started at http://127.1.135.194:43851/ using document root <none> and password file <none>
I20260923 22:16:14.998298 21409 fs_manager.cc:359] Metadata directory not provided
I20260923 22:16:14.998371 21409 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260923 22:16:14.998514 21409 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260923 22:16:14.999518 21409 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskRiQe_a/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790201539513534-1567-0/minicluster-data/ts-1/data/instance:
uuid: "ed2016d3874a444fa51ea3152e78e354"
format_stamp: "Formatted at 2026-09-23 22:16:14 on dist-test-slave-s2fj"
server_key: "557b3c82a88ba6aa2c99455fa829f47d"
server_key_iv: "0156ac87d45571fcafb644e8839e7c56"
server_key_version: "encryptionkey@0"
I20260923 22:16:14.999871 21409 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskRiQe_a/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790201539513534-1567-0/minicluster-data/ts-1/wal/instance:
uuid: "ed2016d3874a444fa51ea3152e78e354"
format_stamp: "Formatted at 2026-09-23 22:16:14 on dist-test-slave-s2fj"
server_key: "557b3c82a88ba6aa2c99455fa829f47d"
server_key_iv: "0156ac87d45571fcafb644e8839e7c56"
server_key_version: "encryptionkey@0"
I20260923 22:16:15.001168 21409 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.001s sys 0.000s
I20260923 22:16:15.001860 21423 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260923 22:16:15.002027 21409 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260923 22:16:15.002128 21409 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskRiQe_a/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790201539513534-1567-0/minicluster-data/ts-1/data,/tmp/dist-test-taskRiQe_a/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790201539513534-1567-0/minicluster-data/ts-1/wal
uuid: "ed2016d3874a444fa51ea3152e78e354"
format_stamp: "Formatted at 2026-09-23 22:16:14 on dist-test-slave-s2fj"
server_key: "557b3c82a88ba6aa2c99455fa829f47d"
server_key_iv: "0156ac87d45571fcafb644e8839e7c56"
server_key_version: "encryptionkey@0"
I20260923 22:16:15.002220 21409 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskRiQe_a/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790201539513534-1567-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskRiQe_a/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790201539513534-1567-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskRiQe_a/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790201539513534-1567-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260923 22:16:15.026062 21409 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260923 22:16:15.026381 21409 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260923 22:16:15.026520 21409 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260923 22:16:15.026736 21409 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260923 22:16:15.027047 21409 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260923 22:16:15.027096 21409 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260923 22:16:15.027132 21409 ts_tablet_manager.cc:610] Registered 0 tablets
I20260923 22:16:15.027158 21409 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260923 22:16:15.032760 21409 rpc_server.cc:307] RPC server started. Bound to: 127.1.135.194:46433
I20260923 22:16:15.032806 21536 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.1.135.194:46433 every 8 connection(s)
I20260923 22:16:15.033115 21409 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskRiQe_a/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790201539513534-1567-0/minicluster-data/ts-1/data/info.pb
I20260923 22:16:15.033397 1567 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskRiQe_a/build/release/bin/kudu as pid 21409
I20260923 22:16:15.033509 1567 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskRiQe_a/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790201539513534-1567-0/minicluster-data/ts-1/wal/instance
I20260923 22:16:15.033703 1567 external_mini_cluster.cc:1428] Setting key 7f5116a882a18c8006b36f758203de57
I20260923 22:16:15.034754 1567 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskRiQe_a/build/release/bin/kudu
/tmp/dist-test-taskRiQe_a/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskRiQe_a/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790201539513534-1567-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskRiQe_a/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790201539513534-1567-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskRiQe_a/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790201539513534-1567-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskRiQe_a/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790201539513534-1567-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.1.135.195:0
--local_ip_for_outbound_sockets=127.1.135.195
--webserver_interface=127.1.135.195
--webserver_port=0
--tserver_master_addrs=127.1.135.254:46849
--builtin_ntp_servers=127.1.135.212:41113
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
I20260923 22:16:15.039009 21277 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 22:16:15.033842 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.135.194:48365 (local address 127.1.135.254:46849)
0923 22:16:15.033900 (+ 58us) server_negotiation.cc:206] Beginning negotiation
0923 22:16:15.033903 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0923 22:16:15.034165 (+ 262us) server_negotiation.cc:407] Connection header received
0923 22:16:15.034805 (+ 640us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 22:16:15.034805 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 22:16:15.034828 (+ 23us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 22:16:15.034885 (+ 57us) server_negotiation.cc:226] Negotiated authn=SASL
0923 22:16:15.035789 (+ 904us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 22:16:15.036490 (+ 701us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 22:16:15.037462 (+ 972us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 22:16:15.037818 (+ 356us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 22:16:15.037888 (+ 70us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 22:16:15.038707 (+ 819us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0923 22:16:15.038709 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0923 22:16:15.038709 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0923 22:16:15.038715 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0923 22:16:15.038732 (+ 17us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0923 22:16:15.038737 (+ 5us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0923 22:16:15.038796 (+ 59us) server_negotiation.cc:1035] Waiting for connection context
0923 22:16:15.038866 (+ 70us) server_negotiation.cc:299] Negotiation successful
0923 22:16:15.038895 (+ 29us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":24}
I20260923 22:16:15.039121 21539 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 22:16:15.033845 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.135.254:46849 (local address 127.1.135.194:48365)
0923 22:16:15.034053 (+ 208us) negotiation.cc:107] Waiting for socket to connect
0923 22:16:15.034095 (+ 42us) client_negotiation.cc:174] Beginning negotiation
0923 22:16:15.034689 (+ 594us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0923 22:16:15.034971 (+ 282us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0923 22:16:15.034976 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0923 22:16:15.034985 (+ 9us) client_negotiation.cc:189] Negotiated authn=SASL
0923 22:16:15.035698 (+ 713us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 22:16:15.035704 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 22:16:15.036582 (+ 878us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 22:16:15.036584 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 22:16:15.037374 (+ 790us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 22:16:15.037376 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 22:16:15.037900 (+ 524us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 22:16:15.037901 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 22:16:15.038001 (+ 100us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 22:16:15.038573 (+ 572us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0923 22:16:15.038573 (+ 0us) client_negotiation.cc:656] calling sasl_client_start()
0923 22:16:15.038599 (+ 26us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0923 22:16:15.038619 (+ 20us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0923 22:16:15.038811 (+ 192us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0923 22:16:15.038815 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0923 22:16:15.038817 (+ 2us) client_negotiation.cc:769] Sending connection context
0923 22:16:15.038864 (+ 47us) client_negotiation.cc:240] Negotiation successful
0923 22:16:15.038937 (+ 73us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":142,"thread_start_us":79,"threads_started":1}
I20260923 22:16:15.039577 21537 heartbeater.cc:347] Connected to a master server at 127.1.135.254:46849
I20260923 22:16:15.039685 21537 heartbeater.cc:464] Registering TS with master...
I20260923 22:16:15.039852 21537 heartbeater.cc:511] Master 127.1.135.254:46849 requested a full tablet report, sending...
I20260923 22:16:15.040242 21218 ts_manager.cc:194] Registered new tserver with Master: ed2016d3874a444fa51ea3152e78e354 (127.1.135.194:46433)
I20260923 22:16:15.040782 21218 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.1.135.194:48365
W20260923 22:16:15.115090 21540 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260923 22:16:15.115262 21540 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260923 22:16:15.115283 21540 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260923 22:16:15.116379 21540 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260923 22:16:15.116413 21540 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260923 22:16:15.116437 21540 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.1.135.195
I20260923 22:16:15.117673 21540 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.1.135.212:41113
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskRiQe_a/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790201539513534-1567-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskRiQe_a/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790201539513534-1567-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.1.135.195:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskRiQe_a/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790201539513534-1567-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.1.135.195
--webserver_port=0
--tserver_master_addrs=127.1.135.254:46849
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.21540
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.1.135.195
--log_dir=/tmp/dist-test-taskRiQe_a/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790201539513534-1567-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 23 Sep 2026 20:53:37 UTC on ccycloud.quasar-rkviyx.root.comops.site
build id 4668083
I20260923 22:16:15.117833 21540 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260923 22:16:15.118034 21540 file_cache.cc:493] Constructed file cache file cache with capacity 419430
I20260923 22:16:15.122815 21540 server_base.cc:1034] running on GCE node
W20260923 22:16:15.122788 21546 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260923 22:16:15.122779 21545 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260923 22:16:15.122784 21548 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260923 22:16:15.123183 21540 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260923 22:16:15.123445 21540 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260923 22:16:15.124593 21540 hybrid_clock.cc:648] HybridClock initialized: now 1790201775124565 us; error 39 us; skew 500 ppm
I20260923 22:16:15.125782 21540 webserver.cc:460] Webserver started at http://127.1.135.195:41389/ using document root <none> and password file <none>
I20260923 22:16:15.126009 21540 fs_manager.cc:359] Metadata directory not provided
I20260923 22:16:15.126094 21540 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260923 22:16:15.126231 21540 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260923 22:16:15.127184 21540 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskRiQe_a/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790201539513534-1567-0/minicluster-data/ts-2/data/instance:
uuid: "49582794a7274034b760d63fd930d65f"
format_stamp: "Formatted at 2026-09-23 22:16:15 on dist-test-slave-s2fj"
server_key: "6e50caaf9f542f6c7906402e4d2721bc"
server_key_iv: "6a50f5ffb8bbf9808e5a3b2423e909c8"
server_key_version: "encryptionkey@0"
I20260923 22:16:15.127518 21540 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskRiQe_a/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790201539513534-1567-0/minicluster-data/ts-2/wal/instance:
uuid: "49582794a7274034b760d63fd930d65f"
format_stamp: "Formatted at 2026-09-23 22:16:15 on dist-test-slave-s2fj"
server_key: "6e50caaf9f542f6c7906402e4d2721bc"
server_key_iv: "6a50f5ffb8bbf9808e5a3b2423e909c8"
server_key_version: "encryptionkey@0"
I20260923 22:16:15.128777 21540 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.003s
I20260923 22:16:15.129416 21554 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260923 22:16:15.129655 21540 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.001s
I20260923 22:16:15.129747 21540 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskRiQe_a/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790201539513534-1567-0/minicluster-data/ts-2/data,/tmp/dist-test-taskRiQe_a/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790201539513534-1567-0/minicluster-data/ts-2/wal
uuid: "49582794a7274034b760d63fd930d65f"
format_stamp: "Formatted at 2026-09-23 22:16:15 on dist-test-slave-s2fj"
server_key: "6e50caaf9f542f6c7906402e4d2721bc"
server_key_iv: "6a50f5ffb8bbf9808e5a3b2423e909c8"
server_key_version: "encryptionkey@0"
I20260923 22:16:15.129822 21540 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskRiQe_a/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790201539513534-1567-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskRiQe_a/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790201539513534-1567-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskRiQe_a/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790201539513534-1567-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260923 22:16:15.144481 21540 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260923 22:16:15.144724 21540 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260923 22:16:15.144860 21540 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260923 22:16:15.145051 21540 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260923 22:16:15.145329 21540 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260923 22:16:15.145380 21540 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260923 22:16:15.145417 21540 ts_tablet_manager.cc:610] Registered 0 tablets
I20260923 22:16:15.145443 21540 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260923 22:16:15.150817 21540 rpc_server.cc:307] RPC server started. Bound to: 127.1.135.195:42169
I20260923 22:16:15.150898 21667 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.1.135.195:42169 every 8 connection(s)
I20260923 22:16:15.151175 21540 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskRiQe_a/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790201539513534-1567-0/minicluster-data/ts-2/data/info.pb
I20260923 22:16:15.155712 21277 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 22:16:15.151848 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.135.195:47233 (local address 127.1.135.254:46849)
0923 22:16:15.151907 (+ 59us) server_negotiation.cc:206] Beginning negotiation
0923 22:16:15.151909 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0923 22:16:15.152115 (+ 206us) server_negotiation.cc:407] Connection header received
0923 22:16:15.152547 (+ 432us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 22:16:15.152548 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 22:16:15.152556 (+ 8us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 22:16:15.152612 (+ 56us) server_negotiation.cc:226] Negotiated authn=SASL
0923 22:16:15.153165 (+ 553us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 22:16:15.153661 (+ 496us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 22:16:15.154551 (+ 890us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 22:16:15.154732 (+ 181us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 22:16:15.154777 (+ 45us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 22:16:15.155531 (+ 754us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0923 22:16:15.155531 (+ 0us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0923 22:16:15.155532 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0923 22:16:15.155535 (+ 3us) server_negotiation.cc:922] calling sasl_server_start()
0923 22:16:15.155545 (+ 10us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0923 22:16:15.155548 (+ 3us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0923 22:16:15.155584 (+ 36us) server_negotiation.cc:1035] Waiting for connection context
0923 22:16:15.155641 (+ 57us) server_negotiation.cc:299] Negotiation successful
0923 22:16:15.155660 (+ 19us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":37}
I20260923 22:16:15.155877 21670 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 22:16:15.151835 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.135.254:46849 (local address 127.1.135.195:47233)
0923 22:16:15.151996 (+ 161us) negotiation.cc:107] Waiting for socket to connect
0923 22:16:15.152031 (+ 35us) client_negotiation.cc:174] Beginning negotiation
0923 22:16:15.152470 (+ 439us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0923 22:16:15.152660 (+ 190us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0923 22:16:15.152663 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0923 22:16:15.152670 (+ 7us) client_negotiation.cc:189] Negotiated authn=SASL
0923 22:16:15.153104 (+ 434us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 22:16:15.153107 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 22:16:15.153738 (+ 631us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 22:16:15.153739 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 22:16:15.154482 (+ 743us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 22:16:15.154484 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 22:16:15.154800 (+ 316us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 22:16:15.154800 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 22:16:15.154895 (+ 95us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 22:16:15.155423 (+ 528us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0923 22:16:15.155423 (+ 0us) client_negotiation.cc:656] calling sasl_client_start()
0923 22:16:15.155447 (+ 24us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0923 22:16:15.155467 (+ 20us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0923 22:16:15.155586 (+ 119us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0923 22:16:15.155588 (+ 2us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0923 22:16:15.155590 (+ 2us) client_negotiation.cc:769] Sending connection context
0923 22:16:15.155632 (+ 42us) client_negotiation.cc:240] Negotiation successful
0923 22:16:15.155698 (+ 66us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":103,"thread_start_us":78,"threads_started":1}
I20260923 22:16:15.156313 21668 heartbeater.cc:347] Connected to a master server at 127.1.135.254:46849
I20260923 22:16:15.156402 21668 heartbeater.cc:464] Registering TS with master...
I20260923 22:16:15.156533 21668 heartbeater.cc:511] Master 127.1.135.254:46849 requested a full tablet report, sending...
I20260923 22:16:15.156891 21218 ts_manager.cc:194] Registered new tserver with Master: 49582794a7274034b760d63fd930d65f (127.1.135.195:42169)
I20260923 22:16:15.157195 21218 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.1.135.195:47233
I20260923 22:16:15.159993 1567 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskRiQe_a/build/release/bin/kudu as pid 21540
I20260923 22:16:15.160145 1567 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskRiQe_a/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790201539513534-1567-0/minicluster-data/ts-2/wal/instance
I20260923 22:16:15.160337 1567 external_mini_cluster.cc:1428] Setting key 447ae085b57e0546532c6a04670d0b96
I20260923 22:16:15.161163 1567 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260923 22:16:15.164803 21277 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 22:16:15.162416 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43670 (local address 127.1.135.254:46849)
0923 22:16:15.162475 (+ 59us) server_negotiation.cc:206] Beginning negotiation
0923 22:16:15.162476 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0923 22:16:15.162573 (+ 97us) server_negotiation.cc:407] Connection header received
0923 22:16:15.162608 (+ 35us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 22:16:15.162609 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 22:16:15.162618 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 22:16:15.162664 (+ 46us) server_negotiation.cc:226] Negotiated authn=SASL
0923 22:16:15.162795 (+ 131us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 22:16:15.163213 (+ 418us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 22:16:15.163878 (+ 665us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 22:16:15.164022 (+ 144us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 22:16:15.164149 (+ 127us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 22:16:15.164624 (+ 475us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0923 22:16:15.164625 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0923 22:16:15.164625 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0923 22:16:15.164628 (+ 3us) server_negotiation.cc:922] calling sasl_server_start()
0923 22:16:15.164639 (+ 11us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0923 22:16:15.164642 (+ 3us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0923 22:16:15.164686 (+ 44us) server_negotiation.cc:1035] Waiting for connection context
0923 22:16:15.164732 (+ 46us) server_negotiation.cc:299] Negotiation successful
0923 22:16:15.164749 (+ 17us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":40}
I20260923 22:16:15.166064 21218 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:43670:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260923 22:16:15.167455 21218 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260923 22:16:15.172817 21676 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 22:16:15.169312 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.135.194:46433 (local address 127.0.0.1:33572)
0923 22:16:15.169616 (+ 304us) negotiation.cc:107] Waiting for socket to connect
0923 22:16:15.169650 (+ 34us) client_negotiation.cc:174] Beginning negotiation
0923 22:16:15.169725 (+ 75us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0923 22:16:15.169915 (+ 190us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0923 22:16:15.169918 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0923 22:16:15.169920 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0923 22:16:15.169972 (+ 52us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 22:16:15.169973 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 22:16:15.170959 (+ 986us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 22:16:15.170960 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 22:16:15.171706 (+ 746us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 22:16:15.171708 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 22:16:15.172553 (+ 845us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 22:16:15.172554 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 22:16:15.172634 (+ 80us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 22:16:15.172646 (+ 12us) client_negotiation.cc:769] Sending connection context
0923 22:16:15.172704 (+ 58us) client_negotiation.cc:240] Negotiation successful
0923 22:16:15.172749 (+ 45us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":281,"mutex_wait_us":3,"thread_start_us":76,"threads_started":1}
I20260923 22:16:15.173089 21679 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 22:16:15.169413 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33572 (local address 127.1.135.194:46433)
0923 22:16:15.169604 (+ 191us) server_negotiation.cc:206] Beginning negotiation
0923 22:16:15.169608 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0923 22:16:15.169725 (+ 117us) server_negotiation.cc:407] Connection header received
0923 22:16:15.169814 (+ 89us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 22:16:15.169817 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 22:16:15.169832 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 22:16:15.169904 (+ 72us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0923 22:16:15.170026 (+ 122us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 22:16:15.170873 (+ 847us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 22:16:15.171810 (+ 937us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 22:16:15.172461 (+ 651us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 22:16:15.172540 (+ 79us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 22:16:15.172620 (+ 80us) server_negotiation.cc:1035] Waiting for connection context
0923 22:16:15.172979 (+ 359us) server_negotiation.cc:299] Negotiation successful
0923 22:16:15.173014 (+ 35us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":150,"thread_start_us":101,"threads_started":1}
I20260923 22:16:15.173648 21677 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 22:16:15.169460 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.135.195:42169 (local address 127.0.0.1:33258)
0923 22:16:15.169861 (+ 401us) negotiation.cc:107] Waiting for socket to connect
0923 22:16:15.169871 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0923 22:16:15.169945 (+ 74us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0923 22:16:15.170166 (+ 221us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0923 22:16:15.170167 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0923 22:16:15.170169 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0923 22:16:15.170197 (+ 28us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 22:16:15.170198 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 22:16:15.171399 (+ 1201us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 22:16:15.171400 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 22:16:15.172581 (+ 1181us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 22:16:15.172583 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 22:16:15.173487 (+ 904us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 22:16:15.173488 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 22:16:15.173530 (+ 42us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 22:16:15.173532 (+ 2us) client_negotiation.cc:769] Sending connection context
0923 22:16:15.173571 (+ 39us) client_negotiation.cc:240] Negotiation successful
0923 22:16:15.173599 (+ 28us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":386,"mutex_wait_us":44,"thread_start_us":76,"threads_started":1}
I20260923 22:16:15.173729 21681 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 22:16:15.169654 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33258 (local address 127.1.135.195:42169)
0923 22:16:15.169796 (+ 142us) server_negotiation.cc:206] Beginning negotiation
0923 22:16:15.169800 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0923 22:16:15.169919 (+ 119us) server_negotiation.cc:407] Connection header received
0923 22:16:15.170033 (+ 114us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 22:16:15.170035 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 22:16:15.170049 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 22:16:15.170096 (+ 47us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0923 22:16:15.170279 (+ 183us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 22:16:15.171309 (+ 1030us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 22:16:15.172684 (+ 1375us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 22:16:15.173354 (+ 670us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 22:16:15.173390 (+ 36us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 22:16:15.173440 (+ 50us) server_negotiation.cc:1035] Waiting for connection context
0923 22:16:15.173612 (+ 172us) server_negotiation.cc:299] Negotiation successful
0923 22:16:15.173653 (+ 41us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":115,"thread_start_us":68,"threads_started":1}
I20260923 22:16:15.173671 21471 tablet_service.cc:1467] Processing CreateTablet for tablet 69a66bcffb8245d7bee04bb38834afff (DEFAULT_TABLE table=test-table [id=ddca15567d50436dbd4be3734bbee1c4]), partition=RANGE (key) PARTITION UNBOUNDED
I20260923 22:16:15.173988 21471 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 69a66bcffb8245d7bee04bb38834afff. 1 dirs total, 0 dirs full, 0 dirs failed
I20260923 22:16:15.174288 21602 tablet_service.cc:1467] Processing CreateTablet for tablet 69a66bcffb8245d7bee04bb38834afff (DEFAULT_TABLE table=test-table [id=ddca15567d50436dbd4be3734bbee1c4]), partition=RANGE (key) PARTITION UNBOUNDED
I20260923 22:16:15.174510 21602 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 69a66bcffb8245d7bee04bb38834afff. 1 dirs total, 0 dirs full, 0 dirs failed
I20260923 22:16:15.176664 21682 tablet_bootstrap.cc:492] T 69a66bcffb8245d7bee04bb38834afff P ed2016d3874a444fa51ea3152e78e354: Bootstrap starting.
I20260923 22:16:15.177122 21683 tablet_bootstrap.cc:492] T 69a66bcffb8245d7bee04bb38834afff P 49582794a7274034b760d63fd930d65f: Bootstrap starting.
I20260923 22:16:15.177300 21678 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 22:16:15.169363 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54272 (local address 127.1.135.193:35715)
0923 22:16:15.169499 (+ 136us) server_negotiation.cc:206] Beginning negotiation
0923 22:16:15.169503 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0923 22:16:15.169738 (+ 235us) server_negotiation.cc:407] Connection header received
0923 22:16:15.169890 (+ 152us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 22:16:15.169893 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 22:16:15.169911 (+ 18us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 22:16:15.169988 (+ 77us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0923 22:16:15.170188 (+ 200us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 22:16:15.171300 (+ 1112us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 22:16:15.172694 (+ 1394us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 22:16:15.176762 (+ 4068us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 22:16:15.176853 (+ 91us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 22:16:15.176940 (+ 87us) server_negotiation.cc:1035] Waiting for connection context
0923 22:16:15.177146 (+ 206us) server_negotiation.cc:299] Negotiation successful
0923 22:16:15.177195 (+ 49us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":103,"thread_start_us":73,"threads_started":1}
I20260923 22:16:15.177567 21680 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 22:16:15.169260 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.135.193:35715 (local address 127.0.0.1:54272)
0923 22:16:15.169615 (+ 355us) negotiation.cc:107] Waiting for socket to connect
0923 22:16:15.169649 (+ 34us) client_negotiation.cc:174] Beginning negotiation
0923 22:16:15.169769 (+ 120us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0923 22:16:15.170055 (+ 286us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0923 22:16:15.170055 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0923 22:16:15.170057 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0923 22:16:15.170102 (+ 45us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 22:16:15.170104 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 22:16:15.171399 (+ 1295us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 22:16:15.171400 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 22:16:15.172581 (+ 1181us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 22:16:15.172583 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 22:16:15.176978 (+ 4395us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 22:16:15.176979 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 22:16:15.177050 (+ 71us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 22:16:15.177054 (+ 4us) client_negotiation.cc:769] Sending connection context
0923 22:16:15.177426 (+ 372us) client_negotiation.cc:240] Negotiation successful
0923 22:16:15.177481 (+ 55us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":315,"mutex_wait_us":45,"thread_start_us":80,"threads_started":1}
I20260923 22:16:15.177639 21682 tablet_bootstrap.cc:654] T 69a66bcffb8245d7bee04bb38834afff P ed2016d3874a444fa51ea3152e78e354: Neither blocks nor log segments found. Creating new log.
I20260923 22:16:15.177686 21683 tablet_bootstrap.cc:654] T 69a66bcffb8245d7bee04bb38834afff P 49582794a7274034b760d63fd930d65f: Neither blocks nor log segments found. Creating new log.
I20260923 22:16:15.177989 21683 log.cc:824] T 69a66bcffb8245d7bee04bb38834afff P 49582794a7274034b760d63fd930d65f: Log is configured to *not* fsync() on all Append() calls
I20260923 22:16:15.178076 21682 log.cc:824] T 69a66bcffb8245d7bee04bb38834afff P ed2016d3874a444fa51ea3152e78e354: Log is configured to *not* fsync() on all Append() calls
I20260923 22:16:15.178831 21683 tablet_bootstrap.cc:492] T 69a66bcffb8245d7bee04bb38834afff P 49582794a7274034b760d63fd930d65f: No bootstrap required, opened a new log
I20260923 22:16:15.178853 21682 tablet_bootstrap.cc:492] T 69a66bcffb8245d7bee04bb38834afff P ed2016d3874a444fa51ea3152e78e354: No bootstrap required, opened a new log
I20260923 22:16:15.178920 21683 ts_tablet_manager.cc:1397] T 69a66bcffb8245d7bee04bb38834afff P 49582794a7274034b760d63fd930d65f: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20260923 22:16:15.178931 21682 ts_tablet_manager.cc:1397] T 69a66bcffb8245d7bee04bb38834afff P ed2016d3874a444fa51ea3152e78e354: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20260923 22:16:15.181008 21340 tablet_service.cc:1467] Processing CreateTablet for tablet 69a66bcffb8245d7bee04bb38834afff (DEFAULT_TABLE table=test-table [id=ddca15567d50436dbd4be3734bbee1c4]), partition=RANGE (key) PARTITION UNBOUNDED
I20260923 22:16:15.181311 21340 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 69a66bcffb8245d7bee04bb38834afff. 1 dirs total, 0 dirs full, 0 dirs failed
I20260923 22:16:15.181516 21682 raft_consensus.cc:348] T 69a66bcffb8245d7bee04bb38834afff P ed2016d3874a444fa51ea3152e78e354 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a55563674f284fd7b34c511b095befbb" member_type: VOTER last_known_addr { host: "127.1.135.193" port: 35715 } } peers { permanent_uuid: "49582794a7274034b760d63fd930d65f" member_type: VOTER last_known_addr { host: "127.1.135.195" port: 42169 } } peers { permanent_uuid: "ed2016d3874a444fa51ea3152e78e354" member_type: VOTER last_known_addr { host: "127.1.135.194" port: 46433 } }
I20260923 22:16:15.181665 21682 raft_consensus.cc:374] T 69a66bcffb8245d7bee04bb38834afff P ed2016d3874a444fa51ea3152e78e354 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260923 22:16:15.181566 21683 raft_consensus.cc:348] T 69a66bcffb8245d7bee04bb38834afff P 49582794a7274034b760d63fd930d65f [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a55563674f284fd7b34c511b095befbb" member_type: VOTER last_known_addr { host: "127.1.135.193" port: 35715 } } peers { permanent_uuid: "49582794a7274034b760d63fd930d65f" member_type: VOTER last_known_addr { host: "127.1.135.195" port: 42169 } } peers { permanent_uuid: "ed2016d3874a444fa51ea3152e78e354" member_type: VOTER last_known_addr { host: "127.1.135.194" port: 46433 } }
I20260923 22:16:15.181699 21682 raft_consensus.cc:729] T 69a66bcffb8245d7bee04bb38834afff P ed2016d3874a444fa51ea3152e78e354 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: ed2016d3874a444fa51ea3152e78e354, State: Initialized, Role: FOLLOWER
I20260923 22:16:15.181711 21683 raft_consensus.cc:374] T 69a66bcffb8245d7bee04bb38834afff P 49582794a7274034b760d63fd930d65f [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260923 22:16:15.181748 21683 raft_consensus.cc:729] T 69a66bcffb8245d7bee04bb38834afff P 49582794a7274034b760d63fd930d65f [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 49582794a7274034b760d63fd930d65f, State: Initialized, Role: FOLLOWER
I20260923 22:16:15.181807 21682 consensus_queue.cc:260] T 69a66bcffb8245d7bee04bb38834afff P ed2016d3874a444fa51ea3152e78e354 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a55563674f284fd7b34c511b095befbb" member_type: VOTER last_known_addr { host: "127.1.135.193" port: 35715 } } peers { permanent_uuid: "49582794a7274034b760d63fd930d65f" member_type: VOTER last_known_addr { host: "127.1.135.195" port: 42169 } } peers { permanent_uuid: "ed2016d3874a444fa51ea3152e78e354" member_type: VOTER last_known_addr { host: "127.1.135.194" port: 46433 } }
I20260923 22:16:15.181881 21683 consensus_queue.cc:260] T 69a66bcffb8245d7bee04bb38834afff P 49582794a7274034b760d63fd930d65f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a55563674f284fd7b34c511b095befbb" member_type: VOTER last_known_addr { host: "127.1.135.193" port: 35715 } } peers { permanent_uuid: "49582794a7274034b760d63fd930d65f" member_type: VOTER last_known_addr { host: "127.1.135.195" port: 42169 } } peers { permanent_uuid: "ed2016d3874a444fa51ea3152e78e354" member_type: VOTER last_known_addr { host: "127.1.135.194" port: 46433 } }
I20260923 22:16:15.182035 21682 ts_tablet_manager.cc:1428] T 69a66bcffb8245d7bee04bb38834afff P ed2016d3874a444fa51ea3152e78e354: Time spent starting tablet: real 0.003s user 0.003s sys 0.000s
I20260923 22:16:15.182096 21683 ts_tablet_manager.cc:1428] T 69a66bcffb8245d7bee04bb38834afff P 49582794a7274034b760d63fd930d65f: Time spent starting tablet: real 0.003s user 0.003s sys 0.000s
I20260923 22:16:15.182210 21668 heartbeater.cc:503] Master 127.1.135.254:46849 was elected leader, sending a full tablet report...
I20260923 22:16:15.183676 21537 heartbeater.cc:503] Master 127.1.135.254:46849 was elected leader, sending a full tablet report...
I20260923 22:16:15.184624 21688 tablet_bootstrap.cc:492] T 69a66bcffb8245d7bee04bb38834afff P a55563674f284fd7b34c511b095befbb: Bootstrap starting.
I20260923 22:16:15.185125 21688 tablet_bootstrap.cc:654] T 69a66bcffb8245d7bee04bb38834afff P a55563674f284fd7b34c511b095befbb: Neither blocks nor log segments found. Creating new log.
I20260923 22:16:15.185345 21688 log.cc:824] T 69a66bcffb8245d7bee04bb38834afff P a55563674f284fd7b34c511b095befbb: Log is configured to *not* fsync() on all Append() calls
I20260923 22:16:15.185900 21688 tablet_bootstrap.cc:492] T 69a66bcffb8245d7bee04bb38834afff P a55563674f284fd7b34c511b095befbb: No bootstrap required, opened a new log
I20260923 22:16:15.185949 21688 ts_tablet_manager.cc:1397] T 69a66bcffb8245d7bee04bb38834afff P a55563674f284fd7b34c511b095befbb: Time spent bootstrapping tablet: real 0.001s user 0.001s sys 0.000s
I20260923 22:16:15.187680 21688 raft_consensus.cc:348] T 69a66bcffb8245d7bee04bb38834afff P a55563674f284fd7b34c511b095befbb [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a55563674f284fd7b34c511b095befbb" member_type: VOTER last_known_addr { host: "127.1.135.193" port: 35715 } } peers { permanent_uuid: "49582794a7274034b760d63fd930d65f" member_type: VOTER last_known_addr { host: "127.1.135.195" port: 42169 } } peers { permanent_uuid: "ed2016d3874a444fa51ea3152e78e354" member_type: VOTER last_known_addr { host: "127.1.135.194" port: 46433 } }
I20260923 22:16:15.187800 21688 raft_consensus.cc:374] T 69a66bcffb8245d7bee04bb38834afff P a55563674f284fd7b34c511b095befbb [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260923 22:16:15.187837 21688 raft_consensus.cc:729] T 69a66bcffb8245d7bee04bb38834afff P a55563674f284fd7b34c511b095befbb [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: a55563674f284fd7b34c511b095befbb, State: Initialized, Role: FOLLOWER
I20260923 22:16:15.187932 21688 consensus_queue.cc:260] T 69a66bcffb8245d7bee04bb38834afff P a55563674f284fd7b34c511b095befbb [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a55563674f284fd7b34c511b095befbb" member_type: VOTER last_known_addr { host: "127.1.135.193" port: 35715 } } peers { permanent_uuid: "49582794a7274034b760d63fd930d65f" member_type: VOTER last_known_addr { host: "127.1.135.195" port: 42169 } } peers { permanent_uuid: "ed2016d3874a444fa51ea3152e78e354" member_type: VOTER last_known_addr { host: "127.1.135.194" port: 46433 } }
I20260923 22:16:15.188199 21688 ts_tablet_manager.cc:1428] T 69a66bcffb8245d7bee04bb38834afff P a55563674f284fd7b34c511b095befbb: Time spent starting tablet: real 0.002s user 0.000s sys 0.002s
I20260923 22:16:15.188220 21406 heartbeater.cc:503] Master 127.1.135.254:46849 was elected leader, sending a full tablet report...
W20260923 22:16:15.283635 21538 tablet.cc:2369] T 69a66bcffb8245d7bee04bb38834afff P ed2016d3874a444fa51ea3152e78e354: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260923 22:16:15.401717 21669 tablet.cc:2369] T 69a66bcffb8245d7bee04bb38834afff P 49582794a7274034b760d63fd930d65f: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260923 22:16:15.405354 21407 tablet.cc:2369] T 69a66bcffb8245d7bee04bb38834afff P a55563674f284fd7b34c511b095befbb: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260923 22:16:15.452775 21690 raft_consensus.cc:482] T 69a66bcffb8245d7bee04bb38834afff P a55563674f284fd7b34c511b095befbb [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260923 22:16:15.452958 21690 raft_consensus.cc:504] T 69a66bcffb8245d7bee04bb38834afff P a55563674f284fd7b34c511b095befbb [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a55563674f284fd7b34c511b095befbb" member_type: VOTER last_known_addr { host: "127.1.135.193" port: 35715 } } peers { permanent_uuid: "49582794a7274034b760d63fd930d65f" member_type: VOTER last_known_addr { host: "127.1.135.195" port: 42169 } } peers { permanent_uuid: "ed2016d3874a444fa51ea3152e78e354" member_type: VOTER last_known_addr { host: "127.1.135.194" port: 46433 } }
I20260923 22:16:15.453385 21690 leader_election.cc:288] T 69a66bcffb8245d7bee04bb38834afff P a55563674f284fd7b34c511b095befbb [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 49582794a7274034b760d63fd930d65f (127.1.135.195:42169), ed2016d3874a444fa51ea3152e78e354 (127.1.135.194:46433)
I20260923 22:16:15.456346 21679 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 22:16:15.453632 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.135.193:42107 (local address 127.1.135.194:46433)
0923 22:16:15.453677 (+ 45us) server_negotiation.cc:206] Beginning negotiation
0923 22:16:15.453679 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0923 22:16:15.453791 (+ 112us) server_negotiation.cc:407] Connection header received
0923 22:16:15.453880 (+ 89us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 22:16:15.453881 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 22:16:15.453890 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 22:16:15.453955 (+ 65us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0923 22:16:15.454145 (+ 190us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 22:16:15.454594 (+ 449us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 22:16:15.455580 (+ 986us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 22:16:15.455949 (+ 369us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 22:16:15.456031 (+ 82us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 22:16:15.456039 (+ 8us) server_negotiation.cc:1035] Waiting for connection context
0923 22:16:15.456245 (+ 206us) server_negotiation.cc:299] Negotiation successful
0923 22:16:15.456287 (+ 42us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":23}
I20260923 22:16:15.456346 21681 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 22:16:15.453649 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.135.193:36575 (local address 127.1.135.195:42169)
0923 22:16:15.453696 (+ 47us) server_negotiation.cc:206] Beginning negotiation
0923 22:16:15.453698 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0923 22:16:15.453806 (+ 108us) server_negotiation.cc:407] Connection header received
0923 22:16:15.453862 (+ 56us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 22:16:15.453863 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 22:16:15.453879 (+ 16us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 22:16:15.453922 (+ 43us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0923 22:16:15.454107 (+ 185us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 22:16:15.454596 (+ 489us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 22:16:15.455612 (+ 1016us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 22:16:15.455949 (+ 337us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 22:16:15.456038 (+ 89us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 22:16:15.456046 (+ 8us) server_negotiation.cc:1035] Waiting for connection context
0923 22:16:15.456209 (+ 163us) server_negotiation.cc:299] Negotiation successful
0923 22:16:15.456251 (+ 42us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":23}
I20260923 22:16:15.456377 21691 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 22:16:15.453578 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.135.194:46433 (local address 127.1.135.193:42107)
0923 22:16:15.453703 (+ 125us) negotiation.cc:107] Waiting for socket to connect
0923 22:16:15.453714 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0923 22:16:15.453802 (+ 88us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0923 22:16:15.453973 (+ 171us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0923 22:16:15.453974 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0923 22:16:15.453976 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0923 22:16:15.454051 (+ 75us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 22:16:15.454053 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 22:16:15.454680 (+ 627us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 22:16:15.454681 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 22:16:15.455490 (+ 809us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 22:16:15.455492 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 22:16:15.456046 (+ 554us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 22:16:15.456048 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 22:16:15.456153 (+ 105us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 22:16:15.456157 (+ 4us) client_negotiation.cc:769] Sending connection context
0923 22:16:15.456228 (+ 71us) client_negotiation.cc:240] Negotiation successful
0923 22:16:15.456265 (+ 37us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":110,"thread_start_us":60,"threads_started":1}
I20260923 22:16:15.456504 21692 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 22:16:15.453581 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.135.195:42169 (local address 127.1.135.193:36575)
0923 22:16:15.453714 (+ 133us) negotiation.cc:107] Waiting for socket to connect
0923 22:16:15.453728 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0923 22:16:15.453801 (+ 73us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0923 22:16:15.453950 (+ 149us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0923 22:16:15.453951 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0923 22:16:15.453954 (+ 3us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0923 22:16:15.454034 (+ 80us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 22:16:15.454035 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 22:16:15.454681 (+ 646us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 22:16:15.454682 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 22:16:15.455499 (+ 817us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 22:16:15.455502 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 22:16:15.456035 (+ 533us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 22:16:15.456036 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 22:16:15.456138 (+ 102us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 22:16:15.456141 (+ 3us) client_negotiation.cc:769] Sending connection context
0923 22:16:15.456184 (+ 43us) client_negotiation.cc:240] Negotiation successful
0923 22:16:15.456239 (+ 55us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":120,"mutex_wait_us":64,"thread_start_us":62,"threads_started":1}
I20260923 22:16:15.456784 21491 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "69a66bcffb8245d7bee04bb38834afff" candidate_uuid: "a55563674f284fd7b34c511b095befbb" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ed2016d3874a444fa51ea3152e78e354" is_pre_election: true
I20260923 22:16:15.456880 21622 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "69a66bcffb8245d7bee04bb38834afff" candidate_uuid: "a55563674f284fd7b34c511b095befbb" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "49582794a7274034b760d63fd930d65f" is_pre_election: true
I20260923 22:16:15.456945 21491 raft_consensus.cc:2454] T 69a66bcffb8245d7bee04bb38834afff P ed2016d3874a444fa51ea3152e78e354 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a55563674f284fd7b34c511b095befbb in term 0.
I20260923 22:16:15.457008 21622 raft_consensus.cc:2454] T 69a66bcffb8245d7bee04bb38834afff P 49582794a7274034b760d63fd930d65f [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a55563674f284fd7b34c511b095befbb in term 0.
I20260923 22:16:15.457178 21295 leader_election.cc:302] T 69a66bcffb8245d7bee04bb38834afff P a55563674f284fd7b34c511b095befbb [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: a55563674f284fd7b34c511b095befbb, ed2016d3874a444fa51ea3152e78e354; no voters:
I20260923 22:16:15.457294 21690 raft_consensus.cc:2781] T 69a66bcffb8245d7bee04bb38834afff P a55563674f284fd7b34c511b095befbb [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260923 22:16:15.457382 21690 raft_consensus.cc:482] T 69a66bcffb8245d7bee04bb38834afff P a55563674f284fd7b34c511b095befbb [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260923 22:16:15.457407 21690 raft_consensus.cc:3037] T 69a66bcffb8245d7bee04bb38834afff P a55563674f284fd7b34c511b095befbb [term 0 FOLLOWER]: Advancing to term 1
I20260923 22:16:15.458082 21690 raft_consensus.cc:504] T 69a66bcffb8245d7bee04bb38834afff P a55563674f284fd7b34c511b095befbb [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a55563674f284fd7b34c511b095befbb" member_type: VOTER last_known_addr { host: "127.1.135.193" port: 35715 } } peers { permanent_uuid: "49582794a7274034b760d63fd930d65f" member_type: VOTER last_known_addr { host: "127.1.135.195" port: 42169 } } peers { permanent_uuid: "ed2016d3874a444fa51ea3152e78e354" member_type: VOTER last_known_addr { host: "127.1.135.194" port: 46433 } }
I20260923 22:16:15.458210 21690 leader_election.cc:288] T 69a66bcffb8245d7bee04bb38834afff P a55563674f284fd7b34c511b095befbb [CANDIDATE]: Term 1 election: Requested vote from peers 49582794a7274034b760d63fd930d65f (127.1.135.195:42169), ed2016d3874a444fa51ea3152e78e354 (127.1.135.194:46433)
I20260923 22:16:15.458359 21622 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "69a66bcffb8245d7bee04bb38834afff" candidate_uuid: "a55563674f284fd7b34c511b095befbb" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "49582794a7274034b760d63fd930d65f"
I20260923 22:16:15.458360 21491 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "69a66bcffb8245d7bee04bb38834afff" candidate_uuid: "a55563674f284fd7b34c511b095befbb" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ed2016d3874a444fa51ea3152e78e354"
I20260923 22:16:15.458434 21622 raft_consensus.cc:3037] T 69a66bcffb8245d7bee04bb38834afff P 49582794a7274034b760d63fd930d65f [term 0 FOLLOWER]: Advancing to term 1
I20260923 22:16:15.458434 21491 raft_consensus.cc:3037] T 69a66bcffb8245d7bee04bb38834afff P ed2016d3874a444fa51ea3152e78e354 [term 0 FOLLOWER]: Advancing to term 1
I20260923 22:16:15.459385 21622 raft_consensus.cc:2454] T 69a66bcffb8245d7bee04bb38834afff P 49582794a7274034b760d63fd930d65f [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a55563674f284fd7b34c511b095befbb in term 1.
I20260923 22:16:15.459385 21491 raft_consensus.cc:2454] T 69a66bcffb8245d7bee04bb38834afff P ed2016d3874a444fa51ea3152e78e354 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a55563674f284fd7b34c511b095befbb in term 1.
I20260923 22:16:15.459527 21295 leader_election.cc:302] T 69a66bcffb8245d7bee04bb38834afff P a55563674f284fd7b34c511b095befbb [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: a55563674f284fd7b34c511b095befbb, ed2016d3874a444fa51ea3152e78e354; no voters:
I20260923 22:16:15.459640 21690 raft_consensus.cc:2781] T 69a66bcffb8245d7bee04bb38834afff P a55563674f284fd7b34c511b095befbb [term 1 FOLLOWER]: Leader election won for term 1
I20260923 22:16:15.459789 21690 raft_consensus.cc:686] T 69a66bcffb8245d7bee04bb38834afff P a55563674f284fd7b34c511b095befbb [term 1 LEADER]: Becoming Leader. State: Replica: a55563674f284fd7b34c511b095befbb, State: Running, Role: LEADER
I20260923 22:16:15.459897 21690 consensus_queue.cc:237] T 69a66bcffb8245d7bee04bb38834afff P a55563674f284fd7b34c511b095befbb [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a55563674f284fd7b34c511b095befbb" member_type: VOTER last_known_addr { host: "127.1.135.193" port: 35715 } } peers { permanent_uuid: "49582794a7274034b760d63fd930d65f" member_type: VOTER last_known_addr { host: "127.1.135.195" port: 42169 } } peers { permanent_uuid: "ed2016d3874a444fa51ea3152e78e354" member_type: VOTER last_known_addr { host: "127.1.135.194" port: 46433 } }
I20260923 22:16:15.460690 21217 catalog_manager.cc:5495] T 69a66bcffb8245d7bee04bb38834afff P a55563674f284fd7b34c511b095befbb reported cstate change: term changed from 0 to 1, leader changed from <none> to a55563674f284fd7b34c511b095befbb (127.1.135.193). New cstate: current_term: 1 leader_uuid: "a55563674f284fd7b34c511b095befbb" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a55563674f284fd7b34c511b095befbb" member_type: VOTER last_known_addr { host: "127.1.135.193" port: 35715 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "49582794a7274034b760d63fd930d65f" member_type: VOTER last_known_addr { host: "127.1.135.195" port: 42169 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "ed2016d3874a444fa51ea3152e78e354" member_type: VOTER last_known_addr { host: "127.1.135.194" port: 46433 } health_report { overall_health: UNKNOWN } } }
I20260923 22:16:15.531072 21678 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 22:16:15.528863 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54278 (local address 127.1.135.193:35715)
0923 22:16:15.528920 (+ 57us) server_negotiation.cc:206] Beginning negotiation
0923 22:16:15.528922 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0923 22:16:15.528935 (+ 13us) server_negotiation.cc:407] Connection header received
0923 22:16:15.528972 (+ 37us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 22:16:15.528972 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 22:16:15.528992 (+ 20us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 22:16:15.529068 (+ 76us) server_negotiation.cc:226] Negotiated authn=TOKEN
0923 22:16:15.529176 (+ 108us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 22:16:15.529650 (+ 474us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 22:16:15.530444 (+ 794us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 22:16:15.530624 (+ 180us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 22:16:15.530673 (+ 49us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 22:16:15.530812 (+ 139us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0923 22:16:15.530907 (+ 95us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0923 22:16:15.530966 (+ 59us) server_negotiation.cc:1035] Waiting for connection context
0923 22:16:15.530998 (+ 32us) server_negotiation.cc:299] Negotiation successful
0923 22:16:15.531016 (+ 18us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":31}
I20260923 22:16:15.533074 21622 raft_consensus.cc:1261] T 69a66bcffb8245d7bee04bb38834afff P 49582794a7274034b760d63fd930d65f [term 1 FOLLOWER]: Refusing update from remote peer a55563674f284fd7b34c511b095befbb: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260923 22:16:15.533074 21491 raft_consensus.cc:1261] T 69a66bcffb8245d7bee04bb38834afff P ed2016d3874a444fa51ea3152e78e354 [term 1 FOLLOWER]: Refusing update from remote peer a55563674f284fd7b34c511b095befbb: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260923 22:16:15.533500 21693 consensus_queue.cc:1035] T 69a66bcffb8245d7bee04bb38834afff P a55563674f284fd7b34c511b095befbb [LEADER]: Connected to new peer: Peer: permanent_uuid: "49582794a7274034b760d63fd930d65f" member_type: VOTER last_known_addr { host: "127.1.135.195" port: 42169 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260923 22:16:15.533710 21690 consensus_queue.cc:1035] T 69a66bcffb8245d7bee04bb38834afff P a55563674f284fd7b34c511b095befbb [LEADER]: Connected to new peer: Peer: permanent_uuid: "ed2016d3874a444fa51ea3152e78e354" member_type: VOTER last_known_addr { host: "127.1.135.194" port: 46433 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260923 22:16:15.536769 21697 mvcc.cc:205] Tried to move back new op lower bound from 7332666472580288512 to 7332666472284495872. Current Snapshot: MvccSnapshot[applied={T|T < 7332666472580288512 or (T in {7332666472580288512})}]
I20260923 22:16:15.538291 21217 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:43670:
table { table_name: "test-table" } modify_external_catalogs: true
I20260923 22:16:15.538373 21217 catalog_manager.cc:2645] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:43670:
table { table_name: "test-table" } modify_external_catalogs: true
I20260923 22:16:15.539167 21217 catalog_manager.cc:5782] T 00000000000000000000000000000000 P e18deb6b029b4f0b8dbd7f106df83cc8: Sending DeleteTablet for 3 replicas of tablet 69a66bcffb8245d7bee04bb38834afff
I20260923 22:16:15.539510 21340 tablet_service.cc:1514] Processing DeleteTablet for tablet 69a66bcffb8245d7bee04bb38834afff with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-23 22:16:15 UTC) from {username='slave'} at 127.0.0.1:54272
I20260923 22:16:15.539513 21471 tablet_service.cc:1514] Processing DeleteTablet for tablet 69a66bcffb8245d7bee04bb38834afff with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-23 22:16:15 UTC) from {username='slave'} at 127.0.0.1:33572
I20260923 22:16:15.539731 21705 tablet_replica.cc:330] T 69a66bcffb8245d7bee04bb38834afff P ed2016d3874a444fa51ea3152e78e354: stopping tablet replica
I20260923 22:16:15.539723 21602 tablet_service.cc:1514] Processing DeleteTablet for tablet 69a66bcffb8245d7bee04bb38834afff with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-23 22:16:15 UTC) from {username='slave'} at 127.0.0.1:33258
I20260923 22:16:15.539819 21705 raft_consensus.cc:2229] T 69a66bcffb8245d7bee04bb38834afff P ed2016d3874a444fa51ea3152e78e354 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260923 22:16:15.539908 21705 raft_consensus.cc:2258] T 69a66bcffb8245d7bee04bb38834afff P ed2016d3874a444fa51ea3152e78e354 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260923 22:16:15.539963 21707 tablet_replica.cc:330] T 69a66bcffb8245d7bee04bb38834afff P 49582794a7274034b760d63fd930d65f: stopping tablet replica
I20260923 22:16:15.540052 21707 raft_consensus.cc:2229] T 69a66bcffb8245d7bee04bb38834afff P 49582794a7274034b760d63fd930d65f [term 1 FOLLOWER]: Raft consensus shutting down.
I20260923 22:16:15.540279 1567 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskRiQe_a/build/release/bin/kudu with pid 21278
I20260923 22:16:15.541095 21705 ts_tablet_manager.cc:1905] T 69a66bcffb8245d7bee04bb38834afff P ed2016d3874a444fa51ea3152e78e354: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260923 22:16:15.541579 21707 raft_consensus.cc:2258] T 69a66bcffb8245d7bee04bb38834afff P 49582794a7274034b760d63fd930d65f [term 1 FOLLOWER]: Raft consensus is shut down!
I20260923 22:16:15.542086 21707 ts_tablet_manager.cc:1905] T 69a66bcffb8245d7bee04bb38834afff P 49582794a7274034b760d63fd930d65f: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260923 22:16:15.542891 21705 ts_tablet_manager.cc:1918] T 69a66bcffb8245d7bee04bb38834afff P ed2016d3874a444fa51ea3152e78e354: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260923 22:16:15.542963 21705 log.cc:1196] T 69a66bcffb8245d7bee04bb38834afff P ed2016d3874a444fa51ea3152e78e354: Deleting WAL directory at /tmp/dist-test-taskRiQe_a/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790201539513534-1567-0/minicluster-data/ts-1/wal/wals/69a66bcffb8245d7bee04bb38834afff
I20260923 22:16:15.543226 21705 ts_tablet_manager.cc:1939] T 69a66bcffb8245d7bee04bb38834afff P ed2016d3874a444fa51ea3152e78e354: Deleting consensus metadata
I20260923 22:16:15.543644 21204 catalog_manager.cc:4841] TS ed2016d3874a444fa51ea3152e78e354 (127.1.135.194:46433): tablet 69a66bcffb8245d7bee04bb38834afff (table test-table [id=ddca15567d50436dbd4be3734bbee1c4]) successfully deleted
I20260923 22:16:15.543782 21707 ts_tablet_manager.cc:1918] T 69a66bcffb8245d7bee04bb38834afff P 49582794a7274034b760d63fd930d65f: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260923 22:16:15.543856 21707 log.cc:1196] T 69a66bcffb8245d7bee04bb38834afff P 49582794a7274034b760d63fd930d65f: Deleting WAL directory at /tmp/dist-test-taskRiQe_a/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790201539513534-1567-0/minicluster-data/ts-2/wal/wals/69a66bcffb8245d7bee04bb38834afff
I20260923 22:16:15.544036 21707 ts_tablet_manager.cc:1939] T 69a66bcffb8245d7bee04bb38834afff P 49582794a7274034b760d63fd930d65f: Deleting consensus metadata
I20260923 22:16:15.544413 21202 catalog_manager.cc:4841] TS 49582794a7274034b760d63fd930d65f (127.1.135.195:42169): tablet 69a66bcffb8245d7bee04bb38834afff (table test-table [id=ddca15567d50436dbd4be3734bbee1c4]) successfully deleted
W20260923 22:16:15.545954 21202 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.1.135.193:35715 (error 108)
I20260923 22:16:15.546409 21680 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 22:16:15.546274 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.135.193:35715 (local address 127.0.0.1:54292)
0923 22:16:15.546331 (+ 57us) negotiation.cc:107] Waiting for socket to connect
0923 22:16:15.546371 (+ 40us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.135.193:35715: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":36}
I20260923 22:16:15.546547 1567 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskRiQe_a/build/release/bin/kudu with pid 21409
W20260923 22:16:15.546555 21202 catalog_manager.cc:4584] TS a55563674f284fd7b34c511b095befbb (127.1.135.193:35715): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet 69a66bcffb8245d7bee04bb38834afff: Network error: Client connection negotiation failed: client connection to 127.1.135.193:35715: connect: Connection refused (error 111)
I20260923 22:16:15.551563 1567 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskRiQe_a/build/release/bin/kudu with pid 21540
I20260923 22:16:15.555998 1567 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskRiQe_a/build/release/bin/kudu with pid 21187
2026-09-23T22:16:15Z chronyd exiting
[ OK ] EncryptionPolicyTest.TestEncryptionPolicy/5 (935 ms)
[----------] 6 tests from EncryptionPolicyTest (8259 ms total)
[----------] 18 tests from AuthTokenIssuingTest
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/0
Loading random data
Initializing database '/tmp/dist-test-taskRiQe_a/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1790201539513534-1567-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Sep 23 22:16:15 dist-test-slave-s2fj krb5kdc[21711](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Sep 23 22:16:15 dist-test-slave-s2fj krb5kdc[21711](info): set up 2 sockets
Sep 23 22:16:15 dist-test-slave-s2fj krb5kdc[21711](info): commencing operation
krb5kdc: starting...
W20260923 22:16:16.575927 1567 mini_kdc.cc:121] Time spent starting KDC: real 1.000s user 0.000s sys 0.006s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskRiQe_a/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1790201539513534-1567-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskRiQe_a/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1790201539513534-1567-0/krb5kdc/test-user.keytab.
Sep 23 22:16:16 dist-test-slave-s2fj krb5kdc[21711](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790201776, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-09-23T22:16:16Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-09-23T22:16:16Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskRiQe_a/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1790201539513534-1567-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskRiQe_a/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1790201539513534-1567-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260923 22:16:16.686470 1567 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskRiQe_a/build/release/bin/kudu
/tmp/dist-test-taskRiQe_a/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskRiQe_a/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1790201539513534-1567-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskRiQe_a/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1790201539513534-1567-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskRiQe_a/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1790201539513534-1567-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskRiQe_a/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1790201539513534-1567-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:33947
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-taskRiQe_a/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1790201539513534-1567-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:43573
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:33947
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=required
--rpc_encryption=required
--rpc_encrypt_loopback_connections=true with env {KRB5CCNAME=/tmp/dist-test-taskRiQe_a/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1790201539513534-1567-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskRiQe_a/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1790201539513534-1567-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskRiQe_a/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1790201539513534-1567-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260923 22:16:16.763553 21724 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260923 22:16:16.763787 21724 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260923 22:16:16.763820 21724 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260923 22:16:16.764900 21724 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260923 22:16:16.764971 21724 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260923 22:16:16.764993 21724 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260923 22:16:16.765012 21724 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260923 22:16:16.765041 21724 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260923 22:16:16.766413 21724 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:43573
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskRiQe_a/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1790201539513534-1567-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskRiQe_a/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1790201539513534-1567-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:33947
--rpc_encrypt_loopback_connections=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskRiQe_a/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1790201539513534-1567-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:33947
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskRiQe_a/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1790201539513534-1567-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.21724
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskRiQe_a/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1790201539513534-1567-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 23 Sep 2026 20:53:37 UTC on ccycloud.quasar-rkviyx.root.comops.site
build id 4668083
I20260923 22:16:16.766678 21724 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260923 22:16:16.766886 21724 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260923 22:16:16.771544 21729 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260923 22:16:16.771687 21732 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260923 22:16:16.771735 21730 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260923 22:16:16.771828 21724 server_base.cc:1034] running on GCE node
I20260923 22:16:16.772101 21724 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260923 22:16:16.772387 21724 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260923 22:16:16.773547 21724 hybrid_clock.cc:648] HybridClock initialized: now 1790201776773544 us; error 35 us; skew 500 ppm
Sep 23 22:16:16 dist-test-slave-s2fj krb5kdc[21711](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790201776, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260923 22:16:16.775802 21724 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260923 22:16:16.776252 21724 webserver.cc:460] Webserver started at http://127.0.0.1:40985/ using document root <none> and password file <none>
I20260923 22:16:16.776463 21724 fs_manager.cc:359] Metadata directory not provided
I20260923 22:16:16.776501 21724 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260923 22:16:16.776583 21724 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260923 22:16:16.777484 21724 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskRiQe_a/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1790201539513534-1567-0/minicluster-data/master-0/data/instance:
uuid: "33570b8e76a64c6088250efd81b37e20"
format_stamp: "Formatted at 2026-09-23 22:16:16 on dist-test-slave-s2fj"
server_key: "f35465e92858f23c4af95f85170f23bc"
server_key_iv: "44da15b9f626f0ba4c78cafa3e4097db"
server_key_version: "encryptionkey@0"
I20260923 22:16:16.777758 21724 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskRiQe_a/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1790201539513534-1567-0/minicluster-data/master-0/wal/instance:
uuid: "33570b8e76a64c6088250efd81b37e20"
format_stamp: "Formatted at 2026-09-23 22:16:16 on dist-test-slave-s2fj"
server_key: "f35465e92858f23c4af95f85170f23bc"
server_key_iv: "44da15b9f626f0ba4c78cafa3e4097db"
server_key_version: "encryptionkey@0"
I20260923 22:16:16.778946 21724 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.000s
I20260923 22:16:16.779592 21739 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260923 22:16:16.779783 21724 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260923 22:16:16.779868 21724 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskRiQe_a/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1790201539513534-1567-0/minicluster-data/master-0/data,/tmp/dist-test-taskRiQe_a/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1790201539513534-1567-0/minicluster-data/master-0/wal
uuid: "33570b8e76a64c6088250efd81b37e20"
format_stamp: "Formatted at 2026-09-23 22:16:16 on dist-test-slave-s2fj"
server_key: "f35465e92858f23c4af95f85170f23bc"
server_key_iv: "44da15b9f626f0ba4c78cafa3e4097db"
server_key_version: "encryptionkey@0"
I20260923 22:16:16.779927 21724 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskRiQe_a/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1790201539513534-1567-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskRiQe_a/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1790201539513534-1567-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskRiQe_a/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1790201539513534-1567-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260923 22:16:16.801036 21724 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260923 22:16:16.803385 21724 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260923 22:16:16.803524 21724 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260923 22:16:16.807356 21724 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:33947
I20260923 22:16:16.807390 21791 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:33947 every 8 connection(s)
I20260923 22:16:16.807708 21724 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskRiQe_a/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1790201539513534-1567-0/minicluster-data/master-0/data/info.pb
I20260923 22:16:16.808740 21792 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260923 22:16:16.810451 1567 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskRiQe_a/build/release/bin/kudu as pid 21724
I20260923 22:16:16.810539 1567 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskRiQe_a/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1790201539513534-1567-0/minicluster-data/master-0/wal/instance
I20260923 22:16:16.810745 1567 external_mini_cluster.cc:1428] Setting key d97e4fc30272d81660d375af3d250996
I20260923 22:16:16.810968 21792 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 33570b8e76a64c6088250efd81b37e20: Bootstrap starting.
I20260923 22:16:16.811661 21792 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 33570b8e76a64c6088250efd81b37e20: Neither blocks nor log segments found. Creating new log.
I20260923 22:16:16.811980 21792 log.cc:824] T 00000000000000000000000000000000 P 33570b8e76a64c6088250efd81b37e20: Log is configured to *not* fsync() on all Append() calls
I20260923 22:16:16.812713 21792 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 33570b8e76a64c6088250efd81b37e20: No bootstrap required, opened a new log
I20260923 22:16:16.814917 21792 raft_consensus.cc:348] T 00000000000000000000000000000000 P 33570b8e76a64c6088250efd81b37e20 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "33570b8e76a64c6088250efd81b37e20" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 33947 } }
I20260923 22:16:16.815068 21792 raft_consensus.cc:374] T 00000000000000000000000000000000 P 33570b8e76a64c6088250efd81b37e20 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260923 22:16:16.815104 21792 raft_consensus.cc:729] T 00000000000000000000000000000000 P 33570b8e76a64c6088250efd81b37e20 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 33570b8e76a64c6088250efd81b37e20, State: Initialized, Role: FOLLOWER
I20260923 22:16:16.815217 21792 consensus_queue.cc:260] T 00000000000000000000000000000000 P 33570b8e76a64c6088250efd81b37e20 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "33570b8e76a64c6088250efd81b37e20" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 33947 } }
I20260923 22:16:16.815286 21792 raft_consensus.cc:388] T 00000000000000000000000000000000 P 33570b8e76a64c6088250efd81b37e20 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260923 22:16:16.815313 21792 raft_consensus.cc:482] T 00000000000000000000000000000000 P 33570b8e76a64c6088250efd81b37e20 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260923 22:16:16.815356 21792 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 33570b8e76a64c6088250efd81b37e20 [term 0 FOLLOWER]: Advancing to term 1
I20260923 22:16:16.816008 21792 raft_consensus.cc:504] T 00000000000000000000000000000000 P 33570b8e76a64c6088250efd81b37e20 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "33570b8e76a64c6088250efd81b37e20" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 33947 } }
I20260923 22:16:16.816154 21792 leader_election.cc:302] T 00000000000000000000000000000000 P 33570b8e76a64c6088250efd81b37e20 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 33570b8e76a64c6088250efd81b37e20; no voters:
I20260923 22:16:16.816304 21792 leader_election.cc:288] T 00000000000000000000000000000000 P 33570b8e76a64c6088250efd81b37e20 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260923 22:16:16.816344 21797 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 33570b8e76a64c6088250efd81b37e20 [term 1 FOLLOWER]: Leader election won for term 1
I20260923 22:16:16.816481 21797 raft_consensus.cc:686] T 00000000000000000000000000000000 P 33570b8e76a64c6088250efd81b37e20 [term 1 LEADER]: Becoming Leader. State: Replica: 33570b8e76a64c6088250efd81b37e20, State: Running, Role: LEADER
I20260923 22:16:16.816545 21792 sys_catalog.cc:564] T 00000000000000000000000000000000 P 33570b8e76a64c6088250efd81b37e20 [sys.catalog]: configured and running, proceeding with master startup.
I20260923 22:16:16.816547 21797 consensus_queue.cc:237] T 00000000000000000000000000000000 P 33570b8e76a64c6088250efd81b37e20 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "33570b8e76a64c6088250efd81b37e20" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 33947 } }
I20260923 22:16:16.816892 21799 sys_catalog.cc:455] T 00000000000000000000000000000000 P 33570b8e76a64c6088250efd81b37e20 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 33570b8e76a64c6088250efd81b37e20. Latest consensus state: current_term: 1 leader_uuid: "33570b8e76a64c6088250efd81b37e20" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "33570b8e76a64c6088250efd81b37e20" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 33947 } } }
I20260923 22:16:16.816969 21799 sys_catalog.cc:458] T 00000000000000000000000000000000 P 33570b8e76a64c6088250efd81b37e20 [sys.catalog]: This master's current role is: LEADER
I20260923 22:16:16.816875 21798 sys_catalog.cc:455] T 00000000000000000000000000000000 P 33570b8e76a64c6088250efd81b37e20 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "33570b8e76a64c6088250efd81b37e20" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "33570b8e76a64c6088250efd81b37e20" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 33947 } } }
I20260923 22:16:16.817142 21798 sys_catalog.cc:458] T 00000000000000000000000000000000 P 33570b8e76a64c6088250efd81b37e20 [sys.catalog]: This master's current role is: LEADER
Sep 23 22:16:16 dist-test-slave-s2fj krb5kdc[21711](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790201776, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.0.0.1@KRBTEST.COM
W20260923 22:16:16.818197 21813 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 33570b8e76a64c6088250efd81b37e20: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260923 22:16:16.818259 21813 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260923 22:16:16.818318 21802 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260923 22:16:16.818477 21802 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260923 22:16:16.820220 21802 catalog_manager.cc:1347] Generated new cluster ID: 9b91b2a28c7f4ba39ba5e17fb928a675
I20260923 22:16:16.820271 21802 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260923 22:16:16.821949 21795 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 22:16:16.811248 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59749 (local address 127.0.0.1:33947)
0923 22:16:16.811430 (+ 182us) server_negotiation.cc:206] Beginning negotiation
0923 22:16:16.811439 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0923 22:16:16.811469 (+ 30us) server_negotiation.cc:407] Connection header received
0923 22:16:16.812078 (+ 609us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 22:16:16.812125 (+ 47us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 22:16:16.812188 (+ 63us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 22:16:16.812334 (+ 146us) server_negotiation.cc:226] Negotiated authn=SASL
0923 22:16:16.812978 (+ 644us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 22:16:16.814367 (+ 1389us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 22:16:16.815079 (+ 712us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 22:16:16.815680 (+ 601us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 22:16:16.815801 (+ 121us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 22:16:16.818564 (+ 2763us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0923 22:16:16.818567 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0923 22:16:16.818568 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0923 22:16:16.818579 (+ 11us) server_negotiation.cc:922] calling sasl_server_start()
0923 22:16:16.820501 (+ 1922us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0923 22:16:16.820880 (+ 379us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0923 22:16:16.820882 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0923 22:16:16.820883 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0923 22:16:16.820990 (+ 107us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0923 22:16:16.821214 (+ 224us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0923 22:16:16.821215 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0923 22:16:16.821216 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0923 22:16:16.821435 (+ 219us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0923 22:16:16.821616 (+ 181us) server_negotiation.cc:1035] Waiting for connection context
0923 22:16:16.821695 (+ 79us) server_negotiation.cc:299] Negotiation successful
0923 22:16:16.821795 (+ 100us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":131,"thread_start_us":84,"threads_started":1}
I20260923 22:16:16.822849 1567 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260923 22:16:16.829914 21795 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 22:16:16.824213 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55121 (local address 127.0.0.1:33947)
0923 22:16:16.824245 (+ 32us) server_negotiation.cc:206] Beginning negotiation
0923 22:16:16.824246 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0923 22:16:16.824389 (+ 143us) server_negotiation.cc:407] Connection header received
0923 22:16:16.824456 (+ 67us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 22:16:16.824456 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 22:16:16.824465 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 22:16:16.824512 (+ 47us) server_negotiation.cc:226] Negotiated authn=SASL
0923 22:16:16.824979 (+ 467us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 22:16:16.825429 (+ 450us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 22:16:16.826079 (+ 650us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 22:16:16.826311 (+ 232us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 22:16:16.826380 (+ 69us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 22:16:16.827447 (+ 1067us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0923 22:16:16.827448 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0923 22:16:16.827449 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0923 22:16:16.827455 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0923 22:16:16.828755 (+ 1300us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0923 22:16:16.829119 (+ 364us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0923 22:16:16.829120 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0923 22:16:16.829120 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0923 22:16:16.829174 (+ 54us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0923 22:16:16.829394 (+ 220us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0923 22:16:16.829395 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0923 22:16:16.829395 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0923 22:16:16.829496 (+ 101us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0923 22:16:16.829599 (+ 103us) server_negotiation.cc:1035] Waiting for connection context
0923 22:16:16.829720 (+ 121us) server_negotiation.cc:299] Negotiation successful
0923 22:16:16.829804 (+ 84us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":16}
I20260923 22:16:16.839391 21802 catalog_manager.cc:1370] Generated new certificate authority record
I20260923 22:16:16.839875 21802 catalog_manager.cc:1504] Loading token signing keys...
I20260923 22:16:16.844730 21802 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 33570b8e76a64c6088250efd81b37e20: Generated new TSK 0
I20260923 22:16:16.844895 21802 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260923 22:16:16.853770 1567 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskRiQe_a/build/release/bin/kudu with pid 21724
W20260923 22:16:16.858099 21709 connection.cc:535] client connection to 127.0.0.1:33947 recv error: Network error: failed to read from TLS socket (remote: 127.0.0.1:33947): Connection reset by peer (error 104)
2026-09-23T22:16:16Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/0 (1296 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/1
Loading random data
Initializing database '/tmp/dist-test-taskRiQe_a/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1790201539513534-1567-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Sep 23 22:16:16 dist-test-slave-s2fj krb5kdc[21823](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Sep 23 22:16:16 dist-test-slave-s2fj krb5kdc[21823](info): set up 2 sockets
Sep 23 22:16:16 dist-test-slave-s2fj krb5kdc[21823](info): commencing operation
krb5kdc: starting...
W20260923 22:16:17.879944 1567 mini_kdc.cc:121] Time spent starting KDC: real 1.008s user 0.000s sys 0.005s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskRiQe_a/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1790201539513534-1567-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskRiQe_a/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1790201539513534-1567-0/krb5kdc/test-user.keytab.
Sep 23 22:16:17 dist-test-slave-s2fj krb5kdc[21823](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790201777, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-09-23T22:16:17Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-09-23T22:16:17Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskRiQe_a/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1790201539513534-1567-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskRiQe_a/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1790201539513534-1567-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260923 22:16:17.990468 1567 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskRiQe_a/build/release/bin/kudu
/tmp/dist-test-taskRiQe_a/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskRiQe_a/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1790201539513534-1567-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskRiQe_a/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1790201539513534-1567-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskRiQe_a/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1790201539513534-1567-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskRiQe_a/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1790201539513534-1567-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:44583
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-taskRiQe_a/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1790201539513534-1567-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:33441
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:44583
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=required
--rpc_encrypt_loopback_connections=true with env {KRB5CCNAME=/tmp/dist-test-taskRiQe_a/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1790201539513534-1567-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskRiQe_a/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1790201539513534-1567-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskRiQe_a/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1790201539513534-1567-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260923 22:16:18.072921 21836 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260923 22:16:18.073167 21836 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260923 22:16:18.073206 21836 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260923 22:16:18.074363 21836 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260923 22:16:18.074432 21836 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260923 22:16:18.074457 21836 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260923 22:16:18.074486 21836 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260923 22:16:18.074514 21836 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260923 22:16:18.075992 21836 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:33441
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskRiQe_a/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1790201539513534-1567-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskRiQe_a/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1790201539513534-1567-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:44583
--rpc_encrypt_loopback_connections=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskRiQe_a/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1790201539513534-1567-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:44583
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskRiQe_a/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1790201539513534-1567-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.21836
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskRiQe_a/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1790201539513534-1567-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 23 Sep 2026 20:53:37 UTC on ccycloud.quasar-rkviyx.root.comops.site
build id 4668083
I20260923 22:16:18.076316 21836 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260923 22:16:18.076552 21836 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260923 22:16:18.081387 21844 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260923 22:16:18.081521 21836 server_base.cc:1034] running on GCE node
W20260923 22:16:18.081353 21842 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260923 22:16:18.081354 21841 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260923 22:16:18.081910 21836 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260923 22:16:18.082206 21836 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260923 22:16:18.083377 21836 hybrid_clock.cc:648] HybridClock initialized: now 1790201778083373 us; error 38 us; skew 500 ppm
Sep 23 22:16:18 dist-test-slave-s2fj krb5kdc[21823](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790201778, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260923 22:16:18.085915 21836 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260923 22:16:18.086355 21836 webserver.cc:460] Webserver started at http://127.0.0.1:41169/ using document root <none> and password file <none>
I20260923 22:16:18.086537 21836 fs_manager.cc:359] Metadata directory not provided
I20260923 22:16:18.086577 21836 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260923 22:16:18.086661 21836 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260923 22:16:18.087687 21836 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskRiQe_a/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1790201539513534-1567-0/minicluster-data/master-0/data/instance:
uuid: "7cde7db6ba264c9288bc31ced884ffcd"
format_stamp: "Formatted at 2026-09-23 22:16:18 on dist-test-slave-s2fj"
server_key: "1474143b878c86f6ef225cca4b81c362"
server_key_iv: "e448a947e6c0c90671c1c65de3393324"
server_key_version: "encryptionkey@0"
I20260923 22:16:18.087994 21836 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskRiQe_a/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1790201539513534-1567-0/minicluster-data/master-0/wal/instance:
uuid: "7cde7db6ba264c9288bc31ced884ffcd"
format_stamp: "Formatted at 2026-09-23 22:16:18 on dist-test-slave-s2fj"
server_key: "1474143b878c86f6ef225cca4b81c362"
server_key_iv: "e448a947e6c0c90671c1c65de3393324"
server_key_version: "encryptionkey@0"
I20260923 22:16:18.089425 21836 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.001s
I20260923 22:16:18.090116 21851 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260923 22:16:18.090306 21836 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.001s
I20260923 22:16:18.090399 21836 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskRiQe_a/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1790201539513534-1567-0/minicluster-data/master-0/data,/tmp/dist-test-taskRiQe_a/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1790201539513534-1567-0/minicluster-data/master-0/wal
uuid: "7cde7db6ba264c9288bc31ced884ffcd"
format_stamp: "Formatted at 2026-09-23 22:16:18 on dist-test-slave-s2fj"
server_key: "1474143b878c86f6ef225cca4b81c362"
server_key_iv: "e448a947e6c0c90671c1c65de3393324"
server_key_version: "encryptionkey@0"
I20260923 22:16:18.090457 21836 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskRiQe_a/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1790201539513534-1567-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskRiQe_a/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1790201539513534-1567-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskRiQe_a/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1790201539513534-1567-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260923 22:16:18.104194 21836 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260923 22:16:18.106173 21836 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260923 22:16:18.106292 21836 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260923 22:16:18.110203 21836 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:44583
I20260923 22:16:18.110282 21903 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:44583 every 8 connection(s)
I20260923 22:16:18.110610 21836 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskRiQe_a/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1790201539513534-1567-0/minicluster-data/master-0/data/info.pb
I20260923 22:16:18.111627 21904 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260923 22:16:18.113888 21904 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 7cde7db6ba264c9288bc31ced884ffcd: Bootstrap starting.
I20260923 22:16:18.114426 21904 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 7cde7db6ba264c9288bc31ced884ffcd: Neither blocks nor log segments found. Creating new log.
I20260923 22:16:18.114651 21904 log.cc:824] T 00000000000000000000000000000000 P 7cde7db6ba264c9288bc31ced884ffcd: Log is configured to *not* fsync() on all Append() calls
I20260923 22:16:18.114786 1567 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskRiQe_a/build/release/bin/kudu as pid 21836
I20260923 22:16:18.114918 1567 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskRiQe_a/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1790201539513534-1567-0/minicluster-data/master-0/wal/instance
I20260923 22:16:18.115135 1567 external_mini_cluster.cc:1428] Setting key 3e5e3e11ada6acdcc50876e061abe948
I20260923 22:16:18.115473 21904 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 7cde7db6ba264c9288bc31ced884ffcd: No bootstrap required, opened a new log
I20260923 22:16:18.117180 21904 raft_consensus.cc:348] T 00000000000000000000000000000000 P 7cde7db6ba264c9288bc31ced884ffcd [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "7cde7db6ba264c9288bc31ced884ffcd" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 44583 } }
I20260923 22:16:18.117349 21904 raft_consensus.cc:374] T 00000000000000000000000000000000 P 7cde7db6ba264c9288bc31ced884ffcd [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260923 22:16:18.117394 21904 raft_consensus.cc:729] T 00000000000000000000000000000000 P 7cde7db6ba264c9288bc31ced884ffcd [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 7cde7db6ba264c9288bc31ced884ffcd, State: Initialized, Role: FOLLOWER
I20260923 22:16:18.117532 21904 consensus_queue.cc:260] T 00000000000000000000000000000000 P 7cde7db6ba264c9288bc31ced884ffcd [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "7cde7db6ba264c9288bc31ced884ffcd" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 44583 } }
I20260923 22:16:18.117626 21904 raft_consensus.cc:388] T 00000000000000000000000000000000 P 7cde7db6ba264c9288bc31ced884ffcd [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260923 22:16:18.117669 21904 raft_consensus.cc:482] T 00000000000000000000000000000000 P 7cde7db6ba264c9288bc31ced884ffcd [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260923 22:16:18.117718 21904 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 7cde7db6ba264c9288bc31ced884ffcd [term 0 FOLLOWER]: Advancing to term 1
I20260923 22:16:18.118252 21904 raft_consensus.cc:504] T 00000000000000000000000000000000 P 7cde7db6ba264c9288bc31ced884ffcd [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "7cde7db6ba264c9288bc31ced884ffcd" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 44583 } }
I20260923 22:16:18.118377 21904 leader_election.cc:302] T 00000000000000000000000000000000 P 7cde7db6ba264c9288bc31ced884ffcd [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 7cde7db6ba264c9288bc31ced884ffcd; no voters:
I20260923 22:16:18.118590 21904 leader_election.cc:288] T 00000000000000000000000000000000 P 7cde7db6ba264c9288bc31ced884ffcd [CANDIDATE]: Term 1 election: Requested vote from peers
I20260923 22:16:18.118614 21909 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 7cde7db6ba264c9288bc31ced884ffcd [term 1 FOLLOWER]: Leader election won for term 1
I20260923 22:16:18.118727 21909 raft_consensus.cc:686] T 00000000000000000000000000000000 P 7cde7db6ba264c9288bc31ced884ffcd [term 1 LEADER]: Becoming Leader. State: Replica: 7cde7db6ba264c9288bc31ced884ffcd, State: Running, Role: LEADER
I20260923 22:16:18.118816 21909 consensus_queue.cc:237] T 00000000000000000000000000000000 P 7cde7db6ba264c9288bc31ced884ffcd [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "7cde7db6ba264c9288bc31ced884ffcd" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 44583 } }
I20260923 22:16:18.118847 21904 sys_catalog.cc:564] T 00000000000000000000000000000000 P 7cde7db6ba264c9288bc31ced884ffcd [sys.catalog]: configured and running, proceeding with master startup.
I20260923 22:16:18.119128 21909 sys_catalog.cc:455] T 00000000000000000000000000000000 P 7cde7db6ba264c9288bc31ced884ffcd [sys.catalog]: SysCatalogTable state changed. Reason: New leader 7cde7db6ba264c9288bc31ced884ffcd. Latest consensus state: current_term: 1 leader_uuid: "7cde7db6ba264c9288bc31ced884ffcd" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "7cde7db6ba264c9288bc31ced884ffcd" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 44583 } } }
I20260923 22:16:18.119215 21909 sys_catalog.cc:458] T 00000000000000000000000000000000 P 7cde7db6ba264c9288bc31ced884ffcd [sys.catalog]: This master's current role is: LEADER
I20260923 22:16:18.119171 21910 sys_catalog.cc:455] T 00000000000000000000000000000000 P 7cde7db6ba264c9288bc31ced884ffcd [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "7cde7db6ba264c9288bc31ced884ffcd" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "7cde7db6ba264c9288bc31ced884ffcd" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 44583 } } }
I20260923 22:16:18.119359 21910 sys_catalog.cc:458] T 00000000000000000000000000000000 P 7cde7db6ba264c9288bc31ced884ffcd [sys.catalog]: This master's current role is: LEADER
I20260923 22:16:18.119896 21913 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260923 22:16:18.120414 21913 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260923 22:16:18.122093 21913 catalog_manager.cc:1347] Generated new cluster ID: 3e08bdf5586c48c8b5a15cee4bd17a0e
I20260923 22:16:18.122179 21913 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260923 22:16:18.123224 21908 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 22:16:18.115672 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57741 (local address 127.0.0.1:44583)
0923 22:16:18.115878 (+ 206us) server_negotiation.cc:206] Beginning negotiation
0923 22:16:18.115889 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0923 22:16:18.115929 (+ 40us) server_negotiation.cc:407] Connection header received
0923 22:16:18.117168 (+ 1239us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 22:16:18.117176 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 22:16:18.117226 (+ 50us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 22:16:18.117377 (+ 151us) server_negotiation.cc:226] Negotiated authn=SASL
0923 22:16:18.117591 (+ 214us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 22:16:18.119776 (+ 2185us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 22:16:18.120892 (+ 1116us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 22:16:18.121499 (+ 607us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 22:16:18.121615 (+ 116us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 22:16:18.122508 (+ 893us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0923 22:16:18.122512 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0923 22:16:18.122513 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0923 22:16:18.122718 (+ 205us) server_negotiation.cc:922] calling sasl_server_start()
0923 22:16:18.122757 (+ 39us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0923 22:16:18.122783 (+ 26us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0923 22:16:18.122872 (+ 89us) server_negotiation.cc:1035] Waiting for connection context
0923 22:16:18.122981 (+ 109us) server_negotiation.cc:299] Negotiation successful
0923 22:16:18.123009 (+ 28us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":148,"thread_start_us":98,"threads_started":1}
I20260923 22:16:18.124292 1567 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260923 22:16:18.129386 21908 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 22:16:18.125850 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52945 (local address 127.0.0.1:44583)
0923 22:16:18.125900 (+ 50us) server_negotiation.cc:206] Beginning negotiation
0923 22:16:18.125902 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0923 22:16:18.125981 (+ 79us) server_negotiation.cc:407] Connection header received
0923 22:16:18.126005 (+ 24us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 22:16:18.126006 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 22:16:18.126016 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 22:16:18.126082 (+ 66us) server_negotiation.cc:226] Negotiated authn=SASL
0923 22:16:18.126204 (+ 122us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 22:16:18.126945 (+ 741us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 22:16:18.127950 (+ 1005us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 22:16:18.128311 (+ 361us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 22:16:18.128388 (+ 77us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 22:16:18.129121 (+ 733us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0923 22:16:18.129123 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0923 22:16:18.129123 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0923 22:16:18.129128 (+ 5us) server_negotiation.cc:922] calling sasl_server_start()
0923 22:16:18.129145 (+ 17us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0923 22:16:18.129151 (+ 6us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0923 22:16:18.129220 (+ 69us) server_negotiation.cc:1035] Waiting for connection context
0923 22:16:18.129295 (+ 75us) server_negotiation.cc:299] Negotiation successful
0923 22:16:18.129308 (+ 13us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":24}
I20260923 22:16:18.133013 21913 catalog_manager.cc:1370] Generated new certificate authority record
I20260923 22:16:18.133723 21913 catalog_manager.cc:1504] Loading token signing keys...
I20260923 22:16:18.140208 21913 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 7cde7db6ba264c9288bc31ced884ffcd: Generated new TSK 0
I20260923 22:16:18.140419 21913 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260923 22:16:18.147403 1567 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskRiQe_a/build/release/bin/kudu with pid 21836
W20260923 22:16:18.151520 21821 connection.cc:535] client connection to 127.0.0.1:44583 recv error: Network error: failed to read from TLS socket (remote: 127.0.0.1:44583): Connection reset by peer (error 104)
2026-09-23T22:16:18Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/1 (1293 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/2
Loading random data
Initializing database '/tmp/dist-test-taskRiQe_a/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1790201539513534-1567-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Sep 23 22:16:18 dist-test-slave-s2fj krb5kdc[21935](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Sep 23 22:16:18 dist-test-slave-s2fj krb5kdc[21935](info): set up 2 sockets
Sep 23 22:16:18 dist-test-slave-s2fj krb5kdc[21935](info): commencing operation
krb5kdc: starting...
W20260923 22:16:19.172197 1567 mini_kdc.cc:121] Time spent starting KDC: real 1.007s user 0.000s sys 0.006s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskRiQe_a/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1790201539513534-1567-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskRiQe_a/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1790201539513534-1567-0/krb5kdc/test-user.keytab.
Sep 23 22:16:19 dist-test-slave-s2fj krb5kdc[21935](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790201779, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-09-23T22:16:19Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-09-23T22:16:19Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskRiQe_a/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1790201539513534-1567-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskRiQe_a/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1790201539513534-1567-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260923 22:16:19.285094 1567 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskRiQe_a/build/release/bin/kudu
/tmp/dist-test-taskRiQe_a/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskRiQe_a/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1790201539513534-1567-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskRiQe_a/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1790201539513534-1567-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskRiQe_a/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1790201539513534-1567-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskRiQe_a/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1790201539513534-1567-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:33669
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-taskRiQe_a/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1790201539513534-1567-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:36177
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:33669
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=disabled
--rpc_encrypt_loopback_connections=true with env {KRB5CCNAME=/tmp/dist-test-taskRiQe_a/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1790201539513534-1567-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskRiQe_a/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1790201539513534-1567-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskRiQe_a/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1790201539513534-1567-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260923 22:16:19.362615 21948 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260923 22:16:19.362836 21948 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260923 22:16:19.362875 21948 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260923 22:16:19.363971 21948 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260923 22:16:19.364037 21948 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260923 22:16:19.364058 21948 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260923 22:16:19.364117 21948 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260923 22:16:19.364157 21948 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260923 22:16:19.365588 21948 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:36177
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskRiQe_a/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1790201539513534-1567-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskRiQe_a/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1790201539513534-1567-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:33669
--rpc_encrypt_loopback_connections=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskRiQe_a/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1790201539513534-1567-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:33669
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskRiQe_a/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1790201539513534-1567-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.21948
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskRiQe_a/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1790201539513534-1567-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 23 Sep 2026 20:53:37 UTC on ccycloud.quasar-rkviyx.root.comops.site
build id 4668083
I20260923 22:16:19.365836 21948 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260923 22:16:19.366065 21948 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260923 22:16:19.370563 21954 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260923 22:16:19.370592 21956 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260923 22:16:19.370769 21948 server_base.cc:1034] running on GCE node
W20260923 22:16:19.370896 21953 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260923 22:16:19.371129 21948 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260923 22:16:19.371446 21948 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260923 22:16:19.372596 21948 hybrid_clock.cc:648] HybridClock initialized: now 1790201779372594 us; error 32 us; skew 500 ppm
Sep 23 22:16:19 dist-test-slave-s2fj krb5kdc[21935](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790201779, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260923 22:16:19.374850 21948 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260923 22:16:19.375240 21948 webserver.cc:460] Webserver started at http://127.0.0.1:45083/ using document root <none> and password file <none>
I20260923 22:16:19.375478 21948 fs_manager.cc:359] Metadata directory not provided
I20260923 22:16:19.375546 21948 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260923 22:16:19.375689 21948 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260923 22:16:19.376694 21948 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskRiQe_a/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1790201539513534-1567-0/minicluster-data/master-0/data/instance:
uuid: "787be349daf5424a9e6c1c56b08120ca"
format_stamp: "Formatted at 2026-09-23 22:16:19 on dist-test-slave-s2fj"
server_key: "1a93aeba76c105cb7dbabf43ab49a2b1"
server_key_iv: "8171c600428510cc8867cc46a19e7d49"
server_key_version: "encryptionkey@0"
I20260923 22:16:19.377029 21948 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskRiQe_a/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1790201539513534-1567-0/minicluster-data/master-0/wal/instance:
uuid: "787be349daf5424a9e6c1c56b08120ca"
format_stamp: "Formatted at 2026-09-23 22:16:19 on dist-test-slave-s2fj"
server_key: "1a93aeba76c105cb7dbabf43ab49a2b1"
server_key_iv: "8171c600428510cc8867cc46a19e7d49"
server_key_version: "encryptionkey@0"
I20260923 22:16:19.378324 21948 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.000s
I20260923 22:16:19.378991 21963 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260923 22:16:19.379155 21948 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260923 22:16:19.379248 21948 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskRiQe_a/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1790201539513534-1567-0/minicluster-data/master-0/data,/tmp/dist-test-taskRiQe_a/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1790201539513534-1567-0/minicluster-data/master-0/wal
uuid: "787be349daf5424a9e6c1c56b08120ca"
format_stamp: "Formatted at 2026-09-23 22:16:19 on dist-test-slave-s2fj"
server_key: "1a93aeba76c105cb7dbabf43ab49a2b1"
server_key_iv: "8171c600428510cc8867cc46a19e7d49"
server_key_version: "encryptionkey@0"
I20260923 22:16:19.379333 21948 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskRiQe_a/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1790201539513534-1567-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskRiQe_a/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1790201539513534-1567-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskRiQe_a/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1790201539513534-1567-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260923 22:16:19.382530 21948 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260923 22:16:19.384323 21948 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260923 22:16:19.384466 21948 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260923 22:16:19.388101 21948 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:33669
I20260923 22:16:19.388139 22015 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:33669 every 8 connection(s)
I20260923 22:16:19.388441 21948 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskRiQe_a/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1790201539513534-1567-0/minicluster-data/master-0/data/info.pb
I20260923 22:16:19.389215 1567 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskRiQe_a/build/release/bin/kudu as pid 21948
I20260923 22:16:19.389341 22016 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260923 22:16:19.389358 1567 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskRiQe_a/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1790201539513534-1567-0/minicluster-data/master-0/wal/instance
I20260923 22:16:19.389537 1567 external_mini_cluster.cc:1428] Setting key 30b984905ceb2fe1579095698163889b
I20260923 22:16:19.391978 22019 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 22:16:19.390040 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50551 (local address 127.0.0.1:33669)
0923 22:16:19.390249 (+ 209us) server_negotiation.cc:206] Beginning negotiation
0923 22:16:19.390265 (+ 16us) server_negotiation.cc:399] Waiting for connection header
0923 22:16:19.390296 (+ 31us) server_negotiation.cc:407] Connection header received
0923 22:16:19.390787 (+ 491us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 22:16:19.390795 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 22:16:19.390828 (+ 33us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 22:16:19.390974 (+ 146us) server_negotiation.cc:226] Negotiated authn=SASL
0923 22:16:19.391563 (+ 589us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0923 22:16:19.391567 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0923 22:16:19.391567 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0923 22:16:19.391571 (+ 4us) server_negotiation.cc:922] calling sasl_server_start()
0923 22:16:19.391610 (+ 39us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0923 22:16:19.391622 (+ 12us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0923 22:16:19.391710 (+ 88us) server_negotiation.cc:1035] Waiting for connection context
0923 22:16:19.391768 (+ 58us) server_negotiation.cc:299] Negotiation successful
0923 22:16:19.391805 (+ 37us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":159,"thread_start_us":102,"threads_started":1}
I20260923 22:16:19.392684 22016 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 787be349daf5424a9e6c1c56b08120ca: Bootstrap starting.
I20260923 22:16:19.392961 1567 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260923 22:16:19.393365 22016 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 787be349daf5424a9e6c1c56b08120ca: Neither blocks nor log segments found. Creating new log.
I20260923 22:16:19.393667 22016 log.cc:824] T 00000000000000000000000000000000 P 787be349daf5424a9e6c1c56b08120ca: Log is configured to *not* fsync() on all Append() calls
I20260923 22:16:19.394318 22016 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 787be349daf5424a9e6c1c56b08120ca: No bootstrap required, opened a new log
I20260923 22:16:19.395488 22019 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 22:16:19.394495 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40015 (local address 127.0.0.1:33669)
0923 22:16:19.394530 (+ 35us) server_negotiation.cc:206] Beginning negotiation
0923 22:16:19.394532 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0923 22:16:19.394651 (+ 119us) server_negotiation.cc:407] Connection header received
0923 22:16:19.394672 (+ 21us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 22:16:19.394673 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 22:16:19.394677 (+ 4us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 22:16:19.394743 (+ 66us) server_negotiation.cc:226] Negotiated authn=SASL
0923 22:16:19.395290 (+ 547us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0923 22:16:19.395291 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0923 22:16:19.395291 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0923 22:16:19.395295 (+ 4us) server_negotiation.cc:922] calling sasl_server_start()
0923 22:16:19.395307 (+ 12us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0923 22:16:19.395311 (+ 4us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0923 22:16:19.395380 (+ 69us) server_negotiation.cc:1035] Waiting for connection context
0923 22:16:19.395415 (+ 35us) server_negotiation.cc:299] Negotiation successful
0923 22:16:19.395426 (+ 11us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":19}
I20260923 22:16:19.396570 22016 raft_consensus.cc:348] T 00000000000000000000000000000000 P 787be349daf5424a9e6c1c56b08120ca [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "787be349daf5424a9e6c1c56b08120ca" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 33669 } }
I20260923 22:16:19.396692 22016 raft_consensus.cc:374] T 00000000000000000000000000000000 P 787be349daf5424a9e6c1c56b08120ca [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260923 22:16:19.396713 22016 raft_consensus.cc:729] T 00000000000000000000000000000000 P 787be349daf5424a9e6c1c56b08120ca [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 787be349daf5424a9e6c1c56b08120ca, State: Initialized, Role: FOLLOWER
I20260923 22:16:19.396792 22016 consensus_queue.cc:260] T 00000000000000000000000000000000 P 787be349daf5424a9e6c1c56b08120ca [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "787be349daf5424a9e6c1c56b08120ca" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 33669 } }
I20260923 22:16:19.396849 22016 raft_consensus.cc:388] T 00000000000000000000000000000000 P 787be349daf5424a9e6c1c56b08120ca [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260923 22:16:19.396870 22016 raft_consensus.cc:482] T 00000000000000000000000000000000 P 787be349daf5424a9e6c1c56b08120ca [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260923 22:16:19.396895 22016 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 787be349daf5424a9e6c1c56b08120ca [term 0 FOLLOWER]: Advancing to term 1
I20260923 22:16:19.397411 22016 raft_consensus.cc:504] T 00000000000000000000000000000000 P 787be349daf5424a9e6c1c56b08120ca [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "787be349daf5424a9e6c1c56b08120ca" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 33669 } }
I20260923 22:16:19.397493 22016 leader_election.cc:302] T 00000000000000000000000000000000 P 787be349daf5424a9e6c1c56b08120ca [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 787be349daf5424a9e6c1c56b08120ca; no voters:
I20260923 22:16:19.397609 22016 leader_election.cc:288] T 00000000000000000000000000000000 P 787be349daf5424a9e6c1c56b08120ca [CANDIDATE]: Term 1 election: Requested vote from peers
I20260923 22:16:19.397684 22026 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 787be349daf5424a9e6c1c56b08120ca [term 1 FOLLOWER]: Leader election won for term 1
I20260923 22:16:19.397797 22016 sys_catalog.cc:564] T 00000000000000000000000000000000 P 787be349daf5424a9e6c1c56b08120ca [sys.catalog]: configured and running, proceeding with master startup.
I20260923 22:16:19.397840 22026 raft_consensus.cc:686] T 00000000000000000000000000000000 P 787be349daf5424a9e6c1c56b08120ca [term 1 LEADER]: Becoming Leader. State: Replica: 787be349daf5424a9e6c1c56b08120ca, State: Running, Role: LEADER
I20260923 22:16:19.397921 22026 consensus_queue.cc:237] T 00000000000000000000000000000000 P 787be349daf5424a9e6c1c56b08120ca [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "787be349daf5424a9e6c1c56b08120ca" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 33669 } }
I20260923 22:16:19.398224 22027 sys_catalog.cc:455] T 00000000000000000000000000000000 P 787be349daf5424a9e6c1c56b08120ca [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "787be349daf5424a9e6c1c56b08120ca" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "787be349daf5424a9e6c1c56b08120ca" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 33669 } } }
I20260923 22:16:19.398221 22028 sys_catalog.cc:455] T 00000000000000000000000000000000 P 787be349daf5424a9e6c1c56b08120ca [sys.catalog]: SysCatalogTable state changed. Reason: New leader 787be349daf5424a9e6c1c56b08120ca. Latest consensus state: current_term: 1 leader_uuid: "787be349daf5424a9e6c1c56b08120ca" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "787be349daf5424a9e6c1c56b08120ca" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 33669 } } }
I20260923 22:16:19.398362 22027 sys_catalog.cc:458] T 00000000000000000000000000000000 P 787be349daf5424a9e6c1c56b08120ca [sys.catalog]: This master's current role is: LEADER
I20260923 22:16:19.398365 22028 sys_catalog.cc:458] T 00000000000000000000000000000000 P 787be349daf5424a9e6c1c56b08120ca [sys.catalog]: This master's current role is: LEADER
I20260923 22:16:19.398583 22035 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260923 22:16:19.399073 22035 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260923 22:16:19.400494 22035 catalog_manager.cc:1347] Generated new cluster ID: e354253a5cd04bbf9defbdb539af5030
I20260923 22:16:19.400547 22035 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260923 22:16:19.413754 22035 catalog_manager.cc:1370] Generated new certificate authority record
I20260923 22:16:19.413938 22035 catalog_manager.cc:1504] Loading token signing keys...
I20260923 22:16:19.422451 22035 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 787be349daf5424a9e6c1c56b08120ca: Generated new TSK 0
I20260923 22:16:19.422640 22035 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260923 22:16:19.425369 1567 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskRiQe_a/build/release/bin/kudu with pid 21948
2026-09-23T22:16:19Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/2 (1277 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/3
Loading random data
Initializing database '/tmp/dist-test-taskRiQe_a/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1790201539513534-1567-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Sep 23 22:16:19 dist-test-slave-s2fj krb5kdc[22047](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Sep 23 22:16:19 dist-test-slave-s2fj krb5kdc[22047](info): set up 2 sockets
Sep 23 22:16:19 dist-test-slave-s2fj krb5kdc[22047](info): commencing operation
krb5kdc: starting...
W20260923 22:16:20.456139 1567 mini_kdc.cc:121] Time spent starting KDC: real 1.013s user 0.003s sys 0.002s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskRiQe_a/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1790201539513534-1567-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskRiQe_a/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1790201539513534-1567-0/krb5kdc/test-user.keytab.
Sep 23 22:16:20 dist-test-slave-s2fj krb5kdc[22047](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790201780, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-09-23T22:16:20Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-09-23T22:16:20Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskRiQe_a/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1790201539513534-1567-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskRiQe_a/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1790201539513534-1567-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260923 22:16:20.568357 1567 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskRiQe_a/build/release/bin/kudu
/tmp/dist-test-taskRiQe_a/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskRiQe_a/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1790201539513534-1567-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskRiQe_a/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1790201539513534-1567-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskRiQe_a/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1790201539513534-1567-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskRiQe_a/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1790201539513534-1567-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:40203
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-taskRiQe_a/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1790201539513534-1567-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:45103
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:40203
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=required
--rpc_encryption=required
--rpc_encrypt_loopback_connections=false with env {KRB5CCNAME=/tmp/dist-test-taskRiQe_a/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1790201539513534-1567-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskRiQe_a/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1790201539513534-1567-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskRiQe_a/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1790201539513534-1567-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260923 22:16:20.650312 22060 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260923 22:16:20.650579 22060 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260923 22:16:20.650620 22060 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260923 22:16:20.651799 22060 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260923 22:16:20.651872 22060 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260923 22:16:20.651895 22060 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260923 22:16:20.651916 22060 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260923 22:16:20.651943 22060 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260923 22:16:20.653489 22060 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:45103
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskRiQe_a/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1790201539513534-1567-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskRiQe_a/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1790201539513534-1567-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:40203
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskRiQe_a/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1790201539513534-1567-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:40203
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskRiQe_a/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1790201539513534-1567-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.22060
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskRiQe_a/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1790201539513534-1567-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 23 Sep 2026 20:53:37 UTC on ccycloud.quasar-rkviyx.root.comops.site
build id 4668083
I20260923 22:16:20.653779 22060 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260923 22:16:20.654008 22060 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260923 22:16:20.659281 22068 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260923 22:16:20.659291 22066 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260923 22:16:20.659418 22060 server_base.cc:1034] running on GCE node
W20260923 22:16:20.659488 22065 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260923 22:16:20.659755 22060 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260923 22:16:20.660148 22060 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260923 22:16:20.661314 22060 hybrid_clock.cc:648] HybridClock initialized: now 1790201780661297 us; error 28 us; skew 500 ppm
Sep 23 22:16:20 dist-test-slave-s2fj krb5kdc[22047](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790201780, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260923 22:16:20.663906 22060 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260923 22:16:20.664422 22060 webserver.cc:460] Webserver started at http://127.0.0.1:34029/ using document root <none> and password file <none>
I20260923 22:16:20.664660 22060 fs_manager.cc:359] Metadata directory not provided
I20260923 22:16:20.664731 22060 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260923 22:16:20.664896 22060 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260923 22:16:20.665980 22060 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskRiQe_a/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1790201539513534-1567-0/minicluster-data/master-0/data/instance:
uuid: "191099d3f7ff4a30b02f1b2eb5f2d218"
format_stamp: "Formatted at 2026-09-23 22:16:20 on dist-test-slave-s2fj"
server_key: "26b0224e43558cb8bce54e3efeeb7f1d"
server_key_iv: "597ae2a9f7293f4462500323cb953bfc"
server_key_version: "encryptionkey@0"
I20260923 22:16:20.666385 22060 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskRiQe_a/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1790201539513534-1567-0/minicluster-data/master-0/wal/instance:
uuid: "191099d3f7ff4a30b02f1b2eb5f2d218"
format_stamp: "Formatted at 2026-09-23 22:16:20 on dist-test-slave-s2fj"
server_key: "26b0224e43558cb8bce54e3efeeb7f1d"
server_key_iv: "597ae2a9f7293f4462500323cb953bfc"
server_key_version: "encryptionkey@0"
I20260923 22:16:20.667695 22060 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.001s
I20260923 22:16:20.668452 22075 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260923 22:16:20.668618 22060 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.000s
I20260923 22:16:20.668721 22060 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskRiQe_a/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1790201539513534-1567-0/minicluster-data/master-0/data,/tmp/dist-test-taskRiQe_a/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1790201539513534-1567-0/minicluster-data/master-0/wal
uuid: "191099d3f7ff4a30b02f1b2eb5f2d218"
format_stamp: "Formatted at 2026-09-23 22:16:20 on dist-test-slave-s2fj"
server_key: "26b0224e43558cb8bce54e3efeeb7f1d"
server_key_iv: "597ae2a9f7293f4462500323cb953bfc"
server_key_version: "encryptionkey@0"
I20260923 22:16:20.668799 22060 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskRiQe_a/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1790201539513534-1567-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskRiQe_a/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1790201539513534-1567-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskRiQe_a/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1790201539513534-1567-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260923 22:16:20.683729 22060 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260923 22:16:20.686188 22060 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260923 22:16:20.686369 22060 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260923 22:16:20.690436 22060 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:40203
I20260923 22:16:20.690470 22127 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:40203 every 8 connection(s)
I20260923 22:16:20.690840 22060 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskRiQe_a/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1790201539513534-1567-0/minicluster-data/master-0/data/info.pb
I20260923 22:16:20.691870 22128 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260923 22:16:20.693300 1567 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskRiQe_a/build/release/bin/kudu as pid 22060
I20260923 22:16:20.693393 1567 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskRiQe_a/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1790201539513534-1567-0/minicluster-data/master-0/wal/instance
I20260923 22:16:20.693600 1567 external_mini_cluster.cc:1428] Setting key 0c9a0864697fa69296cf6414d4c15537
I20260923 22:16:20.694746 22128 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 191099d3f7ff4a30b02f1b2eb5f2d218: Bootstrap starting.
I20260923 22:16:20.695571 22128 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 191099d3f7ff4a30b02f1b2eb5f2d218: Neither blocks nor log segments found. Creating new log.
I20260923 22:16:20.695912 22128 log.cc:824] T 00000000000000000000000000000000 P 191099d3f7ff4a30b02f1b2eb5f2d218: Log is configured to *not* fsync() on all Append() calls
I20260923 22:16:20.696722 22128 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 191099d3f7ff4a30b02f1b2eb5f2d218: No bootstrap required, opened a new log
I20260923 22:16:20.698997 22128 raft_consensus.cc:348] T 00000000000000000000000000000000 P 191099d3f7ff4a30b02f1b2eb5f2d218 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "191099d3f7ff4a30b02f1b2eb5f2d218" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 40203 } }
I20260923 22:16:20.699178 22128 raft_consensus.cc:374] T 00000000000000000000000000000000 P 191099d3f7ff4a30b02f1b2eb5f2d218 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260923 22:16:20.699223 22128 raft_consensus.cc:729] T 00000000000000000000000000000000 P 191099d3f7ff4a30b02f1b2eb5f2d218 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 191099d3f7ff4a30b02f1b2eb5f2d218, State: Initialized, Role: FOLLOWER
I20260923 22:16:20.699375 22128 consensus_queue.cc:260] T 00000000000000000000000000000000 P 191099d3f7ff4a30b02f1b2eb5f2d218 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "191099d3f7ff4a30b02f1b2eb5f2d218" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 40203 } }
I20260923 22:16:20.699470 22128 raft_consensus.cc:388] T 00000000000000000000000000000000 P 191099d3f7ff4a30b02f1b2eb5f2d218 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260923 22:16:20.699517 22128 raft_consensus.cc:482] T 00000000000000000000000000000000 P 191099d3f7ff4a30b02f1b2eb5f2d218 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260923 22:16:20.699570 22128 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 191099d3f7ff4a30b02f1b2eb5f2d218 [term 0 FOLLOWER]: Advancing to term 1
I20260923 22:16:20.700199 22128 raft_consensus.cc:504] T 00000000000000000000000000000000 P 191099d3f7ff4a30b02f1b2eb5f2d218 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "191099d3f7ff4a30b02f1b2eb5f2d218" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 40203 } }
I20260923 22:16:20.700331 22128 leader_election.cc:302] T 00000000000000000000000000000000 P 191099d3f7ff4a30b02f1b2eb5f2d218 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 191099d3f7ff4a30b02f1b2eb5f2d218; no voters:
I20260923 22:16:20.700526 22128 leader_election.cc:288] T 00000000000000000000000000000000 P 191099d3f7ff4a30b02f1b2eb5f2d218 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260923 22:16:20.700584 22133 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 191099d3f7ff4a30b02f1b2eb5f2d218 [term 1 FOLLOWER]: Leader election won for term 1
I20260923 22:16:20.700783 22133 raft_consensus.cc:686] T 00000000000000000000000000000000 P 191099d3f7ff4a30b02f1b2eb5f2d218 [term 1 LEADER]: Becoming Leader. State: Replica: 191099d3f7ff4a30b02f1b2eb5f2d218, State: Running, Role: LEADER
I20260923 22:16:20.700793 22128 sys_catalog.cc:564] T 00000000000000000000000000000000 P 191099d3f7ff4a30b02f1b2eb5f2d218 [sys.catalog]: configured and running, proceeding with master startup.
Sep 23 22:16:20 dist-test-slave-s2fj krb5kdc[22047](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790201780, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.0.0.1@KRBTEST.COM
I20260923 22:16:20.701490 22133 consensus_queue.cc:237] T 00000000000000000000000000000000 P 191099d3f7ff4a30b02f1b2eb5f2d218 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "191099d3f7ff4a30b02f1b2eb5f2d218" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 40203 } }
I20260923 22:16:20.702175 22134 sys_catalog.cc:455] T 00000000000000000000000000000000 P 191099d3f7ff4a30b02f1b2eb5f2d218 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 191099d3f7ff4a30b02f1b2eb5f2d218. Latest consensus state: current_term: 1 leader_uuid: "191099d3f7ff4a30b02f1b2eb5f2d218" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "191099d3f7ff4a30b02f1b2eb5f2d218" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 40203 } } }
I20260923 22:16:20.702279 22134 sys_catalog.cc:458] T 00000000000000000000000000000000 P 191099d3f7ff4a30b02f1b2eb5f2d218 [sys.catalog]: This master's current role is: LEADER
I20260923 22:16:20.702633 22135 sys_catalog.cc:455] T 00000000000000000000000000000000 P 191099d3f7ff4a30b02f1b2eb5f2d218 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "191099d3f7ff4a30b02f1b2eb5f2d218" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "191099d3f7ff4a30b02f1b2eb5f2d218" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 40203 } } }
I20260923 22:16:20.703022 22135 sys_catalog.cc:458] T 00000000000000000000000000000000 P 191099d3f7ff4a30b02f1b2eb5f2d218 [sys.catalog]: This master's current role is: LEADER
W20260923 22:16:20.703240 22147 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 191099d3f7ff4a30b02f1b2eb5f2d218: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260923 22:16:20.703330 22147 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260923 22:16:20.703469 22148 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260923 22:16:20.703660 22148 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260923 22:16:20.705252 22148 catalog_manager.cc:1347] Generated new cluster ID: ed26ef81e0d74b408b4939234b1cb49b
I20260923 22:16:20.705368 22148 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260923 22:16:20.705904 22131 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 22:16:20.694141 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40769 (local address 127.0.0.1:40203)
0923 22:16:20.694359 (+ 218us) server_negotiation.cc:206] Beginning negotiation
0923 22:16:20.694391 (+ 32us) server_negotiation.cc:399] Waiting for connection header
0923 22:16:20.694426 (+ 35us) server_negotiation.cc:407] Connection header received
0923 22:16:20.694963 (+ 537us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 22:16:20.694972 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 22:16:20.695045 (+ 73us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 22:16:20.695223 (+ 178us) server_negotiation.cc:226] Negotiated authn=SASL
0923 22:16:20.695864 (+ 641us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 22:16:20.697261 (+ 1397us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 22:16:20.697997 (+ 736us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 22:16:20.698703 (+ 706us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 22:16:20.698835 (+ 132us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 22:16:20.701776 (+ 2941us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0923 22:16:20.701780 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0923 22:16:20.701780 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0923 22:16:20.701792 (+ 12us) server_negotiation.cc:922] calling sasl_server_start()
0923 22:16:20.704216 (+ 2424us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0923 22:16:20.704590 (+ 374us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0923 22:16:20.704593 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0923 22:16:20.704593 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0923 22:16:20.704711 (+ 118us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0923 22:16:20.704949 (+ 238us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0923 22:16:20.704949 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0923 22:16:20.704950 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0923 22:16:20.705254 (+ 304us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0923 22:16:20.705415 (+ 161us) server_negotiation.cc:1035] Waiting for connection context
0923 22:16:20.705567 (+ 152us) server_negotiation.cc:299] Negotiation successful
0923 22:16:20.705703 (+ 136us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":162,"thread_start_us":108,"threads_started":1}
I20260923 22:16:20.706939 1567 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260923 22:16:20.716034 22131 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 22:16:20.708460 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57435 (local address 127.0.0.1:40203)
0923 22:16:20.708504 (+ 44us) server_negotiation.cc:206] Beginning negotiation
0923 22:16:20.708506 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0923 22:16:20.708687 (+ 181us) server_negotiation.cc:407] Connection header received
0923 22:16:20.708757 (+ 70us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 22:16:20.708757 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 22:16:20.708767 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 22:16:20.708812 (+ 45us) server_negotiation.cc:226] Negotiated authn=SASL
0923 22:16:20.709470 (+ 658us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 22:16:20.710201 (+ 731us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 22:16:20.711243 (+ 1042us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 22:16:20.711556 (+ 313us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 22:16:20.711616 (+ 60us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 22:16:20.713137 (+ 1521us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0923 22:16:20.713139 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0923 22:16:20.713139 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0923 22:16:20.713146 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0923 22:16:20.714713 (+ 1567us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0923 22:16:20.715118 (+ 405us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0923 22:16:20.715119 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0923 22:16:20.715120 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0923 22:16:20.715178 (+ 58us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0923 22:16:20.715389 (+ 211us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0923 22:16:20.715390 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0923 22:16:20.715390 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0923 22:16:20.715506 (+ 116us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0923 22:16:20.715599 (+ 93us) server_negotiation.cc:1035] Waiting for connection context
0923 22:16:20.715842 (+ 243us) server_negotiation.cc:299] Negotiation successful
0923 22:16:20.715931 (+ 89us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":25}
I20260923 22:16:20.716058 22148 catalog_manager.cc:1370] Generated new certificate authority record
I20260923 22:16:20.716990 22148 catalog_manager.cc:1504] Loading token signing keys...
I20260923 22:16:20.727113 22148 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 191099d3f7ff4a30b02f1b2eb5f2d218: Generated new TSK 0
I20260923 22:16:20.727347 22148 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260923 22:16:20.729985 1567 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskRiQe_a/build/release/bin/kudu with pid 22060
2026-09-23T22:16:20Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/3 (1305 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/4
Loading random data
Initializing database '/tmp/dist-test-taskRiQe_a/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1790201539513534-1567-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Sep 23 22:16:20 dist-test-slave-s2fj krb5kdc[22160](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Sep 23 22:16:20 dist-test-slave-s2fj krb5kdc[22160](info): set up 2 sockets
Sep 23 22:16:20 dist-test-slave-s2fj krb5kdc[22160](info): commencing operation
krb5kdc: starting...
W20260923 22:16:21.747994 1567 mini_kdc.cc:121] Time spent starting KDC: real 1.000s user 0.000s sys 0.005s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskRiQe_a/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1790201539513534-1567-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskRiQe_a/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1790201539513534-1567-0/krb5kdc/test-user.keytab.
Sep 23 22:16:21 dist-test-slave-s2fj krb5kdc[22160](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790201781, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-09-23T22:16:21Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-09-23T22:16:21Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskRiQe_a/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1790201539513534-1567-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskRiQe_a/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1790201539513534-1567-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260923 22:16:21.857966 1567 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskRiQe_a/build/release/bin/kudu
/tmp/dist-test-taskRiQe_a/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskRiQe_a/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1790201539513534-1567-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskRiQe_a/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1790201539513534-1567-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskRiQe_a/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1790201539513534-1567-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskRiQe_a/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1790201539513534-1567-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:39653
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-taskRiQe_a/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1790201539513534-1567-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:38711
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:39653
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=required
--rpc_encrypt_loopback_connections=false with env {KRB5CCNAME=/tmp/dist-test-taskRiQe_a/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1790201539513534-1567-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskRiQe_a/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1790201539513534-1567-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskRiQe_a/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1790201539513534-1567-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260923 22:16:21.935658 22173 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260923 22:16:21.935864 22173 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260923 22:16:21.935886 22173 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260923 22:16:21.937005 22173 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260923 22:16:21.937045 22173 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260923 22:16:21.937059 22173 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260923 22:16:21.937071 22173 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260923 22:16:21.937083 22173 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260923 22:16:21.938380 22173 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:38711
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskRiQe_a/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1790201539513534-1567-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskRiQe_a/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1790201539513534-1567-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:39653
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskRiQe_a/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1790201539513534-1567-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:39653
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskRiQe_a/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1790201539513534-1567-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.22173
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskRiQe_a/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1790201539513534-1567-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 23 Sep 2026 20:53:37 UTC on ccycloud.quasar-rkviyx.root.comops.site
build id 4668083
I20260923 22:16:21.938585 22173 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260923 22:16:21.938730 22173 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260923 22:16:21.943261 22178 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260923 22:16:21.943382 22173 server_base.cc:1034] running on GCE node
W20260923 22:16:21.943230 22179 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260923 22:16:21.943490 22181 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260923 22:16:21.943746 22173 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260923 22:16:21.944051 22173 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260923 22:16:21.945243 22173 hybrid_clock.cc:648] HybridClock initialized: now 1790201781945225 us; error 34 us; skew 500 ppm
Sep 23 22:16:21 dist-test-slave-s2fj krb5kdc[22160](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790201781, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260923 22:16:21.947495 22173 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260923 22:16:21.947909 22173 webserver.cc:460] Webserver started at http://127.0.0.1:38693/ using document root <none> and password file <none>
I20260923 22:16:21.948191 22173 fs_manager.cc:359] Metadata directory not provided
I20260923 22:16:21.948261 22173 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260923 22:16:21.948415 22173 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260923 22:16:21.949537 22173 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskRiQe_a/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1790201539513534-1567-0/minicluster-data/master-0/data/instance:
uuid: "2f9bf2465f0f4d0b9262bf6c8a69eb9d"
format_stamp: "Formatted at 2026-09-23 22:16:21 on dist-test-slave-s2fj"
server_key: "04ee57a5bf37f3ca1c554f05a3c3e23f"
server_key_iv: "935ede98119c75118e9e8e56b938b595"
server_key_version: "encryptionkey@0"
I20260923 22:16:21.949919 22173 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskRiQe_a/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1790201539513534-1567-0/minicluster-data/master-0/wal/instance:
uuid: "2f9bf2465f0f4d0b9262bf6c8a69eb9d"
format_stamp: "Formatted at 2026-09-23 22:16:21 on dist-test-slave-s2fj"
server_key: "04ee57a5bf37f3ca1c554f05a3c3e23f"
server_key_iv: "935ede98119c75118e9e8e56b938b595"
server_key_version: "encryptionkey@0"
I20260923 22:16:21.951205 22173 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.001s
I20260923 22:16:21.951923 22188 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260923 22:16:21.952124 22173 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260923 22:16:21.952222 22173 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskRiQe_a/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1790201539513534-1567-0/minicluster-data/master-0/data,/tmp/dist-test-taskRiQe_a/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1790201539513534-1567-0/minicluster-data/master-0/wal
uuid: "2f9bf2465f0f4d0b9262bf6c8a69eb9d"
format_stamp: "Formatted at 2026-09-23 22:16:21 on dist-test-slave-s2fj"
server_key: "04ee57a5bf37f3ca1c554f05a3c3e23f"
server_key_iv: "935ede98119c75118e9e8e56b938b595"
server_key_version: "encryptionkey@0"
I20260923 22:16:21.952306 22173 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskRiQe_a/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1790201539513534-1567-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskRiQe_a/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1790201539513534-1567-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskRiQe_a/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1790201539513534-1567-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260923 22:16:21.969981 22173 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260923 22:16:21.971751 22173 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260923 22:16:21.971912 22173 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260923 22:16:21.975559 22173 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:39653
I20260923 22:16:21.975594 22240 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:39653 every 8 connection(s)
I20260923 22:16:21.975936 22173 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskRiQe_a/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1790201539513534-1567-0/minicluster-data/master-0/data/info.pb
I20260923 22:16:21.976850 22241 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260923 22:16:21.979009 22241 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 2f9bf2465f0f4d0b9262bf6c8a69eb9d: Bootstrap starting.
I20260923 22:16:21.979576 22241 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 2f9bf2465f0f4d0b9262bf6c8a69eb9d: Neither blocks nor log segments found. Creating new log.
I20260923 22:16:21.979858 22241 log.cc:824] T 00000000000000000000000000000000 P 2f9bf2465f0f4d0b9262bf6c8a69eb9d: Log is configured to *not* fsync() on all Append() calls
I20260923 22:16:21.980571 22241 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 2f9bf2465f0f4d0b9262bf6c8a69eb9d: No bootstrap required, opened a new log
I20260923 22:16:21.981773 1567 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskRiQe_a/build/release/bin/kudu as pid 22173
I20260923 22:16:21.981905 1567 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskRiQe_a/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1790201539513534-1567-0/minicluster-data/master-0/wal/instance
I20260923 22:16:21.982090 1567 external_mini_cluster.cc:1428] Setting key 2ec47d8f951dd9e0367f652f89e9c815
I20260923 22:16:21.982462 22241 raft_consensus.cc:348] T 00000000000000000000000000000000 P 2f9bf2465f0f4d0b9262bf6c8a69eb9d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2f9bf2465f0f4d0b9262bf6c8a69eb9d" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 39653 } }
I20260923 22:16:21.982646 22241 raft_consensus.cc:374] T 00000000000000000000000000000000 P 2f9bf2465f0f4d0b9262bf6c8a69eb9d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260923 22:16:21.982776 22241 raft_consensus.cc:729] T 00000000000000000000000000000000 P 2f9bf2465f0f4d0b9262bf6c8a69eb9d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 2f9bf2465f0f4d0b9262bf6c8a69eb9d, State: Initialized, Role: FOLLOWER
I20260923 22:16:21.982923 22241 consensus_queue.cc:260] T 00000000000000000000000000000000 P 2f9bf2465f0f4d0b9262bf6c8a69eb9d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2f9bf2465f0f4d0b9262bf6c8a69eb9d" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 39653 } }
I20260923 22:16:21.983013 22241 raft_consensus.cc:388] T 00000000000000000000000000000000 P 2f9bf2465f0f4d0b9262bf6c8a69eb9d [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260923 22:16:21.983037 22241 raft_consensus.cc:482] T 00000000000000000000000000000000 P 2f9bf2465f0f4d0b9262bf6c8a69eb9d [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260923 22:16:21.983103 22241 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 2f9bf2465f0f4d0b9262bf6c8a69eb9d [term 0 FOLLOWER]: Advancing to term 1
I20260923 22:16:21.983983 22241 raft_consensus.cc:504] T 00000000000000000000000000000000 P 2f9bf2465f0f4d0b9262bf6c8a69eb9d [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2f9bf2465f0f4d0b9262bf6c8a69eb9d" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 39653 } }
I20260923 22:16:21.984136 22241 leader_election.cc:302] T 00000000000000000000000000000000 P 2f9bf2465f0f4d0b9262bf6c8a69eb9d [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 2f9bf2465f0f4d0b9262bf6c8a69eb9d; no voters:
I20260923 22:16:21.984309 22241 leader_election.cc:288] T 00000000000000000000000000000000 P 2f9bf2465f0f4d0b9262bf6c8a69eb9d [CANDIDATE]: Term 1 election: Requested vote from peers
I20260923 22:16:21.984342 22246 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 2f9bf2465f0f4d0b9262bf6c8a69eb9d [term 1 FOLLOWER]: Leader election won for term 1
I20260923 22:16:21.984535 22246 raft_consensus.cc:686] T 00000000000000000000000000000000 P 2f9bf2465f0f4d0b9262bf6c8a69eb9d [term 1 LEADER]: Becoming Leader. State: Replica: 2f9bf2465f0f4d0b9262bf6c8a69eb9d, State: Running, Role: LEADER
I20260923 22:16:21.984570 22241 sys_catalog.cc:564] T 00000000000000000000000000000000 P 2f9bf2465f0f4d0b9262bf6c8a69eb9d [sys.catalog]: configured and running, proceeding with master startup.
I20260923 22:16:21.984658 22246 consensus_queue.cc:237] T 00000000000000000000000000000000 P 2f9bf2465f0f4d0b9262bf6c8a69eb9d [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2f9bf2465f0f4d0b9262bf6c8a69eb9d" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 39653 } }
I20260923 22:16:21.984995 22246 sys_catalog.cc:455] T 00000000000000000000000000000000 P 2f9bf2465f0f4d0b9262bf6c8a69eb9d [sys.catalog]: SysCatalogTable state changed. Reason: New leader 2f9bf2465f0f4d0b9262bf6c8a69eb9d. Latest consensus state: current_term: 1 leader_uuid: "2f9bf2465f0f4d0b9262bf6c8a69eb9d" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2f9bf2465f0f4d0b9262bf6c8a69eb9d" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 39653 } } }
I20260923 22:16:21.985198 22246 sys_catalog.cc:458] T 00000000000000000000000000000000 P 2f9bf2465f0f4d0b9262bf6c8a69eb9d [sys.catalog]: This master's current role is: LEADER
I20260923 22:16:21.985134 22247 sys_catalog.cc:455] T 00000000000000000000000000000000 P 2f9bf2465f0f4d0b9262bf6c8a69eb9d [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "2f9bf2465f0f4d0b9262bf6c8a69eb9d" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2f9bf2465f0f4d0b9262bf6c8a69eb9d" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 39653 } } }
I20260923 22:16:21.985425 22247 sys_catalog.cc:458] T 00000000000000000000000000000000 P 2f9bf2465f0f4d0b9262bf6c8a69eb9d [sys.catalog]: This master's current role is: LEADER
I20260923 22:16:21.985824 22255 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260923 22:16:21.986354 22255 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260923 22:16:21.987782 22245 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 22:16:21.982604 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47613 (local address 127.0.0.1:39653)
0923 22:16:21.982795 (+ 191us) server_negotiation.cc:206] Beginning negotiation
0923 22:16:21.982804 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0923 22:16:21.982843 (+ 39us) server_negotiation.cc:407] Connection header received
0923 22:16:21.982960 (+ 117us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 22:16:21.982966 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 22:16:21.983024 (+ 58us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 22:16:21.983161 (+ 137us) server_negotiation.cc:226] Negotiated authn=SASL
0923 22:16:21.983354 (+ 193us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 22:16:21.984392 (+ 1038us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 22:16:21.985583 (+ 1191us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 22:16:21.986287 (+ 704us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 22:16:21.986395 (+ 108us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 22:16:21.987069 (+ 674us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0923 22:16:21.987073 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0923 22:16:21.987073 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0923 22:16:21.987316 (+ 243us) server_negotiation.cc:922] calling sasl_server_start()
0923 22:16:21.987364 (+ 48us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0923 22:16:21.987378 (+ 14us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0923 22:16:21.987435 (+ 57us) server_negotiation.cc:1035] Waiting for connection context
0923 22:16:21.987509 (+ 74us) server_negotiation.cc:299] Negotiation successful
0923 22:16:21.987569 (+ 60us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":139,"thread_start_us":83,"threads_started":1}
I20260923 22:16:21.987973 22255 catalog_manager.cc:1347] Generated new cluster ID: 55072073a9b64e44972965c478651d06
I20260923 22:16:21.988008 22255 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260923 22:16:21.988672 1567 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260923 22:16:21.999425 22245 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 22:16:21.992644 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41621 (local address 127.0.0.1:39653)
0923 22:16:21.992701 (+ 57us) server_negotiation.cc:206] Beginning negotiation
0923 22:16:21.992703 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0923 22:16:21.996248 (+ 3545us) server_negotiation.cc:407] Connection header received
0923 22:16:21.996307 (+ 59us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 22:16:21.996307 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 22:16:21.996317 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 22:16:21.996376 (+ 59us) server_negotiation.cc:226] Negotiated authn=SASL
0923 22:16:21.996490 (+ 114us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 22:16:21.997175 (+ 685us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 22:16:21.998250 (+ 1075us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 22:16:21.998523 (+ 273us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 22:16:21.998577 (+ 54us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 22:16:21.999212 (+ 635us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0923 22:16:21.999213 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0923 22:16:21.999214 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0923 22:16:21.999219 (+ 5us) server_negotiation.cc:922] calling sasl_server_start()
0923 22:16:21.999231 (+ 12us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0923 22:16:21.999234 (+ 3us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0923 22:16:21.999281 (+ 47us) server_negotiation.cc:1035] Waiting for connection context
0923 22:16:21.999340 (+ 59us) server_negotiation.cc:299] Negotiation successful
0923 22:16:21.999359 (+ 19us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":38}
I20260923 22:16:22.026446 22255 catalog_manager.cc:1370] Generated new certificate authority record
I20260923 22:16:22.026980 22255 catalog_manager.cc:1504] Loading token signing keys...
I20260923 22:16:22.030193 22255 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 2f9bf2465f0f4d0b9262bf6c8a69eb9d: Generated new TSK 0
I20260923 22:16:22.030398 22255 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260923 22:16:22.037680 1567 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskRiQe_a/build/release/bin/kudu with pid 22173
2026-09-23T22:16:22Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/4 (1307 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/5
Loading random data
Initializing database '/tmp/dist-test-taskRiQe_a/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1790201539513534-1567-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Sep 23 22:16:22 dist-test-slave-s2fj krb5kdc[22272](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Sep 23 22:16:22 dist-test-slave-s2fj krb5kdc[22272](info): set up 2 sockets
Sep 23 22:16:22 dist-test-slave-s2fj krb5kdc[22272](info): commencing operation
krb5kdc: starting...
W20260923 22:16:23.048308 1567 mini_kdc.cc:121] Time spent starting KDC: real 0.993s user 0.000s sys 0.005s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskRiQe_a/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1790201539513534-1567-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskRiQe_a/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1790201539513534-1567-0/krb5kdc/test-user.keytab.
Sep 23 22:16:23 dist-test-slave-s2fj krb5kdc[22272](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790201783, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-09-23T22:16:23Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-09-23T22:16:23Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskRiQe_a/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1790201539513534-1567-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskRiQe_a/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1790201539513534-1567-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260923 22:16:23.166345 1567 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskRiQe_a/build/release/bin/kudu
/tmp/dist-test-taskRiQe_a/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskRiQe_a/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1790201539513534-1567-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskRiQe_a/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1790201539513534-1567-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskRiQe_a/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1790201539513534-1567-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskRiQe_a/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1790201539513534-1567-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:38889
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-taskRiQe_a/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1790201539513534-1567-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:46355
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:38889
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=disabled
--rpc_encrypt_loopback_connections=false with env {KRB5CCNAME=/tmp/dist-test-taskRiQe_a/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1790201539513534-1567-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskRiQe_a/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1790201539513534-1567-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskRiQe_a/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1790201539513534-1567-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260923 22:16:23.245991 22285 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260923 22:16:23.246205 22285 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260923 22:16:23.246243 22285 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260923 22:16:23.247321 22285 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260923 22:16:23.247390 22285 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260923 22:16:23.247414 22285 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260923 22:16:23.247445 22285 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260923 22:16:23.247469 22285 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260923 22:16:23.248960 22285 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:46355
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskRiQe_a/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1790201539513534-1567-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskRiQe_a/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1790201539513534-1567-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:38889
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskRiQe_a/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1790201539513534-1567-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:38889
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskRiQe_a/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1790201539513534-1567-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.22285
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskRiQe_a/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1790201539513534-1567-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 23 Sep 2026 20:53:37 UTC on ccycloud.quasar-rkviyx.root.comops.site
build id 4668083
I20260923 22:16:23.249222 22285 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260923 22:16:23.249440 22285 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260923 22:16:23.254247 22293 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260923 22:16:23.254287 22290 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260923 22:16:23.254279 22285 server_base.cc:1034] running on GCE node
W20260923 22:16:23.254252 22291 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260923 22:16:23.254598 22285 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260923 22:16:23.254887 22285 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260923 22:16:23.256162 22285 hybrid_clock.cc:648] HybridClock initialized: now 1790201783256146 us; error 37 us; skew 500 ppm
Sep 23 22:16:23 dist-test-slave-s2fj krb5kdc[22272](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790201783, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260923 22:16:23.265684 22285 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260923 22:16:23.266152 22285 webserver.cc:460] Webserver started at http://127.0.0.1:36217/ using document root <none> and password file <none>
I20260923 22:16:23.266408 22285 fs_manager.cc:359] Metadata directory not provided
I20260923 22:16:23.266453 22285 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260923 22:16:23.266644 22285 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260923 22:16:23.267702 22285 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskRiQe_a/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1790201539513534-1567-0/minicluster-data/master-0/data/instance:
uuid: "89dff845c41843519c7a24848308a0a4"
format_stamp: "Formatted at 2026-09-23 22:16:23 on dist-test-slave-s2fj"
server_key: "60e1c747608f8ce6d504f01e06c79d84"
server_key_iv: "bc42cdfffdda190602e4655a4e0240f5"
server_key_version: "encryptionkey@0"
I20260923 22:16:23.268045 22285 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskRiQe_a/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1790201539513534-1567-0/minicluster-data/master-0/wal/instance:
uuid: "89dff845c41843519c7a24848308a0a4"
format_stamp: "Formatted at 2026-09-23 22:16:23 on dist-test-slave-s2fj"
server_key: "60e1c747608f8ce6d504f01e06c79d84"
server_key_iv: "bc42cdfffdda190602e4655a4e0240f5"
server_key_version: "encryptionkey@0"
I20260923 22:16:23.269443 22285 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.001s sys 0.000s
I20260923 22:16:23.270171 22300 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260923 22:16:23.270394 22285 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260923 22:16:23.270500 22285 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskRiQe_a/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1790201539513534-1567-0/minicluster-data/master-0/data,/tmp/dist-test-taskRiQe_a/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1790201539513534-1567-0/minicluster-data/master-0/wal
uuid: "89dff845c41843519c7a24848308a0a4"
format_stamp: "Formatted at 2026-09-23 22:16:23 on dist-test-slave-s2fj"
server_key: "60e1c747608f8ce6d504f01e06c79d84"
server_key_iv: "bc42cdfffdda190602e4655a4e0240f5"
server_key_version: "encryptionkey@0"
I20260923 22:16:23.270596 22285 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskRiQe_a/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1790201539513534-1567-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskRiQe_a/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1790201539513534-1567-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskRiQe_a/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1790201539513534-1567-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260923 22:16:23.273892 22285 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260923 22:16:23.275738 22285 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260923 22:16:23.275904 22285 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260923 22:16:23.279692 22285 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:38889
I20260923 22:16:23.279740 22352 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:38889 every 8 connection(s)
I20260923 22:16:23.280112 22285 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskRiQe_a/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1790201539513534-1567-0/minicluster-data/master-0/data/info.pb
I20260923 22:16:23.280534 1567 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskRiQe_a/build/release/bin/kudu as pid 22285
I20260923 22:16:23.280671 1567 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskRiQe_a/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1790201539513534-1567-0/minicluster-data/master-0/wal/instance
I20260923 22:16:23.280871 1567 external_mini_cluster.cc:1428] Setting key 4acbed6d4aa5a6ccff2eda342cedb7ae
I20260923 22:16:23.281282 22353 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260923 22:16:23.283491 22353 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 89dff845c41843519c7a24848308a0a4: Bootstrap starting.
I20260923 22:16:23.283977 22353 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 89dff845c41843519c7a24848308a0a4: Neither blocks nor log segments found. Creating new log.
I20260923 22:16:23.284339 22353 log.cc:824] T 00000000000000000000000000000000 P 89dff845c41843519c7a24848308a0a4: Log is configured to *not* fsync() on all Append() calls
I20260923 22:16:23.285079 22353 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 89dff845c41843519c7a24848308a0a4: No bootstrap required, opened a new log
I20260923 22:16:23.286199 22356 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 22:16:23.281359 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49997 (local address 127.0.0.1:38889)
0923 22:16:23.284210 (+ 2851us) server_negotiation.cc:206] Beginning negotiation
0923 22:16:23.284220 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0923 22:16:23.284256 (+ 36us) server_negotiation.cc:407] Connection header received
0923 22:16:23.284846 (+ 590us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 22:16:23.284855 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 22:16:23.284898 (+ 43us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 22:16:23.285055 (+ 157us) server_negotiation.cc:226] Negotiated authn=SASL
0923 22:16:23.285776 (+ 721us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0923 22:16:23.285780 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0923 22:16:23.285780 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0923 22:16:23.285785 (+ 5us) server_negotiation.cc:922] calling sasl_server_start()
0923 22:16:23.285828 (+ 43us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0923 22:16:23.285845 (+ 17us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0923 22:16:23.285938 (+ 93us) server_negotiation.cc:1035] Waiting for connection context
0923 22:16:23.285975 (+ 37us) server_negotiation.cc:299] Negotiation successful
0923 22:16:23.286013 (+ 38us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":2793,"thread_start_us":82,"threads_started":1}
I20260923 22:16:23.287285 1567 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260923 22:16:23.287410 22353 raft_consensus.cc:348] T 00000000000000000000000000000000 P 89dff845c41843519c7a24848308a0a4 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "89dff845c41843519c7a24848308a0a4" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 38889 } }
I20260923 22:16:23.287582 22353 raft_consensus.cc:374] T 00000000000000000000000000000000 P 89dff845c41843519c7a24848308a0a4 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260923 22:16:23.287612 22353 raft_consensus.cc:729] T 00000000000000000000000000000000 P 89dff845c41843519c7a24848308a0a4 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 89dff845c41843519c7a24848308a0a4, State: Initialized, Role: FOLLOWER
I20260923 22:16:23.287721 22353 consensus_queue.cc:260] T 00000000000000000000000000000000 P 89dff845c41843519c7a24848308a0a4 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "89dff845c41843519c7a24848308a0a4" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 38889 } }
I20260923 22:16:23.287807 22353 raft_consensus.cc:388] T 00000000000000000000000000000000 P 89dff845c41843519c7a24848308a0a4 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260923 22:16:23.287853 22353 raft_consensus.cc:482] T 00000000000000000000000000000000 P 89dff845c41843519c7a24848308a0a4 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260923 22:16:23.287902 22353 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 89dff845c41843519c7a24848308a0a4 [term 0 FOLLOWER]: Advancing to term 1
I20260923 22:16:23.288686 22353 raft_consensus.cc:504] T 00000000000000000000000000000000 P 89dff845c41843519c7a24848308a0a4 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "89dff845c41843519c7a24848308a0a4" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 38889 } }
I20260923 22:16:23.288823 22353 leader_election.cc:302] T 00000000000000000000000000000000 P 89dff845c41843519c7a24848308a0a4 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 89dff845c41843519c7a24848308a0a4; no voters:
I20260923 22:16:23.289007 22353 leader_election.cc:288] T 00000000000000000000000000000000 P 89dff845c41843519c7a24848308a0a4 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260923 22:16:23.289034 22362 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 89dff845c41843519c7a24848308a0a4 [term 1 FOLLOWER]: Leader election won for term 1
I20260923 22:16:23.289251 22362 raft_consensus.cc:686] T 00000000000000000000000000000000 P 89dff845c41843519c7a24848308a0a4 [term 1 LEADER]: Becoming Leader. State: Replica: 89dff845c41843519c7a24848308a0a4, State: Running, Role: LEADER
I20260923 22:16:23.289273 22353 sys_catalog.cc:564] T 00000000000000000000000000000000 P 89dff845c41843519c7a24848308a0a4 [sys.catalog]: configured and running, proceeding with master startup.
I20260923 22:16:23.289395 22362 consensus_queue.cc:237] T 00000000000000000000000000000000 P 89dff845c41843519c7a24848308a0a4 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "89dff845c41843519c7a24848308a0a4" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 38889 } }
I20260923 22:16:23.289738 22362 sys_catalog.cc:455] T 00000000000000000000000000000000 P 89dff845c41843519c7a24848308a0a4 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 89dff845c41843519c7a24848308a0a4. Latest consensus state: current_term: 1 leader_uuid: "89dff845c41843519c7a24848308a0a4" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "89dff845c41843519c7a24848308a0a4" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 38889 } } }
I20260923 22:16:23.289819 22362 sys_catalog.cc:458] T 00000000000000000000000000000000 P 89dff845c41843519c7a24848308a0a4 [sys.catalog]: This master's current role is: LEADER
I20260923 22:16:23.289747 22365 sys_catalog.cc:455] T 00000000000000000000000000000000 P 89dff845c41843519c7a24848308a0a4 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "89dff845c41843519c7a24848308a0a4" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "89dff845c41843519c7a24848308a0a4" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 38889 } } }
I20260923 22:16:23.290041 22365 sys_catalog.cc:458] T 00000000000000000000000000000000 P 89dff845c41843519c7a24848308a0a4 [sys.catalog]: This master's current role is: LEADER
I20260923 22:16:23.290300 22356 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 22:16:23.288941 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57693 (local address 127.0.0.1:38889)
0923 22:16:23.288985 (+ 44us) server_negotiation.cc:206] Beginning negotiation
0923 22:16:23.288987 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0923 22:16:23.289121 (+ 134us) server_negotiation.cc:407] Connection header received
0923 22:16:23.289232 (+ 111us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 22:16:23.289234 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 22:16:23.289239 (+ 5us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 22:16:23.289299 (+ 60us) server_negotiation.cc:226] Negotiated authn=SASL
0923 22:16:23.290055 (+ 756us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0923 22:16:23.290056 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0923 22:16:23.290057 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0923 22:16:23.290061 (+ 4us) server_negotiation.cc:922] calling sasl_server_start()
0923 22:16:23.290075 (+ 14us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0923 22:16:23.290080 (+ 5us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0923 22:16:23.290110 (+ 30us) server_negotiation.cc:1035] Waiting for connection context
0923 22:16:23.290206 (+ 96us) server_negotiation.cc:299] Negotiation successful
0923 22:16:23.290221 (+ 15us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":23}
I20260923 22:16:23.290635 22368 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260923 22:16:23.291148 22368 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260923 22:16:23.292611 22368 catalog_manager.cc:1347] Generated new cluster ID: 26b6acf2562b4a358a9ea40cc3f5b7e6
I20260923 22:16:23.292661 22368 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260923 22:16:23.311061 22368 catalog_manager.cc:1370] Generated new certificate authority record
I20260923 22:16:23.311286 22368 catalog_manager.cc:1504] Loading token signing keys...
I20260923 22:16:23.315107 22368 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 89dff845c41843519c7a24848308a0a4: Generated new TSK 0
I20260923 22:16:23.315328 22368 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260923 22:16:23.322497 1567 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskRiQe_a/build/release/bin/kudu with pid 22285
2026-09-23T22:16:23Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/5 (1284 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/6
Loading random data
Initializing database '/tmp/dist-test-taskRiQe_a/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1790201539513534-1567-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Sep 23 22:16:23 dist-test-slave-s2fj krb5kdc[22384](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Sep 23 22:16:23 dist-test-slave-s2fj krb5kdc[22384](info): set up 2 sockets
Sep 23 22:16:23 dist-test-slave-s2fj krb5kdc[22384](info): commencing operation
krb5kdc: starting...
W20260923 22:16:24.340184 1567 mini_kdc.cc:121] Time spent starting KDC: real 0.999s user 0.000s sys 0.006s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskRiQe_a/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1790201539513534-1567-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskRiQe_a/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1790201539513534-1567-0/krb5kdc/test-user.keytab.
Sep 23 22:16:24 dist-test-slave-s2fj krb5kdc[22384](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790201784, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-09-23T22:16:24Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-09-23T22:16:24Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskRiQe_a/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1790201539513534-1567-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskRiQe_a/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1790201539513534-1567-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260923 22:16:24.458699 1567 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskRiQe_a/build/release/bin/kudu
/tmp/dist-test-taskRiQe_a/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskRiQe_a/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1790201539513534-1567-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskRiQe_a/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1790201539513534-1567-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskRiQe_a/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1790201539513534-1567-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskRiQe_a/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1790201539513534-1567-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:46875
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-taskRiQe_a/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1790201539513534-1567-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:35775
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:46875
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=required
--rpc_encryption=required
--rpc_encrypt_loopback_connections=false with env {KRB5CCNAME=/tmp/dist-test-taskRiQe_a/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1790201539513534-1567-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskRiQe_a/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1790201539513534-1567-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskRiQe_a/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1790201539513534-1567-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260923 22:16:24.539425 22397 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260923 22:16:24.539650 22397 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260923 22:16:24.539673 22397 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260923 22:16:24.540813 22397 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260923 22:16:24.540854 22397 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260923 22:16:24.540868 22397 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260923 22:16:24.540881 22397 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260923 22:16:24.540894 22397 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260923 22:16:24.542510 22397 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:35775
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskRiQe_a/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1790201539513534-1567-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskRiQe_a/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1790201539513534-1567-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:46875
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskRiQe_a/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1790201539513534-1567-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:46875
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskRiQe_a/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1790201539513534-1567-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.22397
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskRiQe_a/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1790201539513534-1567-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 23 Sep 2026 20:53:37 UTC on ccycloud.quasar-rkviyx.root.comops.site
build id 4668083
I20260923 22:16:24.542717 22397 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260923 22:16:24.542887 22397 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260923 22:16:24.547570 22402 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260923 22:16:24.547577 22403 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260923 22:16:24.547670 22397 server_base.cc:1034] running on GCE node
W20260923 22:16:24.547616 22405 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260923 22:16:24.548130 22397 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260923 22:16:24.548408 22397 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260923 22:16:24.549571 22397 hybrid_clock.cc:648] HybridClock initialized: now 1790201784549561 us; error 32 us; skew 500 ppm
Sep 23 22:16:24 dist-test-slave-s2fj krb5kdc[22384](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790201784, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260923 22:16:24.552000 22397 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260923 22:16:24.552502 22397 webserver.cc:460] Webserver started at http://127.0.0.1:34593/ using document root <none> and password file <none>
I20260923 22:16:24.552697 22397 fs_manager.cc:359] Metadata directory not provided
I20260923 22:16:24.552740 22397 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260923 22:16:24.552829 22397 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260923 22:16:24.553851 22397 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskRiQe_a/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1790201539513534-1567-0/minicluster-data/master-0/data/instance:
uuid: "c5d82f60720c4c7c9a8cb4d80f51e4cf"
format_stamp: "Formatted at 2026-09-23 22:16:24 on dist-test-slave-s2fj"
server_key: "ef471dcef0af9c84f2ea73ce2baeabcd"
server_key_iv: "f5372bbb64cb2448e15ebd693dbfa515"
server_key_version: "encryptionkey@0"
I20260923 22:16:24.554167 22397 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskRiQe_a/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1790201539513534-1567-0/minicluster-data/master-0/wal/instance:
uuid: "c5d82f60720c4c7c9a8cb4d80f51e4cf"
format_stamp: "Formatted at 2026-09-23 22:16:24 on dist-test-slave-s2fj"
server_key: "ef471dcef0af9c84f2ea73ce2baeabcd"
server_key_iv: "f5372bbb64cb2448e15ebd693dbfa515"
server_key_version: "encryptionkey@0"
I20260923 22:16:24.555365 22397 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.001s
I20260923 22:16:24.556180 22412 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260923 22:16:24.556361 22397 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260923 22:16:24.556463 22397 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskRiQe_a/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1790201539513534-1567-0/minicluster-data/master-0/data,/tmp/dist-test-taskRiQe_a/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1790201539513534-1567-0/minicluster-data/master-0/wal
uuid: "c5d82f60720c4c7c9a8cb4d80f51e4cf"
format_stamp: "Formatted at 2026-09-23 22:16:24 on dist-test-slave-s2fj"
server_key: "ef471dcef0af9c84f2ea73ce2baeabcd"
server_key_iv: "f5372bbb64cb2448e15ebd693dbfa515"
server_key_version: "encryptionkey@0"
I20260923 22:16:24.556555 22397 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskRiQe_a/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1790201539513534-1567-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskRiQe_a/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1790201539513534-1567-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskRiQe_a/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1790201539513534-1567-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260923 22:16:24.570253 22397 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260923 22:16:24.572330 22397 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260923 22:16:24.572502 22397 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260923 22:16:24.576725 22397 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:46875
I20260923 22:16:24.576766 22464 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:46875 every 8 connection(s)
I20260923 22:16:24.577124 22397 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskRiQe_a/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1790201539513534-1567-0/minicluster-data/master-0/data/info.pb
I20260923 22:16:24.578102 22465 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260923 22:16:24.580469 22465 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P c5d82f60720c4c7c9a8cb4d80f51e4cf: Bootstrap starting.
I20260923 22:16:24.581053 22465 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P c5d82f60720c4c7c9a8cb4d80f51e4cf: Neither blocks nor log segments found. Creating new log.
I20260923 22:16:24.581334 22465 log.cc:824] T 00000000000000000000000000000000 P c5d82f60720c4c7c9a8cb4d80f51e4cf: Log is configured to *not* fsync() on all Append() calls
I20260923 22:16:24.582060 22465 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P c5d82f60720c4c7c9a8cb4d80f51e4cf: No bootstrap required, opened a new log
I20260923 22:16:24.583725 1567 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskRiQe_a/build/release/bin/kudu as pid 22397
I20260923 22:16:24.583853 1567 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskRiQe_a/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1790201539513534-1567-0/minicluster-data/master-0/wal/instance
I20260923 22:16:24.583889 22465 raft_consensus.cc:348] T 00000000000000000000000000000000 P c5d82f60720c4c7c9a8cb4d80f51e4cf [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c5d82f60720c4c7c9a8cb4d80f51e4cf" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 46875 } }
I20260923 22:16:24.584055 1567 external_mini_cluster.cc:1428] Setting key c56d37e4da85b6aed8c059e4018481e7
I20260923 22:16:24.584110 22465 raft_consensus.cc:374] T 00000000000000000000000000000000 P c5d82f60720c4c7c9a8cb4d80f51e4cf [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260923 22:16:24.584152 22465 raft_consensus.cc:729] T 00000000000000000000000000000000 P c5d82f60720c4c7c9a8cb4d80f51e4cf [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c5d82f60720c4c7c9a8cb4d80f51e4cf, State: Initialized, Role: FOLLOWER
I20260923 22:16:24.584312 22465 consensus_queue.cc:260] T 00000000000000000000000000000000 P c5d82f60720c4c7c9a8cb4d80f51e4cf [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c5d82f60720c4c7c9a8cb4d80f51e4cf" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 46875 } }
I20260923 22:16:24.584379 22465 raft_consensus.cc:388] T 00000000000000000000000000000000 P c5d82f60720c4c7c9a8cb4d80f51e4cf [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260923 22:16:24.584422 22465 raft_consensus.cc:482] T 00000000000000000000000000000000 P c5d82f60720c4c7c9a8cb4d80f51e4cf [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260923 22:16:24.584473 22465 raft_consensus.cc:3037] T 00000000000000000000000000000000 P c5d82f60720c4c7c9a8cb4d80f51e4cf [term 0 FOLLOWER]: Advancing to term 1
I20260923 22:16:24.585415 22465 raft_consensus.cc:504] T 00000000000000000000000000000000 P c5d82f60720c4c7c9a8cb4d80f51e4cf [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c5d82f60720c4c7c9a8cb4d80f51e4cf" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 46875 } }
I20260923 22:16:24.585538 22465 leader_election.cc:302] T 00000000000000000000000000000000 P c5d82f60720c4c7c9a8cb4d80f51e4cf [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: c5d82f60720c4c7c9a8cb4d80f51e4cf; no voters:
I20260923 22:16:24.585713 22465 leader_election.cc:288] T 00000000000000000000000000000000 P c5d82f60720c4c7c9a8cb4d80f51e4cf [CANDIDATE]: Term 1 election: Requested vote from peers
I20260923 22:16:24.585767 22470 raft_consensus.cc:2781] T 00000000000000000000000000000000 P c5d82f60720c4c7c9a8cb4d80f51e4cf [term 1 FOLLOWER]: Leader election won for term 1
I20260923 22:16:24.585912 22470 raft_consensus.cc:686] T 00000000000000000000000000000000 P c5d82f60720c4c7c9a8cb4d80f51e4cf [term 1 LEADER]: Becoming Leader. State: Replica: c5d82f60720c4c7c9a8cb4d80f51e4cf, State: Running, Role: LEADER
I20260923 22:16:24.585970 22465 sys_catalog.cc:564] T 00000000000000000000000000000000 P c5d82f60720c4c7c9a8cb4d80f51e4cf [sys.catalog]: configured and running, proceeding with master startup.
I20260923 22:16:24.586093 22470 consensus_queue.cc:237] T 00000000000000000000000000000000 P c5d82f60720c4c7c9a8cb4d80f51e4cf [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c5d82f60720c4c7c9a8cb4d80f51e4cf" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 46875 } }
I20260923 22:16:24.587116 22471 sys_catalog.cc:455] T 00000000000000000000000000000000 P c5d82f60720c4c7c9a8cb4d80f51e4cf [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "c5d82f60720c4c7c9a8cb4d80f51e4cf" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c5d82f60720c4c7c9a8cb4d80f51e4cf" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 46875 } } }
I20260923 22:16:24.587208 22471 sys_catalog.cc:458] T 00000000000000000000000000000000 P c5d82f60720c4c7c9a8cb4d80f51e4cf [sys.catalog]: This master's current role is: LEADER
I20260923 22:16:24.587549 22472 sys_catalog.cc:455] T 00000000000000000000000000000000 P c5d82f60720c4c7c9a8cb4d80f51e4cf [sys.catalog]: SysCatalogTable state changed. Reason: New leader c5d82f60720c4c7c9a8cb4d80f51e4cf. Latest consensus state: current_term: 1 leader_uuid: "c5d82f60720c4c7c9a8cb4d80f51e4cf" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c5d82f60720c4c7c9a8cb4d80f51e4cf" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 46875 } } }
I20260923 22:16:24.587700 22472 sys_catalog.cc:458] T 00000000000000000000000000000000 P c5d82f60720c4c7c9a8cb4d80f51e4cf [sys.catalog]: This master's current role is: LEADER
W20260923 22:16:24.587949 22486 catalog_manager.cc:1558] T 00000000000000000000000000000000 P c5d82f60720c4c7c9a8cb4d80f51e4cf: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260923 22:16:24.587999 22486 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260923 22:16:24.588052 22484 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260923 22:16:24.588227 22484 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260923 22:16:24.589875 22484 catalog_manager.cc:1347] Generated new cluster ID: 5a0a61a3f21c4478862c3dd7349492b4
I20260923 22:16:24.589929 22484 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
Sep 23 22:16:24 dist-test-slave-s2fj krb5kdc[22384](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790201784, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.0.0.1@KRBTEST.COM
I20260923 22:16:24.594956 22469 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 22:16:24.584857 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.138:51387 (local address 127.0.0.1:46875)
0923 22:16:24.585060 (+ 203us) server_negotiation.cc:206] Beginning negotiation
0923 22:16:24.585069 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0923 22:16:24.585092 (+ 23us) server_negotiation.cc:407] Connection header received
0923 22:16:24.585188 (+ 96us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 22:16:24.585193 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 22:16:24.585238 (+ 45us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 22:16:24.585384 (+ 146us) server_negotiation.cc:226] Negotiated authn=SASL
0923 22:16:24.586176 (+ 792us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 22:16:24.587622 (+ 1446us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 22:16:24.588619 (+ 997us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 22:16:24.589050 (+ 431us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 22:16:24.589182 (+ 132us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 22:16:24.591649 (+ 2467us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0923 22:16:24.591653 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0923 22:16:24.591653 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0923 22:16:24.591666 (+ 13us) server_negotiation.cc:922] calling sasl_server_start()
0923 22:16:24.593404 (+ 1738us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0923 22:16:24.593817 (+ 413us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0923 22:16:24.593820 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0923 22:16:24.593821 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0923 22:16:24.593924 (+ 103us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0923 22:16:24.594177 (+ 253us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0923 22:16:24.594178 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0923 22:16:24.594179 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0923 22:16:24.594423 (+ 244us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0923 22:16:24.594610 (+ 187us) server_negotiation.cc:1035] Waiting for connection context
0923 22:16:24.594694 (+ 84us) server_negotiation.cc:299] Negotiation successful
0923 22:16:24.594797 (+ 103us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":156,"thread_start_us":111,"threads_started":1}
I20260923 22:16:24.596169 1567 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260923 22:16:24.610767 22469 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 22:16:24.597695 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.138:45559 (local address 127.0.0.1:46875)
0923 22:16:24.597759 (+ 64us) server_negotiation.cc:206] Beginning negotiation
0923 22:16:24.597761 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0923 22:16:24.604418 (+ 6657us) server_negotiation.cc:407] Connection header received
0923 22:16:24.604494 (+ 76us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 22:16:24.604495 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 22:16:24.604508 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 22:16:24.604583 (+ 75us) server_negotiation.cc:226] Negotiated authn=SASL
0923 22:16:24.605171 (+ 588us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 22:16:24.605655 (+ 484us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 22:16:24.606386 (+ 731us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 22:16:24.606608 (+ 222us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 22:16:24.606708 (+ 100us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 22:16:24.608017 (+ 1309us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0923 22:16:24.608019 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0923 22:16:24.608019 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0923 22:16:24.608026 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0923 22:16:24.609516 (+ 1490us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0923 22:16:24.609927 (+ 411us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0923 22:16:24.609929 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0923 22:16:24.609929 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0923 22:16:24.609989 (+ 60us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0923 22:16:24.610236 (+ 247us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0923 22:16:24.610237 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0923 22:16:24.610238 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0923 22:16:24.610351 (+ 113us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0923 22:16:24.610481 (+ 130us) server_negotiation.cc:1035] Waiting for connection context
0923 22:16:24.610593 (+ 112us) server_negotiation.cc:299] Negotiation successful
0923 22:16:24.610659 (+ 66us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":40}
I20260923 22:16:24.615365 22484 catalog_manager.cc:1370] Generated new certificate authority record
I20260923 22:16:24.615911 22484 catalog_manager.cc:1504] Loading token signing keys...
I20260923 22:16:24.622880 22484 catalog_manager.cc:5868] T 00000000000000000000000000000000 P c5d82f60720c4c7c9a8cb4d80f51e4cf: Generated new TSK 0
I20260923 22:16:24.623097 22484 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260923 22:16:24.629129 1567 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskRiQe_a/build/release/bin/kudu with pid 22397
W20260923 22:16:24.633181 22382 connection.cc:535] client connection to 127.0.0.1:46875 recv error: Network error: failed to read from TLS socket (remote: 127.0.0.1:46875): Connection reset by peer (error 104)
2026-09-23T22:16:24Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/6 (1306 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/7
Loading random data
Initializing database '/tmp/dist-test-taskRiQe_a/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1790201539513534-1567-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Sep 23 22:16:24 dist-test-slave-s2fj krb5kdc[22496](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Sep 23 22:16:24 dist-test-slave-s2fj krb5kdc[22496](info): set up 2 sockets
Sep 23 22:16:24 dist-test-slave-s2fj krb5kdc[22496](info): commencing operation
krb5kdc: starting...
W20260923 22:16:25.664026 1567 mini_kdc.cc:121] Time spent starting KDC: real 1.017s user 0.004s sys 0.002s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskRiQe_a/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1790201539513534-1567-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskRiQe_a/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1790201539513534-1567-0/krb5kdc/test-user.keytab.
Sep 23 22:16:25 dist-test-slave-s2fj krb5kdc[22496](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790201785, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-09-23T22:16:25Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-09-23T22:16:25Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskRiQe_a/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1790201539513534-1567-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskRiQe_a/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1790201539513534-1567-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260923 22:16:25.774896 1567 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskRiQe_a/build/release/bin/kudu
/tmp/dist-test-taskRiQe_a/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskRiQe_a/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1790201539513534-1567-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskRiQe_a/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1790201539513534-1567-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskRiQe_a/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1790201539513534-1567-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskRiQe_a/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1790201539513534-1567-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:39167
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-taskRiQe_a/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1790201539513534-1567-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:33741
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:39167
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=required
--rpc_encrypt_loopback_connections=false with env {KRB5CCNAME=/tmp/dist-test-taskRiQe_a/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1790201539513534-1567-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskRiQe_a/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1790201539513534-1567-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskRiQe_a/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1790201539513534-1567-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260923 22:16:25.854724 22509 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260923 22:16:25.854944 22509 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260923 22:16:25.854967 22509 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260923 22:16:25.856161 22509 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260923 22:16:25.856202 22509 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260923 22:16:25.856217 22509 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260923 22:16:25.856231 22509 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260923 22:16:25.856281 22509 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260923 22:16:25.857810 22509 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:33741
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskRiQe_a/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1790201539513534-1567-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskRiQe_a/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1790201539513534-1567-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:39167
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskRiQe_a/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1790201539513534-1567-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:39167
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskRiQe_a/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1790201539513534-1567-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.22509
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskRiQe_a/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1790201539513534-1567-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 23 Sep 2026 20:53:37 UTC on ccycloud.quasar-rkviyx.root.comops.site
build id 4668083
I20260923 22:16:25.858078 22509 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260923 22:16:25.858345 22509 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260923 22:16:25.863348 22517 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260923 22:16:25.863440 22509 server_base.cc:1034] running on GCE node
W20260923 22:16:25.863348 22514 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260923 22:16:25.863341 22515 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260923 22:16:25.863827 22509 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260923 22:16:25.864188 22509 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260923 22:16:25.865334 22509 hybrid_clock.cc:648] HybridClock initialized: now 1790201785865324 us; error 27 us; skew 500 ppm
Sep 23 22:16:25 dist-test-slave-s2fj krb5kdc[22496](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790201785, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260923 22:16:25.867785 22509 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260923 22:16:25.868309 22509 webserver.cc:460] Webserver started at http://127.0.0.1:44859/ using document root <none> and password file <none>
I20260923 22:16:25.868541 22509 fs_manager.cc:359] Metadata directory not provided
I20260923 22:16:25.868607 22509 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260923 22:16:25.868746 22509 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260923 22:16:25.869860 22509 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskRiQe_a/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1790201539513534-1567-0/minicluster-data/master-0/data/instance:
uuid: "bff6da0cc1e24a9aa633b75447f94435"
format_stamp: "Formatted at 2026-09-23 22:16:25 on dist-test-slave-s2fj"
server_key: "051aabca66a4d149bf02c7dab23b0824"
server_key_iv: "3dfa5f51d40dc471bb2db1e1434e1459"
server_key_version: "encryptionkey@0"
I20260923 22:16:25.870244 22509 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskRiQe_a/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1790201539513534-1567-0/minicluster-data/master-0/wal/instance:
uuid: "bff6da0cc1e24a9aa633b75447f94435"
format_stamp: "Formatted at 2026-09-23 22:16:25 on dist-test-slave-s2fj"
server_key: "051aabca66a4d149bf02c7dab23b0824"
server_key_iv: "3dfa5f51d40dc471bb2db1e1434e1459"
server_key_version: "encryptionkey@0"
I20260923 22:16:25.871601 22509 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260923 22:16:25.872408 22524 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260923 22:16:25.872596 22509 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260923 22:16:25.872712 22509 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskRiQe_a/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1790201539513534-1567-0/minicluster-data/master-0/data,/tmp/dist-test-taskRiQe_a/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1790201539513534-1567-0/minicluster-data/master-0/wal
uuid: "bff6da0cc1e24a9aa633b75447f94435"
format_stamp: "Formatted at 2026-09-23 22:16:25 on dist-test-slave-s2fj"
server_key: "051aabca66a4d149bf02c7dab23b0824"
server_key_iv: "3dfa5f51d40dc471bb2db1e1434e1459"
server_key_version: "encryptionkey@0"
I20260923 22:16:25.872792 22509 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskRiQe_a/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1790201539513534-1567-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskRiQe_a/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1790201539513534-1567-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskRiQe_a/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1790201539513534-1567-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260923 22:16:25.883953 22509 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260923 22:16:25.885883 22509 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260923 22:16:25.886059 22509 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260923 22:16:25.890400 22509 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:39167
I20260923 22:16:25.890431 22576 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:39167 every 8 connection(s)
I20260923 22:16:25.890811 22509 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskRiQe_a/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1790201539513534-1567-0/minicluster-data/master-0/data/info.pb
I20260923 22:16:25.891837 22577 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260923 22:16:25.894124 22577 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P bff6da0cc1e24a9aa633b75447f94435: Bootstrap starting.
I20260923 22:16:25.894651 22577 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P bff6da0cc1e24a9aa633b75447f94435: Neither blocks nor log segments found. Creating new log.
I20260923 22:16:25.894896 22577 log.cc:824] T 00000000000000000000000000000000 P bff6da0cc1e24a9aa633b75447f94435: Log is configured to *not* fsync() on all Append() calls
I20260923 22:16:25.895541 22577 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P bff6da0cc1e24a9aa633b75447f94435: No bootstrap required, opened a new log
I20260923 22:16:25.897095 22577 raft_consensus.cc:348] T 00000000000000000000000000000000 P bff6da0cc1e24a9aa633b75447f94435 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "bff6da0cc1e24a9aa633b75447f94435" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 39167 } }
I20260923 22:16:25.897238 22577 raft_consensus.cc:374] T 00000000000000000000000000000000 P bff6da0cc1e24a9aa633b75447f94435 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260923 22:16:25.897261 22577 raft_consensus.cc:729] T 00000000000000000000000000000000 P bff6da0cc1e24a9aa633b75447f94435 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: bff6da0cc1e24a9aa633b75447f94435, State: Initialized, Role: FOLLOWER
I20260923 22:16:25.897408 22577 consensus_queue.cc:260] T 00000000000000000000000000000000 P bff6da0cc1e24a9aa633b75447f94435 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "bff6da0cc1e24a9aa633b75447f94435" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 39167 } }
I20260923 22:16:25.897492 22577 raft_consensus.cc:388] T 00000000000000000000000000000000 P bff6da0cc1e24a9aa633b75447f94435 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260923 22:16:25.897531 22577 raft_consensus.cc:482] T 00000000000000000000000000000000 P bff6da0cc1e24a9aa633b75447f94435 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260923 22:16:25.897559 22577 raft_consensus.cc:3037] T 00000000000000000000000000000000 P bff6da0cc1e24a9aa633b75447f94435 [term 0 FOLLOWER]: Advancing to term 1
I20260923 22:16:25.898043 22577 raft_consensus.cc:504] T 00000000000000000000000000000000 P bff6da0cc1e24a9aa633b75447f94435 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "bff6da0cc1e24a9aa633b75447f94435" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 39167 } }
I20260923 22:16:25.898151 22577 leader_election.cc:302] T 00000000000000000000000000000000 P bff6da0cc1e24a9aa633b75447f94435 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: bff6da0cc1e24a9aa633b75447f94435; no voters:
I20260923 22:16:25.898284 22577 leader_election.cc:288] T 00000000000000000000000000000000 P bff6da0cc1e24a9aa633b75447f94435 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260923 22:16:25.898360 22580 raft_consensus.cc:2781] T 00000000000000000000000000000000 P bff6da0cc1e24a9aa633b75447f94435 [term 1 FOLLOWER]: Leader election won for term 1
I20260923 22:16:25.898545 22580 raft_consensus.cc:686] T 00000000000000000000000000000000 P bff6da0cc1e24a9aa633b75447f94435 [term 1 LEADER]: Becoming Leader. State: Replica: bff6da0cc1e24a9aa633b75447f94435, State: Running, Role: LEADER
I20260923 22:16:25.898587 22577 sys_catalog.cc:564] T 00000000000000000000000000000000 P bff6da0cc1e24a9aa633b75447f94435 [sys.catalog]: configured and running, proceeding with master startup.
I20260923 22:16:25.898666 22580 consensus_queue.cc:237] T 00000000000000000000000000000000 P bff6da0cc1e24a9aa633b75447f94435 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "bff6da0cc1e24a9aa633b75447f94435" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 39167 } }
I20260923 22:16:25.899084 22582 sys_catalog.cc:455] T 00000000000000000000000000000000 P bff6da0cc1e24a9aa633b75447f94435 [sys.catalog]: SysCatalogTable state changed. Reason: New leader bff6da0cc1e24a9aa633b75447f94435. Latest consensus state: current_term: 1 leader_uuid: "bff6da0cc1e24a9aa633b75447f94435" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "bff6da0cc1e24a9aa633b75447f94435" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 39167 } } }
I20260923 22:16:25.899165 22582 sys_catalog.cc:458] T 00000000000000000000000000000000 P bff6da0cc1e24a9aa633b75447f94435 [sys.catalog]: This master's current role is: LEADER
I20260923 22:16:25.899266 22581 sys_catalog.cc:455] T 00000000000000000000000000000000 P bff6da0cc1e24a9aa633b75447f94435 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "bff6da0cc1e24a9aa633b75447f94435" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "bff6da0cc1e24a9aa633b75447f94435" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 39167 } } }
I20260923 22:16:25.899350 22581 sys_catalog.cc:458] T 00000000000000000000000000000000 P bff6da0cc1e24a9aa633b75447f94435 [sys.catalog]: This master's current role is: LEADER
I20260923 22:16:25.899523 1567 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskRiQe_a/build/release/bin/kudu as pid 22509
I20260923 22:16:25.899662 1567 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskRiQe_a/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1790201539513534-1567-0/minicluster-data/master-0/wal/instance
I20260923 22:16:25.899865 1567 external_mini_cluster.cc:1428] Setting key 2f3081e04c8efb639528edf09811220e
I20260923 22:16:25.899905 22588 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260923 22:16:25.900575 22588 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260923 22:16:25.902297 22588 catalog_manager.cc:1347] Generated new cluster ID: 6b49cb3be30848ccbac7cdb8d1734303
I20260923 22:16:25.902348 22588 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260923 22:16:25.905848 22599 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 22:16:25.900418 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.138:58445 (local address 127.0.0.1:39167)
0923 22:16:25.900587 (+ 169us) server_negotiation.cc:206] Beginning negotiation
0923 22:16:25.900597 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0923 22:16:25.900636 (+ 39us) server_negotiation.cc:407] Connection header received
0923 22:16:25.900771 (+ 135us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 22:16:25.900779 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 22:16:25.900840 (+ 61us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 22:16:25.901001 (+ 161us) server_negotiation.cc:226] Negotiated authn=SASL
0923 22:16:25.901227 (+ 226us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 22:16:25.902506 (+ 1279us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 22:16:25.903618 (+ 1112us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 22:16:25.904304 (+ 686us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 22:16:25.904399 (+ 95us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 22:16:25.905164 (+ 765us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0923 22:16:25.905168 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0923 22:16:25.905168 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0923 22:16:25.905381 (+ 213us) server_negotiation.cc:922] calling sasl_server_start()
0923 22:16:25.905418 (+ 37us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0923 22:16:25.905431 (+ 13us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0923 22:16:25.905503 (+ 72us) server_negotiation.cc:1035] Waiting for connection context
0923 22:16:25.905613 (+ 110us) server_negotiation.cc:299] Negotiation successful
0923 22:16:25.905638 (+ 25us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":118,"thread_start_us":64,"threads_started":1}
I20260923 22:16:25.906785 1567 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260923 22:16:25.911970 22599 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 22:16:25.908390 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.138:41003 (local address 127.0.0.1:39167)
0923 22:16:25.908429 (+ 39us) server_negotiation.cc:206] Beginning negotiation
0923 22:16:25.908430 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0923 22:16:25.908559 (+ 129us) server_negotiation.cc:407] Connection header received
0923 22:16:25.908615 (+ 56us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 22:16:25.908616 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 22:16:25.908627 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 22:16:25.908679 (+ 52us) server_negotiation.cc:226] Negotiated authn=SASL
0923 22:16:25.908821 (+ 142us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 22:16:25.909574 (+ 753us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 22:16:25.910609 (+ 1035us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 22:16:25.910924 (+ 315us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 22:16:25.910982 (+ 58us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 22:16:25.911709 (+ 727us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0923 22:16:25.911710 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0923 22:16:25.911710 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0923 22:16:25.911715 (+ 5us) server_negotiation.cc:922] calling sasl_server_start()
0923 22:16:25.911730 (+ 15us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0923 22:16:25.911734 (+ 4us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0923 22:16:25.911791 (+ 57us) server_negotiation.cc:1035] Waiting for connection context
0923 22:16:25.911883 (+ 92us) server_negotiation.cc:299] Negotiation successful
0923 22:16:25.911895 (+ 12us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":19}
I20260923 22:16:25.914161 22588 catalog_manager.cc:1370] Generated new certificate authority record
I20260923 22:16:25.914654 22588 catalog_manager.cc:1504] Loading token signing keys...
I20260923 22:16:25.921751 22588 catalog_manager.cc:5868] T 00000000000000000000000000000000 P bff6da0cc1e24a9aa633b75447f94435: Generated new TSK 0
I20260923 22:16:25.921979 22588 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260923 22:16:25.924726 1567 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskRiQe_a/build/release/bin/kudu with pid 22509
W20260923 22:16:25.928786 22494 connection.cc:535] client connection to 127.0.0.1:39167 recv error: Network error: failed to read from TLS socket (remote: 127.0.0.1:39167): Connection reset by peer (error 104)
2026-09-23T22:16:25Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/7 (1295 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/8
Loading random data
Initializing database '/tmp/dist-test-taskRiQe_a/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1790201539513534-1567-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Sep 23 22:16:25 dist-test-slave-s2fj krb5kdc[22608](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Sep 23 22:16:25 dist-test-slave-s2fj krb5kdc[22608](info): set up 2 sockets
Sep 23 22:16:25 dist-test-slave-s2fj krb5kdc[22608](info): commencing operation
krb5kdc: starting...
W20260923 22:16:26.935942 1567 mini_kdc.cc:121] Time spent starting KDC: real 0.993s user 0.003s sys 0.002s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskRiQe_a/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1790201539513534-1567-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskRiQe_a/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1790201539513534-1567-0/krb5kdc/test-user.keytab.
Sep 23 22:16:27 dist-test-slave-s2fj krb5kdc[22608](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790201787, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-09-23T22:16:27Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-09-23T22:16:27Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskRiQe_a/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1790201539513534-1567-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskRiQe_a/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1790201539513534-1567-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260923 22:16:27.046437 1567 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskRiQe_a/build/release/bin/kudu
/tmp/dist-test-taskRiQe_a/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskRiQe_a/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1790201539513534-1567-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskRiQe_a/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1790201539513534-1567-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskRiQe_a/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1790201539513534-1567-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskRiQe_a/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1790201539513534-1567-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:41271
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-taskRiQe_a/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1790201539513534-1567-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:43393
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:41271
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=disabled
--rpc_encrypt_loopback_connections=false with env {KRB5CCNAME=/tmp/dist-test-taskRiQe_a/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1790201539513534-1567-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskRiQe_a/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1790201539513534-1567-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskRiQe_a/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1790201539513534-1567-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260923 22:16:27.127435 22621 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260923 22:16:27.127684 22621 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260923 22:16:27.127722 22621 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260923 22:16:27.128973 22621 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260923 22:16:27.129043 22621 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260923 22:16:27.129071 22621 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260923 22:16:27.129114 22621 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260923 22:16:27.129156 22621 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260923 22:16:27.130671 22621 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:43393
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskRiQe_a/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1790201539513534-1567-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskRiQe_a/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1790201539513534-1567-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:41271
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskRiQe_a/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1790201539513534-1567-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:41271
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskRiQe_a/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1790201539513534-1567-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.22621
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskRiQe_a/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1790201539513534-1567-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 23 Sep 2026 20:53:37 UTC on ccycloud.quasar-rkviyx.root.comops.site
build id 4668083
I20260923 22:16:27.130966 22621 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260923 22:16:27.131214 22621 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260923 22:16:27.136046 22627 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260923 22:16:27.136234 22626 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260923 22:16:27.136046 22629 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260923 22:16:27.136336 22621 server_base.cc:1034] running on GCE node
I20260923 22:16:27.136644 22621 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260923 22:16:27.136976 22621 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260923 22:16:27.138149 22621 hybrid_clock.cc:648] HybridClock initialized: now 1790201787138134 us; error 49 us; skew 500 ppm
Sep 23 22:16:27 dist-test-slave-s2fj krb5kdc[22608](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790201787, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260923 22:16:27.140691 22621 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260923 22:16:27.141088 22621 webserver.cc:460] Webserver started at http://127.0.0.1:46379/ using document root <none> and password file <none>
I20260923 22:16:27.141283 22621 fs_manager.cc:359] Metadata directory not provided
I20260923 22:16:27.141323 22621 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260923 22:16:27.141410 22621 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260923 22:16:27.142488 22621 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskRiQe_a/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1790201539513534-1567-0/minicluster-data/master-0/data/instance:
uuid: "4122a41652db43eb9e6afa1647c2403b"
format_stamp: "Formatted at 2026-09-23 22:16:27 on dist-test-slave-s2fj"
server_key: "6e8d2c0e6cad1c92987373736835d6b9"
server_key_iv: "a8cddcc75e138c91e3fa7cf0f0fa2623"
server_key_version: "encryptionkey@0"
I20260923 22:16:27.142827 22621 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskRiQe_a/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1790201539513534-1567-0/minicluster-data/master-0/wal/instance:
uuid: "4122a41652db43eb9e6afa1647c2403b"
format_stamp: "Formatted at 2026-09-23 22:16:27 on dist-test-slave-s2fj"
server_key: "6e8d2c0e6cad1c92987373736835d6b9"
server_key_iv: "a8cddcc75e138c91e3fa7cf0f0fa2623"
server_key_version: "encryptionkey@0"
I20260923 22:16:27.144178 22621 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.001s
I20260923 22:16:27.144866 22636 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260923 22:16:27.145018 22621 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260923 22:16:27.145112 22621 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskRiQe_a/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1790201539513534-1567-0/minicluster-data/master-0/data,/tmp/dist-test-taskRiQe_a/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1790201539513534-1567-0/minicluster-data/master-0/wal
uuid: "4122a41652db43eb9e6afa1647c2403b"
format_stamp: "Formatted at 2026-09-23 22:16:27 on dist-test-slave-s2fj"
server_key: "6e8d2c0e6cad1c92987373736835d6b9"
server_key_iv: "a8cddcc75e138c91e3fa7cf0f0fa2623"
server_key_version: "encryptionkey@0"
I20260923 22:16:27.145172 22621 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskRiQe_a/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1790201539513534-1567-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskRiQe_a/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1790201539513534-1567-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskRiQe_a/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1790201539513534-1567-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260923 22:16:27.148963 22621 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260923 22:16:27.151283 22621 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260923 22:16:27.151389 22621 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260923 22:16:27.155150 22621 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:41271
I20260923 22:16:27.155243 22688 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:41271 every 8 connection(s)
I20260923 22:16:27.155563 22621 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskRiQe_a/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1790201539513534-1567-0/minicluster-data/master-0/data/info.pb
I20260923 22:16:27.156642 22689 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260923 22:16:27.158918 22689 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 4122a41652db43eb9e6afa1647c2403b: Bootstrap starting.
I20260923 22:16:27.159523 22689 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 4122a41652db43eb9e6afa1647c2403b: Neither blocks nor log segments found. Creating new log.
I20260923 22:16:27.159798 22689 log.cc:824] T 00000000000000000000000000000000 P 4122a41652db43eb9e6afa1647c2403b: Log is configured to *not* fsync() on all Append() calls
I20260923 22:16:27.160283 1567 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskRiQe_a/build/release/bin/kudu as pid 22621
I20260923 22:16:27.160434 1567 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskRiQe_a/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1790201539513534-1567-0/minicluster-data/master-0/wal/instance
I20260923 22:16:27.160635 1567 external_mini_cluster.cc:1428] Setting key 44a70624468736b8b2595959421ffc93
I20260923 22:16:27.160694 22689 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 4122a41652db43eb9e6afa1647c2403b: No bootstrap required, opened a new log
I20260923 22:16:27.163039 22689 raft_consensus.cc:348] T 00000000000000000000000000000000 P 4122a41652db43eb9e6afa1647c2403b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "4122a41652db43eb9e6afa1647c2403b" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 41271 } }
I20260923 22:16:27.163245 22689 raft_consensus.cc:374] T 00000000000000000000000000000000 P 4122a41652db43eb9e6afa1647c2403b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260923 22:16:27.163309 22689 raft_consensus.cc:729] T 00000000000000000000000000000000 P 4122a41652db43eb9e6afa1647c2403b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 4122a41652db43eb9e6afa1647c2403b, State: Initialized, Role: FOLLOWER
I20260923 22:16:27.163430 22689 consensus_queue.cc:260] T 00000000000000000000000000000000 P 4122a41652db43eb9e6afa1647c2403b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "4122a41652db43eb9e6afa1647c2403b" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 41271 } }
I20260923 22:16:27.163533 22689 raft_consensus.cc:388] T 00000000000000000000000000000000 P 4122a41652db43eb9e6afa1647c2403b [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260923 22:16:27.163561 22689 raft_consensus.cc:482] T 00000000000000000000000000000000 P 4122a41652db43eb9e6afa1647c2403b [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260923 22:16:27.163609 22689 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 4122a41652db43eb9e6afa1647c2403b [term 0 FOLLOWER]: Advancing to term 1
I20260923 22:16:27.163661 22693 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 22:16:27.161229 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.138:37635 (local address 127.0.0.1:41271)
0923 22:16:27.161424 (+ 195us) server_negotiation.cc:206] Beginning negotiation
0923 22:16:27.161434 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0923 22:16:27.161472 (+ 38us) server_negotiation.cc:407] Connection header received
0923 22:16:27.162169 (+ 697us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 22:16:27.162177 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 22:16:27.162213 (+ 36us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 22:16:27.162367 (+ 154us) server_negotiation.cc:226] Negotiated authn=SASL
0923 22:16:27.163006 (+ 639us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0923 22:16:27.163009 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0923 22:16:27.163009 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0923 22:16:27.163014 (+ 5us) server_negotiation.cc:922] calling sasl_server_start()
0923 22:16:27.163058 (+ 44us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0923 22:16:27.163073 (+ 15us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0923 22:16:27.163146 (+ 73us) server_negotiation.cc:1035] Waiting for connection context
0923 22:16:27.163199 (+ 53us) server_negotiation.cc:299] Negotiation successful
0923 22:16:27.163240 (+ 41us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":133,"thread_start_us":79,"threads_started":1}
I20260923 22:16:27.164388 22689 raft_consensus.cc:504] T 00000000000000000000000000000000 P 4122a41652db43eb9e6afa1647c2403b [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "4122a41652db43eb9e6afa1647c2403b" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 41271 } }
I20260923 22:16:27.164530 22689 leader_election.cc:302] T 00000000000000000000000000000000 P 4122a41652db43eb9e6afa1647c2403b [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 4122a41652db43eb9e6afa1647c2403b; no voters:
I20260923 22:16:27.164654 1567 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260923 22:16:27.164693 22689 leader_election.cc:288] T 00000000000000000000000000000000 P 4122a41652db43eb9e6afa1647c2403b [CANDIDATE]: Term 1 election: Requested vote from peers
I20260923 22:16:27.164716 22694 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 4122a41652db43eb9e6afa1647c2403b [term 1 FOLLOWER]: Leader election won for term 1
I20260923 22:16:27.164872 22694 raft_consensus.cc:686] T 00000000000000000000000000000000 P 4122a41652db43eb9e6afa1647c2403b [term 1 LEADER]: Becoming Leader. State: Replica: 4122a41652db43eb9e6afa1647c2403b, State: Running, Role: LEADER
I20260923 22:16:27.164938 22689 sys_catalog.cc:564] T 00000000000000000000000000000000 P 4122a41652db43eb9e6afa1647c2403b [sys.catalog]: configured and running, proceeding with master startup.
I20260923 22:16:27.165030 22694 consensus_queue.cc:237] T 00000000000000000000000000000000 P 4122a41652db43eb9e6afa1647c2403b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "4122a41652db43eb9e6afa1647c2403b" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 41271 } }
I20260923 22:16:27.165407 22694 sys_catalog.cc:455] T 00000000000000000000000000000000 P 4122a41652db43eb9e6afa1647c2403b [sys.catalog]: SysCatalogTable state changed. Reason: New leader 4122a41652db43eb9e6afa1647c2403b. Latest consensus state: current_term: 1 leader_uuid: "4122a41652db43eb9e6afa1647c2403b" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "4122a41652db43eb9e6afa1647c2403b" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 41271 } } }
I20260923 22:16:27.165673 22694 sys_catalog.cc:458] T 00000000000000000000000000000000 P 4122a41652db43eb9e6afa1647c2403b [sys.catalog]: This master's current role is: LEADER
I20260923 22:16:27.165576 22696 sys_catalog.cc:455] T 00000000000000000000000000000000 P 4122a41652db43eb9e6afa1647c2403b [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "4122a41652db43eb9e6afa1647c2403b" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "4122a41652db43eb9e6afa1647c2403b" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 41271 } } }
I20260923 22:16:27.166083 22696 sys_catalog.cc:458] T 00000000000000000000000000000000 P 4122a41652db43eb9e6afa1647c2403b [sys.catalog]: This master's current role is: LEADER
I20260923 22:16:27.166201 22706 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260923 22:16:27.166828 22706 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260923 22:16:27.167958 22693 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 22:16:27.166687 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.138:54709 (local address 127.0.0.1:41271)
0923 22:16:27.166725 (+ 38us) server_negotiation.cc:206] Beginning negotiation
0923 22:16:27.166727 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0923 22:16:27.166847 (+ 120us) server_negotiation.cc:407] Connection header received
0923 22:16:27.166949 (+ 102us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 22:16:27.166950 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 22:16:27.166955 (+ 5us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 22:16:27.166984 (+ 29us) server_negotiation.cc:226] Negotiated authn=SASL
0923 22:16:27.167695 (+ 711us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0923 22:16:27.167696 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0923 22:16:27.167696 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0923 22:16:27.167701 (+ 5us) server_negotiation.cc:922] calling sasl_server_start()
0923 22:16:27.167716 (+ 15us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0923 22:16:27.167720 (+ 4us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0923 22:16:27.167782 (+ 62us) server_negotiation.cc:1035] Waiting for connection context
0923 22:16:27.167881 (+ 99us) server_negotiation.cc:299] Negotiation successful
0923 22:16:27.167897 (+ 16us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":18}
I20260923 22:16:27.168402 22706 catalog_manager.cc:1347] Generated new cluster ID: d299a45679724b10bd64255cc4563c7c
I20260923 22:16:27.168491 22706 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260923 22:16:27.183835 22706 catalog_manager.cc:1370] Generated new certificate authority record
I20260923 22:16:27.184048 22706 catalog_manager.cc:1504] Loading token signing keys...
I20260923 22:16:27.187268 22706 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 4122a41652db43eb9e6afa1647c2403b: Generated new TSK 0
I20260923 22:16:27.187465 22706 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260923 22:16:27.196868 1567 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskRiQe_a/build/release/bin/kudu with pid 22621
2026-09-23T22:16:27Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/8 (1272 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/9
Loading random data
Initializing database '/tmp/dist-test-taskRiQe_a/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1790201539513534-1567-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Sep 23 22:16:27 dist-test-slave-s2fj krb5kdc[22720](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Sep 23 22:16:27 dist-test-slave-s2fj krb5kdc[22720](info): set up 2 sockets
Sep 23 22:16:27 dist-test-slave-s2fj krb5kdc[22720](info): commencing operation
krb5kdc: starting...
W20260923 22:16:28.232197 1567 mini_kdc.cc:121] Time spent starting KDC: real 1.017s user 0.000s sys 0.005s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskRiQe_a/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1790201539513534-1567-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskRiQe_a/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1790201539513534-1567-0/krb5kdc/test-user.keytab.
Sep 23 22:16:28 dist-test-slave-s2fj krb5kdc[22720](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790201788, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-09-23T22:16:28Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-09-23T22:16:28Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskRiQe_a/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1790201539513534-1567-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskRiQe_a/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1790201539513534-1567-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260923 22:16:28.343132 1567 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskRiQe_a/build/release/bin/kudu
/tmp/dist-test-taskRiQe_a/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskRiQe_a/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1790201539513534-1567-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskRiQe_a/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1790201539513534-1567-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskRiQe_a/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1790201539513534-1567-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskRiQe_a/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1790201539513534-1567-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:46131
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-taskRiQe_a/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1790201539513534-1567-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:36943
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:46131
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=required
--rpc_encryption=required
--rpc_encrypt_loopback_connections=true with env {KRB5CCNAME=/tmp/dist-test-taskRiQe_a/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1790201539513534-1567-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskRiQe_a/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1790201539513534-1567-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskRiQe_a/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1790201539513534-1567-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260923 22:16:28.420463 22733 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260923 22:16:28.420714 22733 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260923 22:16:28.420755 22733 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260923 22:16:28.421887 22733 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260923 22:16:28.421955 22733 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260923 22:16:28.421979 22733 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260923 22:16:28.422009 22733 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260923 22:16:28.422030 22733 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260923 22:16:28.423504 22733 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:36943
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskRiQe_a/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1790201539513534-1567-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskRiQe_a/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1790201539513534-1567-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:46131
--rpc_encrypt_loopback_connections=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskRiQe_a/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1790201539513534-1567-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:46131
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskRiQe_a/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1790201539513534-1567-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.22733
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskRiQe_a/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1790201539513534-1567-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 23 Sep 2026 20:53:37 UTC on ccycloud.quasar-rkviyx.root.comops.site
build id 4668083
I20260923 22:16:28.423756 22733 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260923 22:16:28.423982 22733 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260923 22:16:28.428715 22738 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260923 22:16:28.428774 22739 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260923 22:16:28.428772 22741 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260923 22:16:28.428808 22733 server_base.cc:1034] running on GCE node
I20260923 22:16:28.429129 22733 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260923 22:16:28.429407 22733 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260923 22:16:28.430546 22733 hybrid_clock.cc:648] HybridClock initialized: now 1790201788430527 us; error 35 us; skew 500 ppm
Sep 23 22:16:28 dist-test-slave-s2fj krb5kdc[22720](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790201788, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260923 22:16:28.436338 22733 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260923 22:16:28.436774 22733 webserver.cc:460] Webserver started at http://127.0.0.1:35791/ using document root <none> and password file <none>
I20260923 22:16:28.437004 22733 fs_manager.cc:359] Metadata directory not provided
I20260923 22:16:28.437070 22733 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260923 22:16:28.437216 22733 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260923 22:16:28.438206 22733 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskRiQe_a/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1790201539513534-1567-0/minicluster-data/master-0/data/instance:
uuid: "12df06db05c14b06b581ad692bf4be53"
format_stamp: "Formatted at 2026-09-23 22:16:28 on dist-test-slave-s2fj"
server_key: "26c2deed4a447a595b5a9326c0b68bae"
server_key_iv: "5feddb1e807cd6d7bfb83ec113f3ab2c"
server_key_version: "encryptionkey@0"
I20260923 22:16:28.438575 22733 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskRiQe_a/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1790201539513534-1567-0/minicluster-data/master-0/wal/instance:
uuid: "12df06db05c14b06b581ad692bf4be53"
format_stamp: "Formatted at 2026-09-23 22:16:28 on dist-test-slave-s2fj"
server_key: "26c2deed4a447a595b5a9326c0b68bae"
server_key_iv: "5feddb1e807cd6d7bfb83ec113f3ab2c"
server_key_version: "encryptionkey@0"
I20260923 22:16:28.439847 22733 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.001s sys 0.002s
I20260923 22:16:28.440637 22748 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260923 22:16:28.440802 22733 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.000s
I20260923 22:16:28.440894 22733 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskRiQe_a/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1790201539513534-1567-0/minicluster-data/master-0/data,/tmp/dist-test-taskRiQe_a/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1790201539513534-1567-0/minicluster-data/master-0/wal
uuid: "12df06db05c14b06b581ad692bf4be53"
format_stamp: "Formatted at 2026-09-23 22:16:28 on dist-test-slave-s2fj"
server_key: "26c2deed4a447a595b5a9326c0b68bae"
server_key_iv: "5feddb1e807cd6d7bfb83ec113f3ab2c"
server_key_version: "encryptionkey@0"
I20260923 22:16:28.440973 22733 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskRiQe_a/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1790201539513534-1567-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskRiQe_a/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1790201539513534-1567-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskRiQe_a/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1790201539513534-1567-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260923 22:16:28.452868 22733 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260923 22:16:28.454463 22733 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260923 22:16:28.454623 22733 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260923 22:16:28.458546 22733 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:46131
I20260923 22:16:28.458580 22800 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:46131 every 8 connection(s)
I20260923 22:16:28.458916 22733 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskRiQe_a/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1790201539513534-1567-0/minicluster-data/master-0/data/info.pb
I20260923 22:16:28.459797 22801 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260923 22:16:28.461925 22801 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 12df06db05c14b06b581ad692bf4be53: Bootstrap starting.
I20260923 22:16:28.462488 22801 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 12df06db05c14b06b581ad692bf4be53: Neither blocks nor log segments found. Creating new log.
I20260923 22:16:28.462771 22801 log.cc:824] T 00000000000000000000000000000000 P 12df06db05c14b06b581ad692bf4be53: Log is configured to *not* fsync() on all Append() calls
I20260923 22:16:28.463406 22801 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 12df06db05c14b06b581ad692bf4be53: No bootstrap required, opened a new log
I20260923 22:16:28.464923 22801 raft_consensus.cc:348] T 00000000000000000000000000000000 P 12df06db05c14b06b581ad692bf4be53 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "12df06db05c14b06b581ad692bf4be53" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 46131 } }
I20260923 22:16:28.465051 22801 raft_consensus.cc:374] T 00000000000000000000000000000000 P 12df06db05c14b06b581ad692bf4be53 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260923 22:16:28.465075 22801 raft_consensus.cc:729] T 00000000000000000000000000000000 P 12df06db05c14b06b581ad692bf4be53 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 12df06db05c14b06b581ad692bf4be53, State: Initialized, Role: FOLLOWER
I20260923 22:16:28.465267 22801 consensus_queue.cc:260] T 00000000000000000000000000000000 P 12df06db05c14b06b581ad692bf4be53 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "12df06db05c14b06b581ad692bf4be53" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 46131 } }
I20260923 22:16:28.465353 22801 raft_consensus.cc:388] T 00000000000000000000000000000000 P 12df06db05c14b06b581ad692bf4be53 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260923 22:16:28.465394 22801 raft_consensus.cc:482] T 00000000000000000000000000000000 P 12df06db05c14b06b581ad692bf4be53 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260923 22:16:28.465443 22801 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 12df06db05c14b06b581ad692bf4be53 [term 0 FOLLOWER]: Advancing to term 1
I20260923 22:16:28.465934 22801 raft_consensus.cc:504] T 00000000000000000000000000000000 P 12df06db05c14b06b581ad692bf4be53 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "12df06db05c14b06b581ad692bf4be53" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 46131 } }
I20260923 22:16:28.466041 22801 leader_election.cc:302] T 00000000000000000000000000000000 P 12df06db05c14b06b581ad692bf4be53 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 12df06db05c14b06b581ad692bf4be53; no voters:
I20260923 22:16:28.466220 22801 leader_election.cc:288] T 00000000000000000000000000000000 P 12df06db05c14b06b581ad692bf4be53 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260923 22:16:28.466252 22804 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 12df06db05c14b06b581ad692bf4be53 [term 1 FOLLOWER]: Leader election won for term 1
I20260923 22:16:28.466413 22804 raft_consensus.cc:686] T 00000000000000000000000000000000 P 12df06db05c14b06b581ad692bf4be53 [term 1 LEADER]: Becoming Leader. State: Replica: 12df06db05c14b06b581ad692bf4be53, State: Running, Role: LEADER
I20260923 22:16:28.466466 22801 sys_catalog.cc:564] T 00000000000000000000000000000000 P 12df06db05c14b06b581ad692bf4be53 [sys.catalog]: configured and running, proceeding with master startup.
I20260923 22:16:28.466544 22804 consensus_queue.cc:237] T 00000000000000000000000000000000 P 12df06db05c14b06b581ad692bf4be53 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "12df06db05c14b06b581ad692bf4be53" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 46131 } }
I20260923 22:16:28.467124 22806 sys_catalog.cc:455] T 00000000000000000000000000000000 P 12df06db05c14b06b581ad692bf4be53 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 12df06db05c14b06b581ad692bf4be53. Latest consensus state: current_term: 1 leader_uuid: "12df06db05c14b06b581ad692bf4be53" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "12df06db05c14b06b581ad692bf4be53" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 46131 } } }
I20260923 22:16:28.467228 22806 sys_catalog.cc:458] T 00000000000000000000000000000000 P 12df06db05c14b06b581ad692bf4be53 [sys.catalog]: This master's current role is: LEADER
I20260923 22:16:28.467298 1567 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskRiQe_a/build/release/bin/kudu as pid 22733
I20260923 22:16:28.467418 1567 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskRiQe_a/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1790201539513534-1567-0/minicluster-data/master-0/wal/instance
I20260923 22:16:28.467592 1567 external_mini_cluster.cc:1428] Setting key 0ce8f4c7606e50737170b90cea9ca184
I20260923 22:16:28.467600 22813 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260923 22:16:28.467622 22805 sys_catalog.cc:455] T 00000000000000000000000000000000 P 12df06db05c14b06b581ad692bf4be53 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "12df06db05c14b06b581ad692bf4be53" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "12df06db05c14b06b581ad692bf4be53" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 46131 } } }
I20260923 22:16:28.467711 22805 sys_catalog.cc:458] T 00000000000000000000000000000000 P 12df06db05c14b06b581ad692bf4be53 [sys.catalog]: This master's current role is: LEADER
I20260923 22:16:28.468120 22813 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260923 22:16:28.469563 22813 catalog_manager.cc:1347] Generated new cluster ID: 9267883aff654484a22bbeb91b4e341c
I20260923 22:16:28.469610 22813 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
Sep 23 22:16:28 dist-test-slave-s2fj krb5kdc[22720](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790201788, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.0.0.1@KRBTEST.COM
I20260923 22:16:28.481956 22813 catalog_manager.cc:1370] Generated new certificate authority record
I20260923 22:16:28.482623 22813 catalog_manager.cc:1504] Loading token signing keys...
I20260923 22:16:28.482952 22823 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 22:16:28.468166 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.138:41791 (local address 127.0.0.1:46131)
0923 22:16:28.472183 (+ 4017us) server_negotiation.cc:206] Beginning negotiation
0923 22:16:28.472191 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0923 22:16:28.472222 (+ 31us) server_negotiation.cc:407] Connection header received
0923 22:16:28.472323 (+ 101us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 22:16:28.472328 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 22:16:28.472389 (+ 61us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 22:16:28.472515 (+ 126us) server_negotiation.cc:226] Negotiated authn=SASL
0923 22:16:28.473313 (+ 798us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 22:16:28.474663 (+ 1350us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 22:16:28.475783 (+ 1120us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 22:16:28.476379 (+ 596us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 22:16:28.476469 (+ 90us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 22:16:28.479231 (+ 2762us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0923 22:16:28.479235 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0923 22:16:28.479235 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0923 22:16:28.479247 (+ 12us) server_negotiation.cc:922] calling sasl_server_start()
0923 22:16:28.481158 (+ 1911us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0923 22:16:28.481628 (+ 470us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0923 22:16:28.481631 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0923 22:16:28.481631 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0923 22:16:28.481745 (+ 114us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0923 22:16:28.482035 (+ 290us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0923 22:16:28.482036 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0923 22:16:28.482036 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0923 22:16:28.482313 (+ 277us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0923 22:16:28.482442 (+ 129us) server_negotiation.cc:1035] Waiting for connection context
0923 22:16:28.482634 (+ 192us) server_negotiation.cc:299] Negotiation successful
0923 22:16:28.482742 (+ 108us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":3970,"thread_start_us":76,"threads_started":1}
I20260923 22:16:28.483937 1567 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260923 22:16:28.490960 22813 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 12df06db05c14b06b581ad692bf4be53: Generated new TSK 0
I20260923 22:16:28.491143 22813 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260923 22:16:28.492728 22823 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 22:16:28.485424 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.138:59925 (local address 127.0.0.1:46131)
0923 22:16:28.485478 (+ 54us) server_negotiation.cc:206] Beginning negotiation
0923 22:16:28.485479 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0923 22:16:28.485675 (+ 196us) server_negotiation.cc:407] Connection header received
0923 22:16:28.485703 (+ 28us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 22:16:28.485704 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 22:16:28.485715 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 22:16:28.485779 (+ 64us) server_negotiation.cc:226] Negotiated authn=SASL
0923 22:16:28.486328 (+ 549us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 22:16:28.487079 (+ 751us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 22:16:28.488040 (+ 961us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 22:16:28.488731 (+ 691us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 22:16:28.488804 (+ 73us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 22:16:28.490039 (+ 1235us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0923 22:16:28.490041 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0923 22:16:28.490041 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0923 22:16:28.490048 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0923 22:16:28.491600 (+ 1552us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0923 22:16:28.491934 (+ 334us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0923 22:16:28.491935 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0923 22:16:28.491935 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0923 22:16:28.491991 (+ 56us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0923 22:16:28.492262 (+ 271us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0923 22:16:28.492274 (+ 12us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0923 22:16:28.492275 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0923 22:16:28.492394 (+ 119us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0923 22:16:28.492490 (+ 96us) server_negotiation.cc:1035] Waiting for connection context
0923 22:16:28.492585 (+ 95us) server_negotiation.cc:299] Negotiation successful
0923 22:16:28.492643 (+ 58us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":33}
I20260923 22:16:28.494760 1567 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskRiQe_a/build/release/bin/kudu with pid 22733
W20260923 22:16:28.498513 22718 connection.cc:535] client connection to 127.0.0.1:46131 recv error: Network error: failed to read from TLS socket (remote: 127.0.0.1:46131): Connection reset by peer (error 104)
2026-09-23T22:16:28Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/9 (1296 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/10
Loading random data
Initializing database '/tmp/dist-test-taskRiQe_a/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1790201539513534-1567-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Sep 23 22:16:28 dist-test-slave-s2fj krb5kdc[22832](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Sep 23 22:16:28 dist-test-slave-s2fj krb5kdc[22832](info): set up 2 sockets
Sep 23 22:16:28 dist-test-slave-s2fj krb5kdc[22832](info): commencing operation
krb5kdc: starting...
W20260923 22:16:29.504350 1567 mini_kdc.cc:121] Time spent starting KDC: real 0.992s user 0.002s sys 0.003s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskRiQe_a/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1790201539513534-1567-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskRiQe_a/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1790201539513534-1567-0/krb5kdc/test-user.keytab.
Sep 23 22:16:29 dist-test-slave-s2fj krb5kdc[22832](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790201789, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-09-23T22:16:29Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-09-23T22:16:29Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskRiQe_a/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1790201539513534-1567-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskRiQe_a/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1790201539513534-1567-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260923 22:16:29.613018 1567 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskRiQe_a/build/release/bin/kudu
/tmp/dist-test-taskRiQe_a/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskRiQe_a/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1790201539513534-1567-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskRiQe_a/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1790201539513534-1567-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskRiQe_a/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1790201539513534-1567-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskRiQe_a/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1790201539513534-1567-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:39505
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-taskRiQe_a/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1790201539513534-1567-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:43337
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:39505
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=required
--rpc_encrypt_loopback_connections=true with env {KRB5CCNAME=/tmp/dist-test-taskRiQe_a/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1790201539513534-1567-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskRiQe_a/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1790201539513534-1567-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskRiQe_a/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1790201539513534-1567-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260923 22:16:29.691455 22845 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260923 22:16:29.691675 22845 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260923 22:16:29.691715 22845 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260923 22:16:29.692906 22845 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260923 22:16:29.692972 22845 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260923 22:16:29.692999 22845 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260923 22:16:29.693028 22845 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260923 22:16:29.693058 22845 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260923 22:16:29.694450 22845 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:43337
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskRiQe_a/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1790201539513534-1567-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskRiQe_a/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1790201539513534-1567-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:39505
--rpc_encrypt_loopback_connections=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskRiQe_a/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1790201539513534-1567-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:39505
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskRiQe_a/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1790201539513534-1567-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.22845
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskRiQe_a/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1790201539513534-1567-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 23 Sep 2026 20:53:37 UTC on ccycloud.quasar-rkviyx.root.comops.site
build id 4668083
I20260923 22:16:29.694733 22845 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260923 22:16:29.694957 22845 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260923 22:16:29.699436 22850 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260923 22:16:29.699446 22851 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260923 22:16:29.699446 22853 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260923 22:16:29.699638 22845 server_base.cc:1034] running on GCE node
I20260923 22:16:29.699921 22845 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260923 22:16:29.700222 22845 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260923 22:16:29.701393 22845 hybrid_clock.cc:648] HybridClock initialized: now 1790201789701373 us; error 33 us; skew 500 ppm
Sep 23 22:16:29 dist-test-slave-s2fj krb5kdc[22832](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790201789, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260923 22:16:29.703770 22845 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260923 22:16:29.704265 22845 webserver.cc:460] Webserver started at http://127.0.0.1:32961/ using document root <none> and password file <none>
I20260923 22:16:29.704484 22845 fs_manager.cc:359] Metadata directory not provided
I20260923 22:16:29.704548 22845 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260923 22:16:29.704681 22845 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260923 22:16:29.705600 22845 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskRiQe_a/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1790201539513534-1567-0/minicluster-data/master-0/data/instance:
uuid: "351938c63fe44104a3b57b396f1a8260"
format_stamp: "Formatted at 2026-09-23 22:16:29 on dist-test-slave-s2fj"
server_key: "7181cbeae037588b70ee21e85b560b9d"
server_key_iv: "7dac4ebf146c658d39bcbb2b5443a88d"
server_key_version: "encryptionkey@0"
I20260923 22:16:29.705974 22845 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskRiQe_a/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1790201539513534-1567-0/minicluster-data/master-0/wal/instance:
uuid: "351938c63fe44104a3b57b396f1a8260"
format_stamp: "Formatted at 2026-09-23 22:16:29 on dist-test-slave-s2fj"
server_key: "7181cbeae037588b70ee21e85b560b9d"
server_key_iv: "7dac4ebf146c658d39bcbb2b5443a88d"
server_key_version: "encryptionkey@0"
I20260923 22:16:29.707242 22845 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260923 22:16:29.707875 22860 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260923 22:16:29.708041 22845 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260923 22:16:29.708158 22845 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskRiQe_a/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1790201539513534-1567-0/minicluster-data/master-0/data,/tmp/dist-test-taskRiQe_a/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1790201539513534-1567-0/minicluster-data/master-0/wal
uuid: "351938c63fe44104a3b57b396f1a8260"
format_stamp: "Formatted at 2026-09-23 22:16:29 on dist-test-slave-s2fj"
server_key: "7181cbeae037588b70ee21e85b560b9d"
server_key_iv: "7dac4ebf146c658d39bcbb2b5443a88d"
server_key_version: "encryptionkey@0"
I20260923 22:16:29.708244 22845 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskRiQe_a/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1790201539513534-1567-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskRiQe_a/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1790201539513534-1567-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskRiQe_a/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1790201539513534-1567-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260923 22:16:29.724037 22845 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260923 22:16:29.726150 22845 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260923 22:16:29.726300 22845 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260923 22:16:29.730080 22845 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:39505
I20260923 22:16:29.730109 22912 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:39505 every 8 connection(s)
I20260923 22:16:29.730413 22845 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskRiQe_a/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1790201539513534-1567-0/minicluster-data/master-0/data/info.pb
I20260923 22:16:29.731158 22913 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260923 22:16:29.733268 22913 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 351938c63fe44104a3b57b396f1a8260: Bootstrap starting.
I20260923 22:16:29.733773 22913 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 351938c63fe44104a3b57b396f1a8260: Neither blocks nor log segments found. Creating new log.
I20260923 22:16:29.733994 22913 log.cc:824] T 00000000000000000000000000000000 P 351938c63fe44104a3b57b396f1a8260: Log is configured to *not* fsync() on all Append() calls
I20260923 22:16:29.734584 22913 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 351938c63fe44104a3b57b396f1a8260: No bootstrap required, opened a new log
I20260923 22:16:29.735994 22913 raft_consensus.cc:348] T 00000000000000000000000000000000 P 351938c63fe44104a3b57b396f1a8260 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "351938c63fe44104a3b57b396f1a8260" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 39505 } }
I20260923 22:16:29.736135 22913 raft_consensus.cc:374] T 00000000000000000000000000000000 P 351938c63fe44104a3b57b396f1a8260 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260923 22:16:29.736158 22913 raft_consensus.cc:729] T 00000000000000000000000000000000 P 351938c63fe44104a3b57b396f1a8260 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 351938c63fe44104a3b57b396f1a8260, State: Initialized, Role: FOLLOWER
I20260923 22:16:29.736279 22913 consensus_queue.cc:260] T 00000000000000000000000000000000 P 351938c63fe44104a3b57b396f1a8260 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "351938c63fe44104a3b57b396f1a8260" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 39505 } }
I20260923 22:16:29.736356 22913 raft_consensus.cc:388] T 00000000000000000000000000000000 P 351938c63fe44104a3b57b396f1a8260 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260923 22:16:29.736377 22913 raft_consensus.cc:482] T 00000000000000000000000000000000 P 351938c63fe44104a3b57b396f1a8260 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260923 22:16:29.736404 22913 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 351938c63fe44104a3b57b396f1a8260 [term 0 FOLLOWER]: Advancing to term 1
I20260923 22:16:29.736840 22913 raft_consensus.cc:504] T 00000000000000000000000000000000 P 351938c63fe44104a3b57b396f1a8260 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "351938c63fe44104a3b57b396f1a8260" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 39505 } }
I20260923 22:16:29.736948 22913 leader_election.cc:302] T 00000000000000000000000000000000 P 351938c63fe44104a3b57b396f1a8260 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 351938c63fe44104a3b57b396f1a8260; no voters:
I20260923 22:16:29.737092 1567 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskRiQe_a/build/release/bin/kudu as pid 22845
I20260923 22:16:29.737112 22913 leader_election.cc:288] T 00000000000000000000000000000000 P 351938c63fe44104a3b57b396f1a8260 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260923 22:16:29.737134 22916 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 351938c63fe44104a3b57b396f1a8260 [term 1 FOLLOWER]: Leader election won for term 1
I20260923 22:16:29.737226 1567 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskRiQe_a/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1790201539513534-1567-0/minicluster-data/master-0/wal/instance
I20260923 22:16:29.737347 22916 raft_consensus.cc:686] T 00000000000000000000000000000000 P 351938c63fe44104a3b57b396f1a8260 [term 1 LEADER]: Becoming Leader. State: Replica: 351938c63fe44104a3b57b396f1a8260, State: Running, Role: LEADER
I20260923 22:16:29.737376 22913 sys_catalog.cc:564] T 00000000000000000000000000000000 P 351938c63fe44104a3b57b396f1a8260 [sys.catalog]: configured and running, proceeding with master startup.
I20260923 22:16:29.737443 1567 external_mini_cluster.cc:1428] Setting key 5babe1c0ca1d72a15ac40bc2717c21b7
I20260923 22:16:29.737426 22916 consensus_queue.cc:237] T 00000000000000000000000000000000 P 351938c63fe44104a3b57b396f1a8260 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "351938c63fe44104a3b57b396f1a8260" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 39505 } }
I20260923 22:16:29.737733 22916 sys_catalog.cc:455] T 00000000000000000000000000000000 P 351938c63fe44104a3b57b396f1a8260 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 351938c63fe44104a3b57b396f1a8260. Latest consensus state: current_term: 1 leader_uuid: "351938c63fe44104a3b57b396f1a8260" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "351938c63fe44104a3b57b396f1a8260" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 39505 } } }
I20260923 22:16:29.737818 22916 sys_catalog.cc:458] T 00000000000000000000000000000000 P 351938c63fe44104a3b57b396f1a8260 [sys.catalog]: This master's current role is: LEADER
I20260923 22:16:29.738178 22917 sys_catalog.cc:455] T 00000000000000000000000000000000 P 351938c63fe44104a3b57b396f1a8260 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "351938c63fe44104a3b57b396f1a8260" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "351938c63fe44104a3b57b396f1a8260" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 39505 } } }
I20260923 22:16:29.738319 22917 sys_catalog.cc:458] T 00000000000000000000000000000000 P 351938c63fe44104a3b57b396f1a8260 [sys.catalog]: This master's current role is: LEADER
I20260923 22:16:29.738251 22922 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260923 22:16:29.739090 22922 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260923 22:16:29.740697 22922 catalog_manager.cc:1347] Generated new cluster ID: b19be9a844854cb28f00d7646d5dc24e
I20260923 22:16:29.740746 22922 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260923 22:16:29.744432 22927 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 22:16:29.738186 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.138:37935 (local address 127.0.0.1:39505)
0923 22:16:29.738746 (+ 560us) server_negotiation.cc:206] Beginning negotiation
0923 22:16:29.738756 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0923 22:16:29.738791 (+ 35us) server_negotiation.cc:407] Connection header received
0923 22:16:29.738919 (+ 128us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 22:16:29.738926 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 22:16:29.738996 (+ 70us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 22:16:29.739131 (+ 135us) server_negotiation.cc:226] Negotiated authn=SASL
0923 22:16:29.739509 (+ 378us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 22:16:29.740981 (+ 1472us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 22:16:29.742091 (+ 1110us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 22:16:29.742659 (+ 568us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 22:16:29.743423 (+ 764us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 22:16:29.743546 (+ 123us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0923 22:16:29.743549 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0923 22:16:29.743549 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0923 22:16:29.743729 (+ 180us) server_negotiation.cc:922] calling sasl_server_start()
0923 22:16:29.743761 (+ 32us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0923 22:16:29.743775 (+ 14us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0923 22:16:29.743983 (+ 208us) server_negotiation.cc:1035] Waiting for connection context
0923 22:16:29.744021 (+ 38us) server_negotiation.cc:299] Negotiation successful
0923 22:16:29.744044 (+ 23us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":506,"thread_start_us":71,"threads_started":1}
I20260923 22:16:29.745078 22922 catalog_manager.cc:1370] Generated new certificate authority record
I20260923 22:16:29.745337 1567 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260923 22:16:29.745704 22922 catalog_manager.cc:1504] Loading token signing keys...
I20260923 22:16:29.751751 22927 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 22:16:29.748356 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.138:55171 (local address 127.0.0.1:39505)
0923 22:16:29.748384 (+ 28us) server_negotiation.cc:206] Beginning negotiation
0923 22:16:29.748385 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0923 22:16:29.748508 (+ 123us) server_negotiation.cc:407] Connection header received
0923 22:16:29.748564 (+ 56us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 22:16:29.748564 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 22:16:29.748573 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 22:16:29.748618 (+ 45us) server_negotiation.cc:226] Negotiated authn=SASL
0923 22:16:29.748769 (+ 151us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 22:16:29.749475 (+ 706us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 22:16:29.750503 (+ 1028us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 22:16:29.750793 (+ 290us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 22:16:29.750846 (+ 53us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 22:16:29.751506 (+ 660us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0923 22:16:29.751508 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0923 22:16:29.751508 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0923 22:16:29.751512 (+ 4us) server_negotiation.cc:922] calling sasl_server_start()
0923 22:16:29.751524 (+ 12us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0923 22:16:29.751528 (+ 4us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0923 22:16:29.751582 (+ 54us) server_negotiation.cc:1035] Waiting for connection context
0923 22:16:29.751675 (+ 93us) server_negotiation.cc:299] Negotiation successful
0923 22:16:29.751686 (+ 11us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":13}
I20260923 22:16:29.756918 22922 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 351938c63fe44104a3b57b396f1a8260: Generated new TSK 0
I20260923 22:16:29.757076 22922 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260923 22:16:29.765309 1567 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskRiQe_a/build/release/bin/kudu with pid 22845
W20260923 22:16:29.768993 22830 connection.cc:535] client connection to 127.0.0.1:39505 recv error: Network error: failed to read from TLS socket (remote: 127.0.0.1:39505): Connection reset by peer (error 104)
2026-09-23T22:16:29Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/10 (1270 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/11
Loading random data
Initializing database '/tmp/dist-test-taskRiQe_a/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1790201539513534-1567-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Sep 23 22:16:29 dist-test-slave-s2fj krb5kdc[22944](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Sep 23 22:16:29 dist-test-slave-s2fj krb5kdc[22944](info): set up 2 sockets
Sep 23 22:16:29 dist-test-slave-s2fj krb5kdc[22944](info): commencing operation
krb5kdc: starting...
W20260923 22:16:30.784636 1567 mini_kdc.cc:121] Time spent starting KDC: real 1.002s user 0.000s sys 0.005s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskRiQe_a/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1790201539513534-1567-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskRiQe_a/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1790201539513534-1567-0/krb5kdc/test-user.keytab.
Sep 23 22:16:30 dist-test-slave-s2fj krb5kdc[22944](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790201790, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-09-23T22:16:30Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-09-23T22:16:30Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskRiQe_a/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1790201539513534-1567-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskRiQe_a/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1790201539513534-1567-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260923 22:16:30.903103 1567 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskRiQe_a/build/release/bin/kudu
/tmp/dist-test-taskRiQe_a/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskRiQe_a/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1790201539513534-1567-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskRiQe_a/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1790201539513534-1567-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskRiQe_a/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1790201539513534-1567-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskRiQe_a/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1790201539513534-1567-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:39743
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-taskRiQe_a/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1790201539513534-1567-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:33723
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:39743
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=disabled
--rpc_encrypt_loopback_connections=true with env {KRB5CCNAME=/tmp/dist-test-taskRiQe_a/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1790201539513534-1567-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskRiQe_a/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1790201539513534-1567-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskRiQe_a/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1790201539513534-1567-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260923 22:16:30.984884 22957 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260923 22:16:30.985149 22957 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260923 22:16:30.985191 22957 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260923 22:16:30.986385 22957 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260923 22:16:30.986460 22957 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260923 22:16:30.986485 22957 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260923 22:16:30.986521 22957 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260923 22:16:30.986567 22957 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260923 22:16:30.988169 22957 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:33723
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskRiQe_a/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1790201539513534-1567-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskRiQe_a/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1790201539513534-1567-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:39743
--rpc_encrypt_loopback_connections=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskRiQe_a/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1790201539513534-1567-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:39743
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskRiQe_a/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1790201539513534-1567-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.22957
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskRiQe_a/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1790201539513534-1567-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 23 Sep 2026 20:53:37 UTC on ccycloud.quasar-rkviyx.root.comops.site
build id 4668083
I20260923 22:16:30.988445 22957 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260923 22:16:30.988691 22957 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260923 22:16:30.993458 22965 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260923 22:16:30.993468 22962 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260923 22:16:30.993468 22963 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260923 22:16:30.993500 22957 server_base.cc:1034] running on GCE node
I20260923 22:16:30.993865 22957 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260923 22:16:30.994158 22957 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260923 22:16:30.995299 22957 hybrid_clock.cc:648] HybridClock initialized: now 1790201790995279 us; error 37 us; skew 500 ppm
Sep 23 22:16:30 dist-test-slave-s2fj krb5kdc[22944](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790201790, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260923 22:16:30.997967 22957 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260923 22:16:30.998359 22957 webserver.cc:460] Webserver started at http://127.0.0.1:40677/ using document root <none> and password file <none>
I20260923 22:16:30.998557 22957 fs_manager.cc:359] Metadata directory not provided
I20260923 22:16:30.998597 22957 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260923 22:16:30.998692 22957 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260923 22:16:30.999756 22957 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskRiQe_a/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1790201539513534-1567-0/minicluster-data/master-0/data/instance:
uuid: "45bdb42a5d4148d383ac026f230ea481"
format_stamp: "Formatted at 2026-09-23 22:16:30 on dist-test-slave-s2fj"
server_key: "ade1a7b498463a4f8c7024463b260e59"
server_key_iv: "21b6c74f19696d2867d4384cf61f2df2"
server_key_version: "encryptionkey@0"
I20260923 22:16:31.000124 22957 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskRiQe_a/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1790201539513534-1567-0/minicluster-data/master-0/wal/instance:
uuid: "45bdb42a5d4148d383ac026f230ea481"
format_stamp: "Formatted at 2026-09-23 22:16:30 on dist-test-slave-s2fj"
server_key: "ade1a7b498463a4f8c7024463b260e59"
server_key_iv: "21b6c74f19696d2867d4384cf61f2df2"
server_key_version: "encryptionkey@0"
I20260923 22:16:31.001395 22957 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.001s
I20260923 22:16:31.002094 22972 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260923 22:16:31.002296 22957 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.001s
I20260923 22:16:31.002388 22957 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskRiQe_a/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1790201539513534-1567-0/minicluster-data/master-0/data,/tmp/dist-test-taskRiQe_a/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1790201539513534-1567-0/minicluster-data/master-0/wal
uuid: "45bdb42a5d4148d383ac026f230ea481"
format_stamp: "Formatted at 2026-09-23 22:16:30 on dist-test-slave-s2fj"
server_key: "ade1a7b498463a4f8c7024463b260e59"
server_key_iv: "21b6c74f19696d2867d4384cf61f2df2"
server_key_version: "encryptionkey@0"
I20260923 22:16:31.002444 22957 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskRiQe_a/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1790201539513534-1567-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskRiQe_a/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1790201539513534-1567-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskRiQe_a/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1790201539513534-1567-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260923 22:16:31.006191 22957 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260923 22:16:31.008519 22957 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260923 22:16:31.008628 22957 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260923 22:16:31.012310 22957 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:39743
I20260923 22:16:31.012377 23024 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:39743 every 8 connection(s)
I20260923 22:16:31.012722 22957 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskRiQe_a/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1790201539513534-1567-0/minicluster-data/master-0/data/info.pb
I20260923 22:16:31.013733 23025 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260923 22:16:31.015991 23025 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 45bdb42a5d4148d383ac026f230ea481: Bootstrap starting.
I20260923 22:16:31.016603 23025 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 45bdb42a5d4148d383ac026f230ea481: Neither blocks nor log segments found. Creating new log.
I20260923 22:16:31.016891 23025 log.cc:824] T 00000000000000000000000000000000 P 45bdb42a5d4148d383ac026f230ea481: Log is configured to *not* fsync() on all Append() calls
I20260923 22:16:31.017580 23025 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 45bdb42a5d4148d383ac026f230ea481: No bootstrap required, opened a new log
I20260923 22:16:31.018059 1567 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskRiQe_a/build/release/bin/kudu as pid 22957
I20260923 22:16:31.018184 1567 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskRiQe_a/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1790201539513534-1567-0/minicluster-data/master-0/wal/instance
I20260923 22:16:31.018396 1567 external_mini_cluster.cc:1428] Setting key 87cb8d9eb26c1065a65a0e6c110c2473
I20260923 22:16:31.020159 23025 raft_consensus.cc:348] T 00000000000000000000000000000000 P 45bdb42a5d4148d383ac026f230ea481 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "45bdb42a5d4148d383ac026f230ea481" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 39743 } }
I20260923 22:16:31.020354 23025 raft_consensus.cc:374] T 00000000000000000000000000000000 P 45bdb42a5d4148d383ac026f230ea481 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260923 22:16:31.020399 23025 raft_consensus.cc:729] T 00000000000000000000000000000000 P 45bdb42a5d4148d383ac026f230ea481 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 45bdb42a5d4148d383ac026f230ea481, State: Initialized, Role: FOLLOWER
I20260923 22:16:31.020557 23025 consensus_queue.cc:260] T 00000000000000000000000000000000 P 45bdb42a5d4148d383ac026f230ea481 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "45bdb42a5d4148d383ac026f230ea481" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 39743 } }
I20260923 22:16:31.020679 23025 raft_consensus.cc:388] T 00000000000000000000000000000000 P 45bdb42a5d4148d383ac026f230ea481 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260923 22:16:31.020720 23025 raft_consensus.cc:482] T 00000000000000000000000000000000 P 45bdb42a5d4148d383ac026f230ea481 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260923 22:16:31.020804 23025 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 45bdb42a5d4148d383ac026f230ea481 [term 0 FOLLOWER]: Advancing to term 1
I20260923 22:16:31.021021 23029 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 22:16:31.018942 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.138:45103 (local address 127.0.0.1:39743)
0923 22:16:31.019139 (+ 197us) server_negotiation.cc:206] Beginning negotiation
0923 22:16:31.019158 (+ 19us) server_negotiation.cc:399] Waiting for connection header
0923 22:16:31.019185 (+ 27us) server_negotiation.cc:407] Connection header received
0923 22:16:31.019598 (+ 413us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 22:16:31.019604 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 22:16:31.019632 (+ 28us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 22:16:31.019755 (+ 123us) server_negotiation.cc:226] Negotiated authn=SASL
0923 22:16:31.020497 (+ 742us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0923 22:16:31.020501 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0923 22:16:31.020502 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0923 22:16:31.020506 (+ 4us) server_negotiation.cc:922] calling sasl_server_start()
0923 22:16:31.020554 (+ 48us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0923 22:16:31.020568 (+ 14us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0923 22:16:31.020639 (+ 71us) server_negotiation.cc:1035] Waiting for connection context
0923 22:16:31.020743 (+ 104us) server_negotiation.cc:299] Negotiation successful
0923 22:16:31.020791 (+ 48us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":131,"thread_start_us":101,"threads_started":1}
I20260923 22:16:31.021955 23025 raft_consensus.cc:504] T 00000000000000000000000000000000 P 45bdb42a5d4148d383ac026f230ea481 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "45bdb42a5d4148d383ac026f230ea481" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 39743 } }
I20260923 22:16:31.022002 1567 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260923 22:16:31.022078 23025 leader_election.cc:302] T 00000000000000000000000000000000 P 45bdb42a5d4148d383ac026f230ea481 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 45bdb42a5d4148d383ac026f230ea481; no voters:
I20260923 22:16:31.022236 23025 leader_election.cc:288] T 00000000000000000000000000000000 P 45bdb42a5d4148d383ac026f230ea481 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260923 22:16:31.022264 23030 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 45bdb42a5d4148d383ac026f230ea481 [term 1 FOLLOWER]: Leader election won for term 1
I20260923 22:16:31.022454 23030 raft_consensus.cc:686] T 00000000000000000000000000000000 P 45bdb42a5d4148d383ac026f230ea481 [term 1 LEADER]: Becoming Leader. State: Replica: 45bdb42a5d4148d383ac026f230ea481, State: Running, Role: LEADER
I20260923 22:16:31.022476 23025 sys_catalog.cc:564] T 00000000000000000000000000000000 P 45bdb42a5d4148d383ac026f230ea481 [sys.catalog]: configured and running, proceeding with master startup.
I20260923 22:16:31.023927 23030 consensus_queue.cc:237] T 00000000000000000000000000000000 P 45bdb42a5d4148d383ac026f230ea481 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "45bdb42a5d4148d383ac026f230ea481" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 39743 } }
I20260923 22:16:31.024628 23031 sys_catalog.cc:455] T 00000000000000000000000000000000 P 45bdb42a5d4148d383ac026f230ea481 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 45bdb42a5d4148d383ac026f230ea481. Latest consensus state: current_term: 1 leader_uuid: "45bdb42a5d4148d383ac026f230ea481" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "45bdb42a5d4148d383ac026f230ea481" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 39743 } } }
I20260923 22:16:31.024662 23032 sys_catalog.cc:455] T 00000000000000000000000000000000 P 45bdb42a5d4148d383ac026f230ea481 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "45bdb42a5d4148d383ac026f230ea481" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "45bdb42a5d4148d383ac026f230ea481" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 39743 } } }
I20260923 22:16:31.024909 23031 sys_catalog.cc:458] T 00000000000000000000000000000000 P 45bdb42a5d4148d383ac026f230ea481 [sys.catalog]: This master's current role is: LEADER
I20260923 22:16:31.024914 23032 sys_catalog.cc:458] T 00000000000000000000000000000000 P 45bdb42a5d4148d383ac026f230ea481 [sys.catalog]: This master's current role is: LEADER
I20260923 22:16:31.025074 23029 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 22:16:31.023628 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.138:56327 (local address 127.0.0.1:39743)
0923 22:16:31.023672 (+ 44us) server_negotiation.cc:206] Beginning negotiation
0923 22:16:31.023674 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0923 22:16:31.023874 (+ 200us) server_negotiation.cc:407] Connection header received
0923 22:16:31.023948 (+ 74us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 22:16:31.023948 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 22:16:31.023955 (+ 7us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 22:16:31.024028 (+ 73us) server_negotiation.cc:226] Negotiated authn=SASL
0923 22:16:31.024777 (+ 749us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0923 22:16:31.024778 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0923 22:16:31.024779 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0923 22:16:31.024783 (+ 4us) server_negotiation.cc:922] calling sasl_server_start()
0923 22:16:31.024797 (+ 14us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0923 22:16:31.024801 (+ 4us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0923 22:16:31.024848 (+ 47us) server_negotiation.cc:1035] Waiting for connection context
0923 22:16:31.024987 (+ 139us) server_negotiation.cc:299] Negotiation successful
0923 22:16:31.025001 (+ 14us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":21}
W20260923 22:16:31.025133 23049 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 45bdb42a5d4148d383ac026f230ea481: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260923 22:16:31.025195 23049 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260923 22:16:31.025264 23051 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260923 22:16:31.025430 23051 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260923 22:16:31.027005 23051 catalog_manager.cc:1347] Generated new cluster ID: 0134d207339249a9b3f4097496c47d33
I20260923 22:16:31.027055 23051 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260923 22:16:31.042804 23051 catalog_manager.cc:1370] Generated new certificate authority record
I20260923 22:16:31.043017 23051 catalog_manager.cc:1504] Loading token signing keys...
I20260923 22:16:31.055747 23051 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 45bdb42a5d4148d383ac026f230ea481: Generated new TSK 0
I20260923 22:16:31.055986 23051 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260923 22:16:31.058218 1567 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskRiQe_a/build/release/bin/kudu with pid 22957
2026-09-23T22:16:31Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/11 (1293 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/12
Loading random data
Initializing database '/tmp/dist-test-taskRiQe_a/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1790201539513534-1567-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Sep 23 22:16:31 dist-test-slave-s2fj krb5kdc[23056](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Sep 23 22:16:31 dist-test-slave-s2fj krb5kdc[23056](info): set up 2 sockets
Sep 23 22:16:31 dist-test-slave-s2fj krb5kdc[23056](info): commencing operation
krb5kdc: starting...
W20260923 22:16:32.076256 1567 mini_kdc.cc:121] Time spent starting KDC: real 1.000s user 0.001s sys 0.004s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskRiQe_a/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1790201539513534-1567-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskRiQe_a/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1790201539513534-1567-0/krb5kdc/test-user.keytab.
Sep 23 22:16:32 dist-test-slave-s2fj krb5kdc[23056](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790201792, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-09-23T22:16:32Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-09-23T22:16:32Z Disabled control of system clock
WARNING: no policy specified for kudu/127.1.135.254@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.1.135.254@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.1.135.254 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskRiQe_a/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1790201539513534-1567-0/krb5kdc/kudu_127.1.135.254.keytab.
Entry for principal kudu/127.1.135.254 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskRiQe_a/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1790201539513534-1567-0/krb5kdc/kudu_127.1.135.254.keytab.
I20260923 22:16:32.195199 1567 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskRiQe_a/build/release/bin/kudu
/tmp/dist-test-taskRiQe_a/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskRiQe_a/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1790201539513534-1567-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskRiQe_a/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1790201539513534-1567-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskRiQe_a/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1790201539513534-1567-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskRiQe_a/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1790201539513534-1567-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.1.135.254:45677
--webserver_interface=127.1.135.254
--webserver_port=0
--keytab_file=/tmp/dist-test-taskRiQe_a/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1790201539513534-1567-0/krb5kdc/kudu_127.1.135.254.keytab
--principal=kudu/127.1.135.254
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.1.135.212:40291
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.1.135.254:45677
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=required
--rpc_encryption=required
--rpc_encrypt_loopback_connections=true with env {KRB5CCNAME=/tmp/dist-test-taskRiQe_a/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1790201539513534-1567-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskRiQe_a/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1790201539513534-1567-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskRiQe_a/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1790201539513534-1567-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260923 22:16:32.276646 23069 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260923 22:16:32.276868 23069 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260923 22:16:32.276892 23069 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260923 22:16:32.278015 23069 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260923 22:16:32.278055 23069 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260923 22:16:32.278069 23069 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260923 22:16:32.278082 23069 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260923 22:16:32.278095 23069 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260923 22:16:32.279544 23069 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.1.135.212:40291
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskRiQe_a/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1790201539513534-1567-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskRiQe_a/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1790201539513534-1567-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.1.135.254:45677
--rpc_encrypt_loopback_connections=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskRiQe_a/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1790201539513534-1567-0/krb5kdc/kudu_127.1.135.254.keytab
--principal=kudu/127.1.135.254
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.1.135.254:45677
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskRiQe_a/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1790201539513534-1567-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.1.135.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.23069
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskRiQe_a/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1790201539513534-1567-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 23 Sep 2026 20:53:37 UTC on ccycloud.quasar-rkviyx.root.comops.site
build id 4668083
I20260923 22:16:32.279744 23069 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260923 22:16:32.279917 23069 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260923 22:16:32.284811 23077 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260923 22:16:32.284868 23075 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260923 22:16:32.284855 23074 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260923 22:16:32.285071 23069 server_base.cc:1034] running on GCE node
I20260923 22:16:32.285347 23069 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260923 22:16:32.285696 23069 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260923 22:16:32.286880 23069 hybrid_clock.cc:648] HybridClock initialized: now 1790201792286860 us; error 37 us; skew 500 ppm
Sep 23 22:16:32 dist-test-slave-s2fj krb5kdc[23056](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790201792, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.135.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260923 22:16:32.289546 23069 init.cc:377] Logged in from keytab as kudu/127.1.135.254@KRBTEST.COM (short username kudu)
I20260923 22:16:32.290016 23069 webserver.cc:460] Webserver started at http://127.1.135.254:44343/ using document root <none> and password file <none>
I20260923 22:16:32.290274 23069 fs_manager.cc:359] Metadata directory not provided
I20260923 22:16:32.290343 23069 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260923 22:16:32.290490 23069 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260923 22:16:32.291563 23069 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskRiQe_a/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1790201539513534-1567-0/minicluster-data/master-0/data/instance:
uuid: "2c93a0ae342f4d58a82026944193c39b"
format_stamp: "Formatted at 2026-09-23 22:16:32 on dist-test-slave-s2fj"
server_key: "338c3288b1336bc3d18bd9f142d7080a"
server_key_iv: "bdc14f6e95b67eb47f995cd839c853b0"
server_key_version: "encryptionkey@0"
I20260923 22:16:32.291919 23069 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskRiQe_a/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1790201539513534-1567-0/minicluster-data/master-0/wal/instance:
uuid: "2c93a0ae342f4d58a82026944193c39b"
format_stamp: "Formatted at 2026-09-23 22:16:32 on dist-test-slave-s2fj"
server_key: "338c3288b1336bc3d18bd9f142d7080a"
server_key_iv: "bdc14f6e95b67eb47f995cd839c853b0"
server_key_version: "encryptionkey@0"
I20260923 22:16:32.293371 23069 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.001s
I20260923 22:16:32.294081 23084 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260923 22:16:32.294255 23069 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.001s
I20260923 22:16:32.294354 23069 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskRiQe_a/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1790201539513534-1567-0/minicluster-data/master-0/data,/tmp/dist-test-taskRiQe_a/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1790201539513534-1567-0/minicluster-data/master-0/wal
uuid: "2c93a0ae342f4d58a82026944193c39b"
format_stamp: "Formatted at 2026-09-23 22:16:32 on dist-test-slave-s2fj"
server_key: "338c3288b1336bc3d18bd9f142d7080a"
server_key_iv: "bdc14f6e95b67eb47f995cd839c853b0"
server_key_version: "encryptionkey@0"
I20260923 22:16:32.294436 23069 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskRiQe_a/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1790201539513534-1567-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskRiQe_a/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1790201539513534-1567-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskRiQe_a/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1790201539513534-1567-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260923 22:16:32.320236 23069 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260923 22:16:32.322773 23069 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260923 22:16:32.322948 23069 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260923 22:16:32.326858 23069 rpc_server.cc:307] RPC server started. Bound to: 127.1.135.254:45677
I20260923 22:16:32.326915 23136 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.1.135.254:45677 every 8 connection(s)
I20260923 22:16:32.327301 23069 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskRiQe_a/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1790201539513534-1567-0/minicluster-data/master-0/data/info.pb
I20260923 22:16:32.328227 23137 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260923 22:16:32.330541 1567 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskRiQe_a/build/release/bin/kudu as pid 23069
I20260923 22:16:32.330607 23137 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 2c93a0ae342f4d58a82026944193c39b: Bootstrap starting.
I20260923 22:16:32.330668 1567 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskRiQe_a/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1790201539513534-1567-0/minicluster-data/master-0/wal/instance
I20260923 22:16:32.330857 1567 external_mini_cluster.cc:1428] Setting key 19a618a29b1941e9fba1f3db68fd2220
I20260923 22:16:32.331284 23137 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 2c93a0ae342f4d58a82026944193c39b: Neither blocks nor log segments found. Creating new log.
I20260923 22:16:32.331647 23137 log.cc:824] T 00000000000000000000000000000000 P 2c93a0ae342f4d58a82026944193c39b: Log is configured to *not* fsync() on all Append() calls
I20260923 22:16:32.332355 23137 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 2c93a0ae342f4d58a82026944193c39b: No bootstrap required, opened a new log
I20260923 22:16:32.334676 23137 raft_consensus.cc:348] T 00000000000000000000000000000000 P 2c93a0ae342f4d58a82026944193c39b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2c93a0ae342f4d58a82026944193c39b" member_type: VOTER last_known_addr { host: "127.1.135.254" port: 45677 } }
I20260923 22:16:32.334857 23137 raft_consensus.cc:374] T 00000000000000000000000000000000 P 2c93a0ae342f4d58a82026944193c39b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260923 22:16:32.334889 23137 raft_consensus.cc:729] T 00000000000000000000000000000000 P 2c93a0ae342f4d58a82026944193c39b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 2c93a0ae342f4d58a82026944193c39b, State: Initialized, Role: FOLLOWER
I20260923 22:16:32.335028 23137 consensus_queue.cc:260] T 00000000000000000000000000000000 P 2c93a0ae342f4d58a82026944193c39b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2c93a0ae342f4d58a82026944193c39b" member_type: VOTER last_known_addr { host: "127.1.135.254" port: 45677 } }
I20260923 22:16:32.335098 23137 raft_consensus.cc:388] T 00000000000000000000000000000000 P 2c93a0ae342f4d58a82026944193c39b [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260923 22:16:32.335125 23137 raft_consensus.cc:482] T 00000000000000000000000000000000 P 2c93a0ae342f4d58a82026944193c39b [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260923 22:16:32.335172 23137 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 2c93a0ae342f4d58a82026944193c39b [term 0 FOLLOWER]: Advancing to term 1
I20260923 22:16:32.335742 23137 raft_consensus.cc:504] T 00000000000000000000000000000000 P 2c93a0ae342f4d58a82026944193c39b [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2c93a0ae342f4d58a82026944193c39b" member_type: VOTER last_known_addr { host: "127.1.135.254" port: 45677 } }
I20260923 22:16:32.335880 23137 leader_election.cc:302] T 00000000000000000000000000000000 P 2c93a0ae342f4d58a82026944193c39b [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 2c93a0ae342f4d58a82026944193c39b; no voters:
I20260923 22:16:32.336109 23137 leader_election.cc:288] T 00000000000000000000000000000000 P 2c93a0ae342f4d58a82026944193c39b [CANDIDATE]: Term 1 election: Requested vote from peers
I20260923 22:16:32.336319 23143 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 2c93a0ae342f4d58a82026944193c39b [term 1 FOLLOWER]: Leader election won for term 1
I20260923 22:16:32.336418 23137 sys_catalog.cc:564] T 00000000000000000000000000000000 P 2c93a0ae342f4d58a82026944193c39b [sys.catalog]: configured and running, proceeding with master startup.
I20260923 22:16:32.336436 23143 raft_consensus.cc:686] T 00000000000000000000000000000000 P 2c93a0ae342f4d58a82026944193c39b [term 1 LEADER]: Becoming Leader. State: Replica: 2c93a0ae342f4d58a82026944193c39b, State: Running, Role: LEADER
I20260923 22:16:32.336572 23143 consensus_queue.cc:237] T 00000000000000000000000000000000 P 2c93a0ae342f4d58a82026944193c39b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2c93a0ae342f4d58a82026944193c39b" member_type: VOTER last_known_addr { host: "127.1.135.254" port: 45677 } }
I20260923 22:16:32.337028 23144 sys_catalog.cc:455] T 00000000000000000000000000000000 P 2c93a0ae342f4d58a82026944193c39b [sys.catalog]: SysCatalogTable state changed. Reason: New leader 2c93a0ae342f4d58a82026944193c39b. Latest consensus state: current_term: 1 leader_uuid: "2c93a0ae342f4d58a82026944193c39b" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2c93a0ae342f4d58a82026944193c39b" member_type: VOTER last_known_addr { host: "127.1.135.254" port: 45677 } } }
I20260923 22:16:32.337244 23142 sys_catalog.cc:455] T 00000000000000000000000000000000 P 2c93a0ae342f4d58a82026944193c39b [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "2c93a0ae342f4d58a82026944193c39b" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2c93a0ae342f4d58a82026944193c39b" member_type: VOTER last_known_addr { host: "127.1.135.254" port: 45677 } } }
I20260923 22:16:32.337335 23144 sys_catalog.cc:458] T 00000000000000000000000000000000 P 2c93a0ae342f4d58a82026944193c39b [sys.catalog]: This master's current role is: LEADER
I20260923 22:16:32.337388 23142 sys_catalog.cc:458] T 00000000000000000000000000000000 P 2c93a0ae342f4d58a82026944193c39b [sys.catalog]: This master's current role is: LEADER
I20260923 22:16:32.337626 23147 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260923 22:16:32.338155 23147 catalog_manager.cc:1484] Initializing Kudu cluster ID...
Sep 23 22:16:32 dist-test-slave-s2fj krb5kdc[23056](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790201792, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.1.135.254@KRBTEST.COM
I20260923 22:16:32.339978 23147 catalog_manager.cc:1347] Generated new cluster ID: e6e4b0b880e34557a20f7c685d11d34b
I20260923 22:16:32.340027 23147 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260923 22:16:32.342991 23140 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 22:16:32.331429 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48383 (local address 127.1.135.254:45677)
0923 22:16:32.331650 (+ 221us) server_negotiation.cc:206] Beginning negotiation
0923 22:16:32.331660 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0923 22:16:32.331700 (+ 40us) server_negotiation.cc:407] Connection header received
0923 22:16:32.332308 (+ 608us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 22:16:32.332315 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 22:16:32.332370 (+ 55us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 22:16:32.332548 (+ 178us) server_negotiation.cc:226] Negotiated authn=SASL
0923 22:16:32.333341 (+ 793us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 22:16:32.334247 (+ 906us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 22:16:32.335315 (+ 1068us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 22:16:32.335721 (+ 406us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 22:16:32.335809 (+ 88us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 22:16:32.339120 (+ 3311us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0923 22:16:32.339124 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0923 22:16:32.339124 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0923 22:16:32.339137 (+ 13us) server_negotiation.cc:922] calling sasl_server_start()
0923 22:16:32.341219 (+ 2082us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0923 22:16:32.341686 (+ 467us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0923 22:16:32.341689 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0923 22:16:32.341689 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0923 22:16:32.341806 (+ 117us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0923 22:16:32.342054 (+ 248us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0923 22:16:32.342055 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0923 22:16:32.342055 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0923 22:16:32.342373 (+ 318us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0923 22:16:32.342538 (+ 165us) server_negotiation.cc:1035] Waiting for connection context
0923 22:16:32.342671 (+ 133us) server_negotiation.cc:299] Negotiation successful
0923 22:16:32.342774 (+ 103us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":163,"thread_start_us":88,"threads_started":1}
I20260923 22:16:32.344128 1567 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260923 22:16:32.345484 23147 catalog_manager.cc:1370] Generated new certificate authority record
I20260923 22:16:32.346204 23147 catalog_manager.cc:1504] Loading token signing keys...
I20260923 22:16:32.352998 23147 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 2c93a0ae342f4d58a82026944193c39b: Generated new TSK 0
I20260923 22:16:32.353232 23147 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260923 22:16:32.353577 23140 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 22:16:32.345730 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60105 (local address 127.1.135.254:45677)
0923 22:16:32.345768 (+ 38us) server_negotiation.cc:206] Beginning negotiation
0923 22:16:32.345769 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0923 22:16:32.345781 (+ 12us) server_negotiation.cc:407] Connection header received
0923 22:16:32.345846 (+ 65us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 22:16:32.345846 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 22:16:32.345857 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 22:16:32.345952 (+ 95us) server_negotiation.cc:226] Negotiated authn=SASL
0923 22:16:32.346610 (+ 658us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 22:16:32.347363 (+ 753us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 22:16:32.348360 (+ 997us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 22:16:32.348660 (+ 300us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 22:16:32.348733 (+ 73us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 22:16:32.350132 (+ 1399us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0923 22:16:32.350133 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0923 22:16:32.350133 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0923 22:16:32.350141 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0923 22:16:32.351737 (+ 1596us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0923 22:16:32.352208 (+ 471us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0923 22:16:32.352209 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0923 22:16:32.352210 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0923 22:16:32.352267 (+ 57us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0923 22:16:32.352525 (+ 258us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0923 22:16:32.352526 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0923 22:16:32.352526 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0923 22:16:32.352658 (+ 132us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0923 22:16:32.352805 (+ 147us) server_negotiation.cc:1035] Waiting for connection context
0923 22:16:32.353427 (+ 622us) server_negotiation.cc:299] Negotiation successful
0923 22:16:32.353491 (+ 64us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":21}
I20260923 22:16:32.355527 1567 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskRiQe_a/build/release/bin/kudu with pid 23069
W20260923 22:16:32.359880 23054 connection.cc:535] client connection to 127.1.135.254:45677 recv error: Network error: failed to read from TLS socket (remote: 127.1.135.254:45677): Connection reset by peer (error 104)
2026-09-23T22:16:32Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/12 (1297 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/13
Loading random data
Initializing database '/tmp/dist-test-taskRiQe_a/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1790201539513534-1567-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Sep 23 22:16:32 dist-test-slave-s2fj krb5kdc[23168](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Sep 23 22:16:32 dist-test-slave-s2fj krb5kdc[23168](info): set up 2 sockets
Sep 23 22:16:32 dist-test-slave-s2fj krb5kdc[23168](info): commencing operation
krb5kdc: starting...
W20260923 22:16:33.383997 1567 mini_kdc.cc:121] Time spent starting KDC: real 1.010s user 0.003s sys 0.003s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskRiQe_a/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1790201539513534-1567-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskRiQe_a/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1790201539513534-1567-0/krb5kdc/test-user.keytab.
Sep 23 22:16:33 dist-test-slave-s2fj krb5kdc[23168](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790201793, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-09-23T22:16:33Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-09-23T22:16:33Z Disabled control of system clock
WARNING: no policy specified for kudu/127.1.135.254@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.1.135.254@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.1.135.254 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskRiQe_a/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1790201539513534-1567-0/krb5kdc/kudu_127.1.135.254.keytab.
Entry for principal kudu/127.1.135.254 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskRiQe_a/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1790201539513534-1567-0/krb5kdc/kudu_127.1.135.254.keytab.
I20260923 22:16:33.494546 1567 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskRiQe_a/build/release/bin/kudu
/tmp/dist-test-taskRiQe_a/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskRiQe_a/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1790201539513534-1567-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskRiQe_a/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1790201539513534-1567-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskRiQe_a/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1790201539513534-1567-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskRiQe_a/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1790201539513534-1567-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.1.135.254:40471
--webserver_interface=127.1.135.254
--webserver_port=0
--keytab_file=/tmp/dist-test-taskRiQe_a/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1790201539513534-1567-0/krb5kdc/kudu_127.1.135.254.keytab
--principal=kudu/127.1.135.254
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.1.135.212:35527
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.1.135.254:40471
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=required
--rpc_encrypt_loopback_connections=true with env {KRB5CCNAME=/tmp/dist-test-taskRiQe_a/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1790201539513534-1567-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskRiQe_a/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1790201539513534-1567-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskRiQe_a/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1790201539513534-1567-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260923 22:16:33.573995 23181 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260923 22:16:33.574199 23181 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260923 22:16:33.574222 23181 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260923 22:16:33.575287 23181 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260923 22:16:33.575320 23181 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260923 22:16:33.575335 23181 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260923 22:16:33.575347 23181 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260923 22:16:33.575359 23181 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260923 22:16:33.576772 23181 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.1.135.212:35527
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskRiQe_a/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1790201539513534-1567-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskRiQe_a/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1790201539513534-1567-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.1.135.254:40471
--rpc_encrypt_loopback_connections=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskRiQe_a/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1790201539513534-1567-0/krb5kdc/kudu_127.1.135.254.keytab
--principal=kudu/127.1.135.254
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.1.135.254:40471
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskRiQe_a/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1790201539513534-1567-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.1.135.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.23181
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskRiQe_a/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1790201539513534-1567-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 23 Sep 2026 20:53:37 UTC on ccycloud.quasar-rkviyx.root.comops.site
build id 4668083
I20260923 22:16:33.576953 23181 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260923 22:16:33.577118 23181 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260923 22:16:33.581815 23186 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260923 22:16:33.581816 23187 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260923 22:16:33.581815 23189 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260923 22:16:33.581840 23181 server_base.cc:1034] running on GCE node
I20260923 22:16:33.582396 23181 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260923 22:16:33.582638 23181 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260923 22:16:33.583815 23181 hybrid_clock.cc:648] HybridClock initialized: now 1790201793583801 us; error 31 us; skew 500 ppm
Sep 23 22:16:33 dist-test-slave-s2fj krb5kdc[23168](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790201793, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.135.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260923 22:16:33.592939 23181 init.cc:377] Logged in from keytab as kudu/127.1.135.254@KRBTEST.COM (short username kudu)
I20260923 22:16:33.593402 23181 webserver.cc:460] Webserver started at http://127.1.135.254:35795/ using document root <none> and password file <none>
I20260923 22:16:33.593602 23181 fs_manager.cc:359] Metadata directory not provided
I20260923 22:16:33.593645 23181 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260923 22:16:33.593732 23181 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260923 22:16:33.594736 23181 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskRiQe_a/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1790201539513534-1567-0/minicluster-data/master-0/data/instance:
uuid: "036c3650ac2440fbaa8b8e8142ccfdc9"
format_stamp: "Formatted at 2026-09-23 22:16:33 on dist-test-slave-s2fj"
server_key: "25ae8183ae8be8aa78e66a4d75f28042"
server_key_iv: "a04f455755e236085b01aeae2c7fd0ad"
server_key_version: "encryptionkey@0"
I20260923 22:16:33.595044 23181 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskRiQe_a/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1790201539513534-1567-0/minicluster-data/master-0/wal/instance:
uuid: "036c3650ac2440fbaa8b8e8142ccfdc9"
format_stamp: "Formatted at 2026-09-23 22:16:33 on dist-test-slave-s2fj"
server_key: "25ae8183ae8be8aa78e66a4d75f28042"
server_key_iv: "a04f455755e236085b01aeae2c7fd0ad"
server_key_version: "encryptionkey@0"
I20260923 22:16:33.596396 23181 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.001s
I20260923 22:16:33.597136 23196 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260923 22:16:33.597326 23181 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260923 22:16:33.597409 23181 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskRiQe_a/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1790201539513534-1567-0/minicluster-data/master-0/data,/tmp/dist-test-taskRiQe_a/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1790201539513534-1567-0/minicluster-data/master-0/wal
uuid: "036c3650ac2440fbaa8b8e8142ccfdc9"
format_stamp: "Formatted at 2026-09-23 22:16:33 on dist-test-slave-s2fj"
server_key: "25ae8183ae8be8aa78e66a4d75f28042"
server_key_iv: "a04f455755e236085b01aeae2c7fd0ad"
server_key_version: "encryptionkey@0"
I20260923 22:16:33.597464 23181 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskRiQe_a/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1790201539513534-1567-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskRiQe_a/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1790201539513534-1567-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskRiQe_a/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1790201539513534-1567-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260923 22:16:33.624835 23181 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260923 22:16:33.626832 23181 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260923 22:16:33.626986 23181 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260923 22:16:33.631093 23181 rpc_server.cc:307] RPC server started. Bound to: 127.1.135.254:40471
I20260923 22:16:33.631184 23248 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.1.135.254:40471 every 8 connection(s)
I20260923 22:16:33.631516 23181 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskRiQe_a/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1790201539513534-1567-0/minicluster-data/master-0/data/info.pb
I20260923 22:16:33.632495 23249 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260923 22:16:33.634944 23249 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 036c3650ac2440fbaa8b8e8142ccfdc9: Bootstrap starting.
I20260923 22:16:33.635538 23249 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 036c3650ac2440fbaa8b8e8142ccfdc9: Neither blocks nor log segments found. Creating new log.
I20260923 22:16:33.635835 23249 log.cc:824] T 00000000000000000000000000000000 P 036c3650ac2440fbaa8b8e8142ccfdc9: Log is configured to *not* fsync() on all Append() calls
I20260923 22:16:33.636615 23249 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 036c3650ac2440fbaa8b8e8142ccfdc9: No bootstrap required, opened a new log
I20260923 22:16:33.638213 23249 raft_consensus.cc:348] T 00000000000000000000000000000000 P 036c3650ac2440fbaa8b8e8142ccfdc9 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "036c3650ac2440fbaa8b8e8142ccfdc9" member_type: VOTER last_known_addr { host: "127.1.135.254" port: 40471 } }
I20260923 22:16:33.638381 23249 raft_consensus.cc:374] T 00000000000000000000000000000000 P 036c3650ac2440fbaa8b8e8142ccfdc9 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260923 22:16:33.638429 23249 raft_consensus.cc:729] T 00000000000000000000000000000000 P 036c3650ac2440fbaa8b8e8142ccfdc9 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 036c3650ac2440fbaa8b8e8142ccfdc9, State: Initialized, Role: FOLLOWER
I20260923 22:16:33.638581 23249 consensus_queue.cc:260] T 00000000000000000000000000000000 P 036c3650ac2440fbaa8b8e8142ccfdc9 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "036c3650ac2440fbaa8b8e8142ccfdc9" member_type: VOTER last_known_addr { host: "127.1.135.254" port: 40471 } }
I20260923 22:16:33.638669 23249 raft_consensus.cc:388] T 00000000000000000000000000000000 P 036c3650ac2440fbaa8b8e8142ccfdc9 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260923 22:16:33.638713 23249 raft_consensus.cc:482] T 00000000000000000000000000000000 P 036c3650ac2440fbaa8b8e8142ccfdc9 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260923 22:16:33.638767 23249 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 036c3650ac2440fbaa8b8e8142ccfdc9 [term 0 FOLLOWER]: Advancing to term 1
I20260923 22:16:33.638906 1567 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskRiQe_a/build/release/bin/kudu as pid 23181
I20260923 22:16:33.639024 1567 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskRiQe_a/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1790201539513534-1567-0/minicluster-data/master-0/wal/instance
I20260923 22:16:33.639214 1567 external_mini_cluster.cc:1428] Setting key 0f84aba984a1c28052cc40675fd8aa68
I20260923 22:16:33.639842 23249 raft_consensus.cc:504] T 00000000000000000000000000000000 P 036c3650ac2440fbaa8b8e8142ccfdc9 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "036c3650ac2440fbaa8b8e8142ccfdc9" member_type: VOTER last_known_addr { host: "127.1.135.254" port: 40471 } }
I20260923 22:16:33.639953 23249 leader_election.cc:302] T 00000000000000000000000000000000 P 036c3650ac2440fbaa8b8e8142ccfdc9 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 036c3650ac2440fbaa8b8e8142ccfdc9; no voters:
I20260923 22:16:33.640164 23254 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 036c3650ac2440fbaa8b8e8142ccfdc9 [term 1 FOLLOWER]: Leader election won for term 1
I20260923 22:16:33.640362 23254 raft_consensus.cc:686] T 00000000000000000000000000000000 P 036c3650ac2440fbaa8b8e8142ccfdc9 [term 1 LEADER]: Becoming Leader. State: Replica: 036c3650ac2440fbaa8b8e8142ccfdc9, State: Running, Role: LEADER
I20260923 22:16:33.640149 23249 leader_election.cc:288] T 00000000000000000000000000000000 P 036c3650ac2440fbaa8b8e8142ccfdc9 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260923 22:16:33.640489 23254 consensus_queue.cc:237] T 00000000000000000000000000000000 P 036c3650ac2440fbaa8b8e8142ccfdc9 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "036c3650ac2440fbaa8b8e8142ccfdc9" member_type: VOTER last_known_addr { host: "127.1.135.254" port: 40471 } }
I20260923 22:16:33.640764 23249 sys_catalog.cc:564] T 00000000000000000000000000000000 P 036c3650ac2440fbaa8b8e8142ccfdc9 [sys.catalog]: configured and running, proceeding with master startup.
I20260923 22:16:33.640811 23255 sys_catalog.cc:455] T 00000000000000000000000000000000 P 036c3650ac2440fbaa8b8e8142ccfdc9 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 036c3650ac2440fbaa8b8e8142ccfdc9. Latest consensus state: current_term: 1 leader_uuid: "036c3650ac2440fbaa8b8e8142ccfdc9" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "036c3650ac2440fbaa8b8e8142ccfdc9" member_type: VOTER last_known_addr { host: "127.1.135.254" port: 40471 } } }
I20260923 22:16:33.640899 23255 sys_catalog.cc:458] T 00000000000000000000000000000000 P 036c3650ac2440fbaa8b8e8142ccfdc9 [sys.catalog]: This master's current role is: LEADER
I20260923 22:16:33.641120 23258 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260923 22:16:33.641098 23256 sys_catalog.cc:455] T 00000000000000000000000000000000 P 036c3650ac2440fbaa8b8e8142ccfdc9 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "036c3650ac2440fbaa8b8e8142ccfdc9" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "036c3650ac2440fbaa8b8e8142ccfdc9" member_type: VOTER last_known_addr { host: "127.1.135.254" port: 40471 } } }
I20260923 22:16:33.641220 23256 sys_catalog.cc:458] T 00000000000000000000000000000000 P 036c3650ac2440fbaa8b8e8142ccfdc9 [sys.catalog]: This master's current role is: LEADER
I20260923 22:16:33.641788 23258 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260923 22:16:33.643308 23258 catalog_manager.cc:1347] Generated new cluster ID: 7371a556f8c84748bf1a9791ef553b14
I20260923 22:16:33.643356 23258 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260923 22:16:33.645403 23253 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 22:16:33.639699 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52369 (local address 127.1.135.254:40471)
0923 22:16:33.639893 (+ 194us) server_negotiation.cc:206] Beginning negotiation
0923 22:16:33.639904 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0923 22:16:33.639940 (+ 36us) server_negotiation.cc:407] Connection header received
0923 22:16:33.640055 (+ 115us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 22:16:33.640103 (+ 48us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 22:16:33.640184 (+ 81us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 22:16:33.640325 (+ 141us) server_negotiation.cc:226] Negotiated authn=SASL
0923 22:16:33.640574 (+ 249us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 22:16:33.642170 (+ 1596us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 22:16:33.643129 (+ 959us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 22:16:33.643731 (+ 602us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 22:16:33.643837 (+ 106us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 22:16:33.644691 (+ 854us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0923 22:16:33.644695 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0923 22:16:33.644695 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0923 22:16:33.644896 (+ 201us) server_negotiation.cc:922] calling sasl_server_start()
0923 22:16:33.644933 (+ 37us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0923 22:16:33.644947 (+ 14us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0923 22:16:33.645026 (+ 79us) server_negotiation.cc:1035] Waiting for connection context
0923 22:16:33.645165 (+ 139us) server_negotiation.cc:299] Negotiation successful
0923 22:16:33.645192 (+ 27us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":137,"thread_start_us":90,"threads_started":1}
I20260923 22:16:33.646466 1567 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260923 22:16:33.651582 23253 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 22:16:33.647903 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49225 (local address 127.1.135.254:40471)
0923 22:16:33.647953 (+ 50us) server_negotiation.cc:206] Beginning negotiation
0923 22:16:33.647954 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0923 22:16:33.648118 (+ 164us) server_negotiation.cc:407] Connection header received
0923 22:16:33.648165 (+ 47us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 22:16:33.648166 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 22:16:33.648176 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 22:16:33.648234 (+ 58us) server_negotiation.cc:226] Negotiated authn=SASL
0923 22:16:33.648371 (+ 137us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 22:16:33.649101 (+ 730us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 22:16:33.650158 (+ 1057us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 22:16:33.650474 (+ 316us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 22:16:33.650532 (+ 58us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 22:16:33.651295 (+ 763us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0923 22:16:33.651296 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0923 22:16:33.651297 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0923 22:16:33.651301 (+ 4us) server_negotiation.cc:922] calling sasl_server_start()
0923 22:16:33.651312 (+ 11us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0923 22:16:33.651316 (+ 4us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0923 22:16:33.651365 (+ 49us) server_negotiation.cc:1035] Waiting for connection context
0923 22:16:33.651488 (+ 123us) server_negotiation.cc:299] Negotiation successful
0923 22:16:33.651499 (+ 11us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":15}
I20260923 22:16:33.657187 23258 catalog_manager.cc:1370] Generated new certificate authority record
I20260923 22:16:33.657902 23258 catalog_manager.cc:1504] Loading token signing keys...
I20260923 22:16:33.663298 23258 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 036c3650ac2440fbaa8b8e8142ccfdc9: Generated new TSK 0
I20260923 22:16:33.663517 23258 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260923 22:16:33.670720 1567 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskRiQe_a/build/release/bin/kudu with pid 23181
W20260923 22:16:33.675052 23166 connection.cc:535] client connection to 127.1.135.254:40471 recv error: Network error: failed to read from TLS socket (remote: 127.1.135.254:40471): Connection reset by peer (error 104)
2026-09-23T22:16:33Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/13 (1314 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/14
Loading random data
Initializing database '/tmp/dist-test-taskRiQe_a/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1790201539513534-1567-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Sep 23 22:16:33 dist-test-slave-s2fj krb5kdc[23280](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Sep 23 22:16:33 dist-test-slave-s2fj krb5kdc[23280](info): set up 2 sockets
Sep 23 22:16:33 dist-test-slave-s2fj krb5kdc[23280](info): commencing operation
krb5kdc: starting...
W20260923 22:16:34.700292 1567 mini_kdc.cc:121] Time spent starting KDC: real 1.012s user 0.000s sys 0.006s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskRiQe_a/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1790201539513534-1567-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskRiQe_a/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1790201539513534-1567-0/krb5kdc/test-user.keytab.
Sep 23 22:16:34 dist-test-slave-s2fj krb5kdc[23280](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790201794, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-09-23T22:16:34Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-09-23T22:16:34Z Disabled control of system clock
WARNING: no policy specified for kudu/127.1.135.254@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.1.135.254@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.1.135.254 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskRiQe_a/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1790201539513534-1567-0/krb5kdc/kudu_127.1.135.254.keytab.
Entry for principal kudu/127.1.135.254 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskRiQe_a/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1790201539513534-1567-0/krb5kdc/kudu_127.1.135.254.keytab.
I20260923 22:16:34.812750 1567 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskRiQe_a/build/release/bin/kudu
/tmp/dist-test-taskRiQe_a/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskRiQe_a/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1790201539513534-1567-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskRiQe_a/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1790201539513534-1567-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskRiQe_a/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1790201539513534-1567-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskRiQe_a/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1790201539513534-1567-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.1.135.254:34017
--webserver_interface=127.1.135.254
--webserver_port=0
--keytab_file=/tmp/dist-test-taskRiQe_a/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1790201539513534-1567-0/krb5kdc/kudu_127.1.135.254.keytab
--principal=kudu/127.1.135.254
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.1.135.212:45785
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.1.135.254:34017
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=disabled
--rpc_encrypt_loopback_connections=true with env {KRB5CCNAME=/tmp/dist-test-taskRiQe_a/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1790201539513534-1567-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskRiQe_a/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1790201539513534-1567-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskRiQe_a/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1790201539513534-1567-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260923 22:16:34.889467 23293 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260923 22:16:34.889655 23293 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260923 22:16:34.889678 23293 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260923 22:16:34.890688 23293 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260923 22:16:34.890722 23293 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260923 22:16:34.890735 23293 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260923 22:16:34.890748 23293 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260923 22:16:34.890759 23293 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260923 22:16:34.892114 23293 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.1.135.212:45785
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskRiQe_a/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1790201539513534-1567-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskRiQe_a/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1790201539513534-1567-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.1.135.254:34017
--rpc_encrypt_loopback_connections=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskRiQe_a/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1790201539513534-1567-0/krb5kdc/kudu_127.1.135.254.keytab
--principal=kudu/127.1.135.254
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.1.135.254:34017
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskRiQe_a/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1790201539513534-1567-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.1.135.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.23293
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskRiQe_a/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1790201539513534-1567-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 23 Sep 2026 20:53:37 UTC on ccycloud.quasar-rkviyx.root.comops.site
build id 4668083
I20260923 22:16:34.892275 23293 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260923 22:16:34.892454 23293 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260923 22:16:34.897364 23299 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260923 22:16:34.897344 23298 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260923 22:16:34.897651 23301 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260923 22:16:34.901562 23293 server_base.cc:1034] running on GCE node
I20260923 22:16:34.901798 23293 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260923 22:16:34.902086 23293 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260923 22:16:34.903241 23293 hybrid_clock.cc:648] HybridClock initialized: now 1790201794903220 us; error 36 us; skew 500 ppm
Sep 23 22:16:34 dist-test-slave-s2fj krb5kdc[23280](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790201794, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.135.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260923 22:16:34.906342 23293 init.cc:377] Logged in from keytab as kudu/127.1.135.254@KRBTEST.COM (short username kudu)
I20260923 22:16:34.906899 23293 webserver.cc:460] Webserver started at http://127.1.135.254:36093/ using document root <none> and password file <none>
I20260923 22:16:34.907164 23293 fs_manager.cc:359] Metadata directory not provided
I20260923 22:16:34.907228 23293 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260923 22:16:34.907366 23293 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260923 22:16:34.908510 23293 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskRiQe_a/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1790201539513534-1567-0/minicluster-data/master-0/data/instance:
uuid: "9fbe5246cee441599e72be99705921e7"
format_stamp: "Formatted at 2026-09-23 22:16:34 on dist-test-slave-s2fj"
server_key: "6349525baaa0cd004cdf34a441bd1189"
server_key_iv: "35b7d916d9f42a8a5f6a76526a4b1a09"
server_key_version: "encryptionkey@0"
I20260923 22:16:34.908888 23293 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskRiQe_a/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1790201539513534-1567-0/minicluster-data/master-0/wal/instance:
uuid: "9fbe5246cee441599e72be99705921e7"
format_stamp: "Formatted at 2026-09-23 22:16:34 on dist-test-slave-s2fj"
server_key: "6349525baaa0cd004cdf34a441bd1189"
server_key_iv: "35b7d916d9f42a8a5f6a76526a4b1a09"
server_key_version: "encryptionkey@0"
I20260923 22:16:34.910168 23293 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.000s
I20260923 22:16:34.910887 23308 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260923 22:16:34.911062 23293 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260923 22:16:34.911164 23293 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskRiQe_a/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1790201539513534-1567-0/minicluster-data/master-0/data,/tmp/dist-test-taskRiQe_a/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1790201539513534-1567-0/minicluster-data/master-0/wal
uuid: "9fbe5246cee441599e72be99705921e7"
format_stamp: "Formatted at 2026-09-23 22:16:34 on dist-test-slave-s2fj"
server_key: "6349525baaa0cd004cdf34a441bd1189"
server_key_iv: "35b7d916d9f42a8a5f6a76526a4b1a09"
server_key_version: "encryptionkey@0"
I20260923 22:16:34.911252 23293 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskRiQe_a/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1790201539513534-1567-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskRiQe_a/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1790201539513534-1567-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskRiQe_a/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1790201539513534-1567-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260923 22:16:34.914489 23293 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260923 22:16:34.916281 23293 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260923 22:16:34.916422 23293 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260923 22:16:34.920284 23293 rpc_server.cc:307] RPC server started. Bound to: 127.1.135.254:34017
I20260923 22:16:34.920341 23360 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.1.135.254:34017 every 8 connection(s)
I20260923 22:16:34.920640 23293 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskRiQe_a/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1790201539513534-1567-0/minicluster-data/master-0/data/info.pb
I20260923 22:16:34.921532 23361 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260923 22:16:34.923614 23361 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 9fbe5246cee441599e72be99705921e7: Bootstrap starting.
I20260923 22:16:34.924216 23361 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 9fbe5246cee441599e72be99705921e7: Neither blocks nor log segments found. Creating new log.
I20260923 22:16:34.924515 23361 log.cc:824] T 00000000000000000000000000000000 P 9fbe5246cee441599e72be99705921e7: Log is configured to *not* fsync() on all Append() calls
I20260923 22:16:34.925161 23361 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 9fbe5246cee441599e72be99705921e7: No bootstrap required, opened a new log
I20260923 22:16:34.926613 23361 raft_consensus.cc:348] T 00000000000000000000000000000000 P 9fbe5246cee441599e72be99705921e7 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9fbe5246cee441599e72be99705921e7" member_type: VOTER last_known_addr { host: "127.1.135.254" port: 34017 } }
I20260923 22:16:34.926740 23361 raft_consensus.cc:374] T 00000000000000000000000000000000 P 9fbe5246cee441599e72be99705921e7 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260923 22:16:34.926761 23361 raft_consensus.cc:729] T 00000000000000000000000000000000 P 9fbe5246cee441599e72be99705921e7 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 9fbe5246cee441599e72be99705921e7, State: Initialized, Role: FOLLOWER
I20260923 22:16:34.926926 23361 consensus_queue.cc:260] T 00000000000000000000000000000000 P 9fbe5246cee441599e72be99705921e7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9fbe5246cee441599e72be99705921e7" member_type: VOTER last_known_addr { host: "127.1.135.254" port: 34017 } }
I20260923 22:16:34.926990 23361 raft_consensus.cc:388] T 00000000000000000000000000000000 P 9fbe5246cee441599e72be99705921e7 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260923 22:16:34.927032 23361 raft_consensus.cc:482] T 00000000000000000000000000000000 P 9fbe5246cee441599e72be99705921e7 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260923 22:16:34.927067 1567 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskRiQe_a/build/release/bin/kudu as pid 23293
I20260923 22:16:34.927079 23361 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 9fbe5246cee441599e72be99705921e7 [term 0 FOLLOWER]: Advancing to term 1
I20260923 22:16:34.927191 1567 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskRiQe_a/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1790201539513534-1567-0/minicluster-data/master-0/wal/instance
I20260923 22:16:34.927374 1567 external_mini_cluster.cc:1428] Setting key 49637871808ae72a66f51e8e6b973ba3
I20260923 22:16:34.928220 23361 raft_consensus.cc:504] T 00000000000000000000000000000000 P 9fbe5246cee441599e72be99705921e7 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9fbe5246cee441599e72be99705921e7" member_type: VOTER last_known_addr { host: "127.1.135.254" port: 34017 } }
I20260923 22:16:34.928339 23361 leader_election.cc:302] T 00000000000000000000000000000000 P 9fbe5246cee441599e72be99705921e7 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 9fbe5246cee441599e72be99705921e7; no voters:
I20260923 22:16:34.928486 23361 leader_election.cc:288] T 00000000000000000000000000000000 P 9fbe5246cee441599e72be99705921e7 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260923 22:16:34.928508 23366 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 9fbe5246cee441599e72be99705921e7 [term 1 FOLLOWER]: Leader election won for term 1
I20260923 22:16:34.928620 23366 raft_consensus.cc:686] T 00000000000000000000000000000000 P 9fbe5246cee441599e72be99705921e7 [term 1 LEADER]: Becoming Leader. State: Replica: 9fbe5246cee441599e72be99705921e7, State: Running, Role: LEADER
I20260923 22:16:34.928722 23361 sys_catalog.cc:564] T 00000000000000000000000000000000 P 9fbe5246cee441599e72be99705921e7 [sys.catalog]: configured and running, proceeding with master startup.
I20260923 22:16:34.928755 23366 consensus_queue.cc:237] T 00000000000000000000000000000000 P 9fbe5246cee441599e72be99705921e7 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9fbe5246cee441599e72be99705921e7" member_type: VOTER last_known_addr { host: "127.1.135.254" port: 34017 } }
I20260923 22:16:34.929047 23366 sys_catalog.cc:455] T 00000000000000000000000000000000 P 9fbe5246cee441599e72be99705921e7 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 9fbe5246cee441599e72be99705921e7. Latest consensus state: current_term: 1 leader_uuid: "9fbe5246cee441599e72be99705921e7" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9fbe5246cee441599e72be99705921e7" member_type: VOTER last_known_addr { host: "127.1.135.254" port: 34017 } } }
I20260923 22:16:34.929080 23368 sys_catalog.cc:455] T 00000000000000000000000000000000 P 9fbe5246cee441599e72be99705921e7 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "9fbe5246cee441599e72be99705921e7" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9fbe5246cee441599e72be99705921e7" member_type: VOTER last_known_addr { host: "127.1.135.254" port: 34017 } } }
I20260923 22:16:34.929171 23368 sys_catalog.cc:458] T 00000000000000000000000000000000 P 9fbe5246cee441599e72be99705921e7 [sys.catalog]: This master's current role is: LEADER
I20260923 22:16:34.929119 23366 sys_catalog.cc:458] T 00000000000000000000000000000000 P 9fbe5246cee441599e72be99705921e7 [sys.catalog]: This master's current role is: LEADER
I20260923 22:16:34.929409 23372 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260923 22:16:34.929934 23372 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260923 22:16:34.931093 23365 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 22:16:34.927892 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33223 (local address 127.1.135.254:34017)
0923 22:16:34.928115 (+ 223us) server_negotiation.cc:206] Beginning negotiation
0923 22:16:34.928126 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0923 22:16:34.928154 (+ 28us) server_negotiation.cc:407] Connection header received
0923 22:16:34.928238 (+ 84us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 22:16:34.928243 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 22:16:34.928265 (+ 22us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 22:16:34.928420 (+ 155us) server_negotiation.cc:226] Negotiated authn=SASL
0923 22:16:34.930592 (+ 2172us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0923 22:16:34.930595 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0923 22:16:34.930596 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0923 22:16:34.930599 (+ 3us) server_negotiation.cc:922] calling sasl_server_start()
0923 22:16:34.930643 (+ 44us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0923 22:16:34.930653 (+ 10us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0923 22:16:34.930736 (+ 83us) server_negotiation.cc:1035] Waiting for connection context
0923 22:16:34.930803 (+ 67us) server_negotiation.cc:299] Negotiation successful
0923 22:16:34.930927 (+ 124us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":128,"thread_start_us":102,"threads_started":1}
I20260923 22:16:34.931473 23372 catalog_manager.cc:1347] Generated new cluster ID: cdd9cde1c8554fdd87ae87fecb9e018e
I20260923 22:16:34.931526 23372 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260923 22:16:34.932178 1567 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260923 22:16:34.937104 23365 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 22:16:34.933671 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55481 (local address 127.1.135.254:34017)
0923 22:16:34.933703 (+ 32us) server_negotiation.cc:206] Beginning negotiation
0923 22:16:34.933704 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0923 22:16:34.936238 (+ 2534us) server_negotiation.cc:407] Connection header received
0923 22:16:34.936267 (+ 29us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 22:16:34.936267 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 22:16:34.936272 (+ 5us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 22:16:34.936340 (+ 68us) server_negotiation.cc:226] Negotiated authn=SASL
0923 22:16:34.936889 (+ 549us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0923 22:16:34.936890 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0923 22:16:34.936890 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0923 22:16:34.936894 (+ 4us) server_negotiation.cc:922] calling sasl_server_start()
0923 22:16:34.936908 (+ 14us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0923 22:16:34.936912 (+ 4us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0923 22:16:34.936975 (+ 63us) server_negotiation.cc:1035] Waiting for connection context
0923 22:16:34.937026 (+ 51us) server_negotiation.cc:299] Negotiation successful
0923 22:16:34.937038 (+ 12us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":17}
I20260923 22:16:34.949931 23372 catalog_manager.cc:1370] Generated new certificate authority record
I20260923 22:16:34.950084 23372 catalog_manager.cc:1504] Loading token signing keys...
I20260923 22:16:34.954748 23372 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 9fbe5246cee441599e72be99705921e7: Generated new TSK 0
I20260923 22:16:34.954926 23372 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260923 22:16:34.961782 1567 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskRiQe_a/build/release/bin/kudu with pid 23293
2026-09-23T22:16:34Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/14 (1290 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/15
Loading random data
Initializing database '/tmp/dist-test-taskRiQe_a/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1790201539513534-1567-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Sep 23 22:16:35 dist-test-slave-s2fj krb5kdc[23392](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Sep 23 22:16:35 dist-test-slave-s2fj krb5kdc[23392](info): set up 2 sockets
Sep 23 22:16:35 dist-test-slave-s2fj krb5kdc[23392](info): commencing operation
krb5kdc: starting...
W20260923 22:16:35.972247 1567 mini_kdc.cc:121] Time spent starting KDC: real 0.993s user 0.000s sys 0.005s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskRiQe_a/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1790201539513534-1567-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskRiQe_a/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1790201539513534-1567-0/krb5kdc/test-user.keytab.
Sep 23 22:16:36 dist-test-slave-s2fj krb5kdc[23392](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790201796, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-09-23T22:16:36Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-09-23T22:16:36Z Disabled control of system clock
WARNING: no policy specified for kudu/127.1.135.254@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.1.135.254@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.1.135.254 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskRiQe_a/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1790201539513534-1567-0/krb5kdc/kudu_127.1.135.254.keytab.
Entry for principal kudu/127.1.135.254 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskRiQe_a/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1790201539513534-1567-0/krb5kdc/kudu_127.1.135.254.keytab.
I20260923 22:16:36.082780 1567 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskRiQe_a/build/release/bin/kudu
/tmp/dist-test-taskRiQe_a/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskRiQe_a/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1790201539513534-1567-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskRiQe_a/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1790201539513534-1567-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskRiQe_a/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1790201539513534-1567-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskRiQe_a/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1790201539513534-1567-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.1.135.254:42367
--webserver_interface=127.1.135.254
--webserver_port=0
--keytab_file=/tmp/dist-test-taskRiQe_a/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1790201539513534-1567-0/krb5kdc/kudu_127.1.135.254.keytab
--principal=kudu/127.1.135.254
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.1.135.212:43563
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.1.135.254:42367
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=required
--rpc_encryption=required
--rpc_encrypt_loopback_connections=false with env {KRB5CCNAME=/tmp/dist-test-taskRiQe_a/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1790201539513534-1567-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskRiQe_a/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1790201539513534-1567-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskRiQe_a/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1790201539513534-1567-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260923 22:16:36.162844 23405 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260923 22:16:36.163039 23405 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260923 22:16:36.163061 23405 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260923 22:16:36.164281 23405 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260923 22:16:36.164321 23405 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260923 22:16:36.164336 23405 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260923 22:16:36.164378 23405 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260923 22:16:36.164407 23405 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260923 22:16:36.165901 23405 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.1.135.212:43563
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskRiQe_a/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1790201539513534-1567-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskRiQe_a/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1790201539513534-1567-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.1.135.254:42367
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskRiQe_a/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1790201539513534-1567-0/krb5kdc/kudu_127.1.135.254.keytab
--principal=kudu/127.1.135.254
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.1.135.254:42367
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskRiQe_a/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1790201539513534-1567-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.1.135.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.23405
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskRiQe_a/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1790201539513534-1567-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 23 Sep 2026 20:53:37 UTC on ccycloud.quasar-rkviyx.root.comops.site
build id 4668083
I20260923 22:16:36.166157 23405 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260923 22:16:36.166390 23405 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260923 22:16:36.171491 23413 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260923 22:16:36.171505 23410 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260923 22:16:36.171504 23411 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260923 22:16:36.171732 23405 server_base.cc:1034] running on GCE node
I20260923 22:16:36.171993 23405 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260923 22:16:36.172305 23405 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260923 22:16:36.173449 23405 hybrid_clock.cc:648] HybridClock initialized: now 1790201796173440 us; error 32 us; skew 500 ppm
Sep 23 22:16:36 dist-test-slave-s2fj krb5kdc[23392](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790201796, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.135.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260923 22:16:36.175722 23405 init.cc:377] Logged in from keytab as kudu/127.1.135.254@KRBTEST.COM (short username kudu)
I20260923 22:16:36.176213 23405 webserver.cc:460] Webserver started at http://127.1.135.254:34129/ using document root <none> and password file <none>
I20260923 22:16:36.176460 23405 fs_manager.cc:359] Metadata directory not provided
I20260923 22:16:36.176503 23405 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260923 22:16:36.176645 23405 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260923 22:16:36.177718 23405 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskRiQe_a/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1790201539513534-1567-0/minicluster-data/master-0/data/instance:
uuid: "299748a88c0347c0b060085475d9d5bb"
format_stamp: "Formatted at 2026-09-23 22:16:36 on dist-test-slave-s2fj"
server_key: "7886e4bbe24f2cab40fe0e13ffa1ebd5"
server_key_iv: "84203ba2700d5b83ab7f6b0fde44201b"
server_key_version: "encryptionkey@0"
I20260923 22:16:36.178063 23405 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskRiQe_a/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1790201539513534-1567-0/minicluster-data/master-0/wal/instance:
uuid: "299748a88c0347c0b060085475d9d5bb"
format_stamp: "Formatted at 2026-09-23 22:16:36 on dist-test-slave-s2fj"
server_key: "7886e4bbe24f2cab40fe0e13ffa1ebd5"
server_key_iv: "84203ba2700d5b83ab7f6b0fde44201b"
server_key_version: "encryptionkey@0"
I20260923 22:16:36.179350 23405 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260923 22:16:36.180042 23420 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260923 22:16:36.180277 23405 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260923 22:16:36.180356 23405 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskRiQe_a/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1790201539513534-1567-0/minicluster-data/master-0/data,/tmp/dist-test-taskRiQe_a/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1790201539513534-1567-0/minicluster-data/master-0/wal
uuid: "299748a88c0347c0b060085475d9d5bb"
format_stamp: "Formatted at 2026-09-23 22:16:36 on dist-test-slave-s2fj"
server_key: "7886e4bbe24f2cab40fe0e13ffa1ebd5"
server_key_iv: "84203ba2700d5b83ab7f6b0fde44201b"
server_key_version: "encryptionkey@0"
I20260923 22:16:36.180454 23405 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskRiQe_a/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1790201539513534-1567-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskRiQe_a/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1790201539513534-1567-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskRiQe_a/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1790201539513534-1567-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260923 22:16:36.198427 23405 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260923 22:16:36.200448 23405 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260923 22:16:36.200599 23405 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260923 22:16:36.204470 23405 rpc_server.cc:307] RPC server started. Bound to: 127.1.135.254:42367
I20260923 22:16:36.204523 23472 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.1.135.254:42367 every 8 connection(s)
I20260923 22:16:36.204898 23405 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskRiQe_a/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1790201539513534-1567-0/minicluster-data/master-0/data/info.pb
I20260923 22:16:36.205822 23473 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260923 22:16:36.206945 1567 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskRiQe_a/build/release/bin/kudu as pid 23405
I20260923 22:16:36.207067 1567 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskRiQe_a/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1790201539513534-1567-0/minicluster-data/master-0/wal/instance
I20260923 22:16:36.207249 1567 external_mini_cluster.cc:1428] Setting key 52acce91c86506816ad42439d58bc1ff
I20260923 22:16:36.208736 23473 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 299748a88c0347c0b060085475d9d5bb: Bootstrap starting.
I20260923 22:16:36.209496 23473 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 299748a88c0347c0b060085475d9d5bb: Neither blocks nor log segments found. Creating new log.
I20260923 22:16:36.209807 23473 log.cc:824] T 00000000000000000000000000000000 P 299748a88c0347c0b060085475d9d5bb: Log is configured to *not* fsync() on all Append() calls
I20260923 22:16:36.210505 23473 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 299748a88c0347c0b060085475d9d5bb: No bootstrap required, opened a new log
I20260923 22:16:36.212983 23473 raft_consensus.cc:348] T 00000000000000000000000000000000 P 299748a88c0347c0b060085475d9d5bb [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "299748a88c0347c0b060085475d9d5bb" member_type: VOTER last_known_addr { host: "127.1.135.254" port: 42367 } }
I20260923 22:16:36.213198 23473 raft_consensus.cc:374] T 00000000000000000000000000000000 P 299748a88c0347c0b060085475d9d5bb [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260923 22:16:36.213241 23473 raft_consensus.cc:729] T 00000000000000000000000000000000 P 299748a88c0347c0b060085475d9d5bb [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 299748a88c0347c0b060085475d9d5bb, State: Initialized, Role: FOLLOWER
I20260923 22:16:36.213393 23473 consensus_queue.cc:260] T 00000000000000000000000000000000 P 299748a88c0347c0b060085475d9d5bb [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "299748a88c0347c0b060085475d9d5bb" member_type: VOTER last_known_addr { host: "127.1.135.254" port: 42367 } }
I20260923 22:16:36.213482 23473 raft_consensus.cc:388] T 00000000000000000000000000000000 P 299748a88c0347c0b060085475d9d5bb [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260923 22:16:36.213511 23473 raft_consensus.cc:482] T 00000000000000000000000000000000 P 299748a88c0347c0b060085475d9d5bb [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260923 22:16:36.213555 23473 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 299748a88c0347c0b060085475d9d5bb [term 0 FOLLOWER]: Advancing to term 1
I20260923 22:16:36.214187 23473 raft_consensus.cc:504] T 00000000000000000000000000000000 P 299748a88c0347c0b060085475d9d5bb [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "299748a88c0347c0b060085475d9d5bb" member_type: VOTER last_known_addr { host: "127.1.135.254" port: 42367 } }
Sep 23 22:16:36 dist-test-slave-s2fj krb5kdc[23392](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790201796, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.1.135.254@KRBTEST.COM
I20260923 22:16:36.214295 23473 leader_election.cc:302] T 00000000000000000000000000000000 P 299748a88c0347c0b060085475d9d5bb [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 299748a88c0347c0b060085475d9d5bb; no voters:
I20260923 22:16:36.214497 23473 leader_election.cc:288] T 00000000000000000000000000000000 P 299748a88c0347c0b060085475d9d5bb [CANDIDATE]: Term 1 election: Requested vote from peers
I20260923 22:16:36.214517 23478 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 299748a88c0347c0b060085475d9d5bb [term 1 FOLLOWER]: Leader election won for term 1
I20260923 22:16:36.214634 23478 raft_consensus.cc:686] T 00000000000000000000000000000000 P 299748a88c0347c0b060085475d9d5bb [term 1 LEADER]: Becoming Leader. State: Replica: 299748a88c0347c0b060085475d9d5bb, State: Running, Role: LEADER
I20260923 22:16:36.214720 23478 consensus_queue.cc:237] T 00000000000000000000000000000000 P 299748a88c0347c0b060085475d9d5bb [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "299748a88c0347c0b060085475d9d5bb" member_type: VOTER last_known_addr { host: "127.1.135.254" port: 42367 } }
I20260923 22:16:36.214771 23473 sys_catalog.cc:564] T 00000000000000000000000000000000 P 299748a88c0347c0b060085475d9d5bb [sys.catalog]: configured and running, proceeding with master startup.
I20260923 22:16:36.215025 23478 sys_catalog.cc:455] T 00000000000000000000000000000000 P 299748a88c0347c0b060085475d9d5bb [sys.catalog]: SysCatalogTable state changed. Reason: New leader 299748a88c0347c0b060085475d9d5bb. Latest consensus state: current_term: 1 leader_uuid: "299748a88c0347c0b060085475d9d5bb" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "299748a88c0347c0b060085475d9d5bb" member_type: VOTER last_known_addr { host: "127.1.135.254" port: 42367 } } }
I20260923 22:16:36.215132 23478 sys_catalog.cc:458] T 00000000000000000000000000000000 P 299748a88c0347c0b060085475d9d5bb [sys.catalog]: This master's current role is: LEADER
I20260923 22:16:36.215039 23479 sys_catalog.cc:455] T 00000000000000000000000000000000 P 299748a88c0347c0b060085475d9d5bb [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "299748a88c0347c0b060085475d9d5bb" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "299748a88c0347c0b060085475d9d5bb" member_type: VOTER last_known_addr { host: "127.1.135.254" port: 42367 } } }
I20260923 22:16:36.215298 23479 sys_catalog.cc:458] T 00000000000000000000000000000000 P 299748a88c0347c0b060085475d9d5bb [sys.catalog]: This master's current role is: LEADER
I20260923 22:16:36.215353 23483 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260923 22:16:36.215870 23483 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260923 22:16:36.217854 23483 catalog_manager.cc:1347] Generated new cluster ID: 3d393725e2a248b695472f29da601962
I20260923 22:16:36.217902 23483 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260923 22:16:36.218632 23476 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 22:16:36.207793 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45039 (local address 127.1.135.254:42367)
0923 22:16:36.207973 (+ 180us) server_negotiation.cc:206] Beginning negotiation
0923 22:16:36.207982 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0923 22:16:36.208017 (+ 35us) server_negotiation.cc:407] Connection header received
0923 22:16:36.208509 (+ 492us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 22:16:36.208516 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 22:16:36.208563 (+ 47us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 22:16:36.208684 (+ 121us) server_negotiation.cc:226] Negotiated authn=SASL
0923 22:16:36.209476 (+ 792us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 22:16:36.210495 (+ 1019us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 22:16:36.211546 (+ 1051us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 22:16:36.212010 (+ 464us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 22:16:36.212183 (+ 173us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 22:16:36.214893 (+ 2710us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0923 22:16:36.214897 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0923 22:16:36.214898 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0923 22:16:36.214911 (+ 13us) server_negotiation.cc:922] calling sasl_server_start()
0923 22:16:36.216976 (+ 2065us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0923 22:16:36.217369 (+ 393us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0923 22:16:36.217372 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0923 22:16:36.217372 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0923 22:16:36.217476 (+ 104us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0923 22:16:36.217710 (+ 234us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0923 22:16:36.217711 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0923 22:16:36.217711 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0923 22:16:36.218017 (+ 306us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0923 22:16:36.218166 (+ 149us) server_negotiation.cc:1035] Waiting for connection context
0923 22:16:36.218302 (+ 136us) server_negotiation.cc:299] Negotiation successful
0923 22:16:36.218447 (+ 145us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":128,"thread_start_us":91,"threads_started":1}
I20260923 22:16:36.219621 1567 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260923 22:16:36.228487 23476 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 22:16:36.221162 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57569 (local address 127.1.135.254:42367)
0923 22:16:36.221194 (+ 32us) server_negotiation.cc:206] Beginning negotiation
0923 22:16:36.221196 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0923 22:16:36.221340 (+ 144us) server_negotiation.cc:407] Connection header received
0923 22:16:36.221399 (+ 59us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 22:16:36.221400 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 22:16:36.221409 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 22:16:36.221464 (+ 55us) server_negotiation.cc:226] Negotiated authn=SASL
0923 22:16:36.222129 (+ 665us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 22:16:36.222845 (+ 716us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 22:16:36.223912 (+ 1067us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 22:16:36.224251 (+ 339us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 22:16:36.224307 (+ 56us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 22:16:36.225720 (+ 1413us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0923 22:16:36.225722 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0923 22:16:36.225722 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0923 22:16:36.225728 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0923 22:16:36.227300 (+ 1572us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0923 22:16:36.227690 (+ 390us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0923 22:16:36.227691 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0923 22:16:36.227691 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0923 22:16:36.227745 (+ 54us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0923 22:16:36.227948 (+ 203us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0923 22:16:36.227949 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0923 22:16:36.227949 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0923 22:16:36.228060 (+ 111us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0923 22:16:36.228187 (+ 127us) server_negotiation.cc:1035] Waiting for connection context
0923 22:16:36.228312 (+ 125us) server_negotiation.cc:299] Negotiation successful
0923 22:16:36.228390 (+ 78us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":15}
I20260923 22:16:36.229800 23483 catalog_manager.cc:1370] Generated new certificate authority record
I20260923 22:16:36.230288 23483 catalog_manager.cc:1504] Loading token signing keys...
I20260923 22:16:36.233520 23483 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 299748a88c0347c0b060085475d9d5bb: Generated new TSK 0
I20260923 22:16:36.233731 23483 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260923 22:16:36.236245 1567 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskRiQe_a/build/release/bin/kudu with pid 23405
2026-09-23T22:16:36Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/15 (1274 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/16
Loading random data
Initializing database '/tmp/dist-test-taskRiQe_a/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1790201539513534-1567-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Sep 23 22:16:36 dist-test-slave-s2fj krb5kdc[23504](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Sep 23 22:16:36 dist-test-slave-s2fj krb5kdc[23504](info): set up 2 sockets
Sep 23 22:16:36 dist-test-slave-s2fj krb5kdc[23504](info): commencing operation
krb5kdc: starting...
W20260923 22:16:37.252099 1567 mini_kdc.cc:121] Time spent starting KDC: real 0.998s user 0.002s sys 0.004s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskRiQe_a/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1790201539513534-1567-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskRiQe_a/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1790201539513534-1567-0/krb5kdc/test-user.keytab.
Sep 23 22:16:37 dist-test-slave-s2fj krb5kdc[23504](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790201797, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-09-23T22:16:37Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-09-23T22:16:37Z Disabled control of system clock
WARNING: no policy specified for kudu/127.1.135.254@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.1.135.254@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.1.135.254 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskRiQe_a/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1790201539513534-1567-0/krb5kdc/kudu_127.1.135.254.keytab.
Entry for principal kudu/127.1.135.254 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskRiQe_a/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1790201539513534-1567-0/krb5kdc/kudu_127.1.135.254.keytab.
I20260923 22:16:37.374677 1567 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskRiQe_a/build/release/bin/kudu
/tmp/dist-test-taskRiQe_a/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskRiQe_a/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1790201539513534-1567-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskRiQe_a/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1790201539513534-1567-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskRiQe_a/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1790201539513534-1567-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskRiQe_a/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1790201539513534-1567-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.1.135.254:44847
--webserver_interface=127.1.135.254
--webserver_port=0
--keytab_file=/tmp/dist-test-taskRiQe_a/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1790201539513534-1567-0/krb5kdc/kudu_127.1.135.254.keytab
--principal=kudu/127.1.135.254
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.1.135.212:44017
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.1.135.254:44847
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=required
--rpc_encrypt_loopback_connections=false with env {KRB5CCNAME=/tmp/dist-test-taskRiQe_a/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1790201539513534-1567-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskRiQe_a/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1790201539513534-1567-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskRiQe_a/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1790201539513534-1567-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260923 22:16:37.452945 23517 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260923 22:16:37.453140 23517 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260923 22:16:37.453162 23517 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260923 22:16:37.454210 23517 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260923 22:16:37.454243 23517 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260923 22:16:37.454257 23517 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260923 22:16:37.454270 23517 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260923 22:16:37.454282 23517 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260923 22:16:37.455639 23517 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.1.135.212:44017
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskRiQe_a/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1790201539513534-1567-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskRiQe_a/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1790201539513534-1567-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.1.135.254:44847
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskRiQe_a/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1790201539513534-1567-0/krb5kdc/kudu_127.1.135.254.keytab
--principal=kudu/127.1.135.254
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.1.135.254:44847
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskRiQe_a/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1790201539513534-1567-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.1.135.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.23517
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskRiQe_a/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1790201539513534-1567-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 23 Sep 2026 20:53:37 UTC on ccycloud.quasar-rkviyx.root.comops.site
build id 4668083
I20260923 22:16:37.455814 23517 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260923 22:16:37.455971 23517 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260923 22:16:37.460572 23525 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260923 22:16:37.460587 23522 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260923 22:16:37.460564 23523 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260923 22:16:37.460587 23517 server_base.cc:1034] running on GCE node
I20260923 22:16:37.461000 23517 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260923 22:16:37.461263 23517 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260923 22:16:37.462424 23517 hybrid_clock.cc:648] HybridClock initialized: now 1790201797462410 us; error 45 us; skew 500 ppm
Sep 23 22:16:37 dist-test-slave-s2fj krb5kdc[23504](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790201797, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.135.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260923 22:16:37.464862 23517 init.cc:377] Logged in from keytab as kudu/127.1.135.254@KRBTEST.COM (short username kudu)
I20260923 22:16:37.465337 23517 webserver.cc:460] Webserver started at http://127.1.135.254:45147/ using document root <none> and password file <none>
I20260923 22:16:37.465519 23517 fs_manager.cc:359] Metadata directory not provided
I20260923 22:16:37.465559 23517 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260923 22:16:37.465649 23517 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260923 22:16:37.466615 23517 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskRiQe_a/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1790201539513534-1567-0/minicluster-data/master-0/data/instance:
uuid: "724be6c83e77415d8526beeb96bece87"
format_stamp: "Formatted at 2026-09-23 22:16:37 on dist-test-slave-s2fj"
server_key: "1540fdd190f1508c3e3585a5f115f6af"
server_key_iv: "692497124c3ea29229d30c76a7a71c11"
server_key_version: "encryptionkey@0"
I20260923 22:16:37.466907 23517 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskRiQe_a/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1790201539513534-1567-0/minicluster-data/master-0/wal/instance:
uuid: "724be6c83e77415d8526beeb96bece87"
format_stamp: "Formatted at 2026-09-23 22:16:37 on dist-test-slave-s2fj"
server_key: "1540fdd190f1508c3e3585a5f115f6af"
server_key_iv: "692497124c3ea29229d30c76a7a71c11"
server_key_version: "encryptionkey@0"
I20260923 22:16:37.468140 23517 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.002s
I20260923 22:16:37.468827 23532 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260923 22:16:37.468993 23517 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260923 22:16:37.469085 23517 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskRiQe_a/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1790201539513534-1567-0/minicluster-data/master-0/data,/tmp/dist-test-taskRiQe_a/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1790201539513534-1567-0/minicluster-data/master-0/wal
uuid: "724be6c83e77415d8526beeb96bece87"
format_stamp: "Formatted at 2026-09-23 22:16:37 on dist-test-slave-s2fj"
server_key: "1540fdd190f1508c3e3585a5f115f6af"
server_key_iv: "692497124c3ea29229d30c76a7a71c11"
server_key_version: "encryptionkey@0"
I20260923 22:16:37.469166 23517 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskRiQe_a/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1790201539513534-1567-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskRiQe_a/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1790201539513534-1567-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskRiQe_a/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1790201539513534-1567-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260923 22:16:37.488943 23517 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260923 22:16:37.491390 23517 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260923 22:16:37.491555 23517 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260923 22:16:37.495672 23517 rpc_server.cc:307] RPC server started. Bound to: 127.1.135.254:44847
I20260923 22:16:37.495801 23584 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.1.135.254:44847 every 8 connection(s)
I20260923 22:16:37.496115 23517 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskRiQe_a/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1790201539513534-1567-0/minicluster-data/master-0/data/info.pb
I20260923 22:16:37.497012 23585 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260923 22:16:37.499132 1567 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskRiQe_a/build/release/bin/kudu as pid 23517
I20260923 22:16:37.499253 1567 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskRiQe_a/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1790201539513534-1567-0/minicluster-data/master-0/wal/instance
I20260923 22:16:37.499440 1567 external_mini_cluster.cc:1428] Setting key 3f6ad7fbbadb7aa6141faf8fdb3fdc85
I20260923 22:16:37.499452 23585 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 724be6c83e77415d8526beeb96bece87: Bootstrap starting.
I20260923 22:16:37.500175 23585 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 724be6c83e77415d8526beeb96bece87: Neither blocks nor log segments found. Creating new log.
I20260923 22:16:37.500487 23585 log.cc:824] T 00000000000000000000000000000000 P 724be6c83e77415d8526beeb96bece87: Log is configured to *not* fsync() on all Append() calls
I20260923 22:16:37.501345 23585 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 724be6c83e77415d8526beeb96bece87: No bootstrap required, opened a new log
I20260923 22:16:37.503532 23585 raft_consensus.cc:348] T 00000000000000000000000000000000 P 724be6c83e77415d8526beeb96bece87 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "724be6c83e77415d8526beeb96bece87" member_type: VOTER last_known_addr { host: "127.1.135.254" port: 44847 } }
I20260923 22:16:37.503674 23585 raft_consensus.cc:374] T 00000000000000000000000000000000 P 724be6c83e77415d8526beeb96bece87 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260923 22:16:37.503697 23585 raft_consensus.cc:729] T 00000000000000000000000000000000 P 724be6c83e77415d8526beeb96bece87 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 724be6c83e77415d8526beeb96bece87, State: Initialized, Role: FOLLOWER
I20260923 22:16:37.503901 23585 consensus_queue.cc:260] T 00000000000000000000000000000000 P 724be6c83e77415d8526beeb96bece87 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "724be6c83e77415d8526beeb96bece87" member_type: VOTER last_known_addr { host: "127.1.135.254" port: 44847 } }
I20260923 22:16:37.503980 23585 raft_consensus.cc:388] T 00000000000000000000000000000000 P 724be6c83e77415d8526beeb96bece87 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260923 22:16:37.504002 23585 raft_consensus.cc:482] T 00000000000000000000000000000000 P 724be6c83e77415d8526beeb96bece87 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260923 22:16:37.504083 23585 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 724be6c83e77415d8526beeb96bece87 [term 0 FOLLOWER]: Advancing to term 1
I20260923 22:16:37.504638 23585 raft_consensus.cc:504] T 00000000000000000000000000000000 P 724be6c83e77415d8526beeb96bece87 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "724be6c83e77415d8526beeb96bece87" member_type: VOTER last_known_addr { host: "127.1.135.254" port: 44847 } }
I20260923 22:16:37.504753 23585 leader_election.cc:302] T 00000000000000000000000000000000 P 724be6c83e77415d8526beeb96bece87 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 724be6c83e77415d8526beeb96bece87; no voters:
I20260923 22:16:37.504932 23585 leader_election.cc:288] T 00000000000000000000000000000000 P 724be6c83e77415d8526beeb96bece87 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260923 22:16:37.504964 23590 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 724be6c83e77415d8526beeb96bece87 [term 1 FOLLOWER]: Leader election won for term 1
I20260923 22:16:37.505177 23590 raft_consensus.cc:686] T 00000000000000000000000000000000 P 724be6c83e77415d8526beeb96bece87 [term 1 LEADER]: Becoming Leader. State: Replica: 724be6c83e77415d8526beeb96bece87, State: Running, Role: LEADER
I20260923 22:16:37.505223 23588 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 22:16:37.499961 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53331 (local address 127.1.135.254:44847)
0923 22:16:37.500161 (+ 200us) server_negotiation.cc:206] Beginning negotiation
0923 22:16:37.500170 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0923 22:16:37.500195 (+ 25us) server_negotiation.cc:407] Connection header received
0923 22:16:37.500760 (+ 565us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 22:16:37.500768 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 22:16:37.500833 (+ 65us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 22:16:37.500988 (+ 155us) server_negotiation.cc:226] Negotiated authn=SASL
0923 22:16:37.501239 (+ 251us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 22:16:37.502156 (+ 917us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 22:16:37.503294 (+ 1138us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 22:16:37.503703 (+ 409us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 22:16:37.503814 (+ 111us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 22:16:37.504489 (+ 675us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0923 22:16:37.504492 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0923 22:16:37.504493 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0923 22:16:37.504673 (+ 180us) server_negotiation.cc:922] calling sasl_server_start()
0923 22:16:37.504703 (+ 30us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0923 22:16:37.504713 (+ 10us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0923 22:16:37.504789 (+ 76us) server_negotiation.cc:1035] Waiting for connection context
0923 22:16:37.504899 (+ 110us) server_negotiation.cc:299] Negotiation successful
0923 22:16:37.504970 (+ 71us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":142,"thread_start_us":83,"threads_started":1}
I20260923 22:16:37.505237 23585 sys_catalog.cc:564] T 00000000000000000000000000000000 P 724be6c83e77415d8526beeb96bece87 [sys.catalog]: configured and running, proceeding with master startup.
I20260923 22:16:37.505303 23590 consensus_queue.cc:237] T 00000000000000000000000000000000 P 724be6c83e77415d8526beeb96bece87 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "724be6c83e77415d8526beeb96bece87" member_type: VOTER last_known_addr { host: "127.1.135.254" port: 44847 } }
I20260923 22:16:37.506039 23591 sys_catalog.cc:455] T 00000000000000000000000000000000 P 724be6c83e77415d8526beeb96bece87 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "724be6c83e77415d8526beeb96bece87" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "724be6c83e77415d8526beeb96bece87" member_type: VOTER last_known_addr { host: "127.1.135.254" port: 44847 } } }
I20260923 22:16:37.506201 23591 sys_catalog.cc:458] T 00000000000000000000000000000000 P 724be6c83e77415d8526beeb96bece87 [sys.catalog]: This master's current role is: LEADER
I20260923 22:16:37.506402 1567 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260923 22:16:37.506804 23592 sys_catalog.cc:455] T 00000000000000000000000000000000 P 724be6c83e77415d8526beeb96bece87 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 724be6c83e77415d8526beeb96bece87. Latest consensus state: current_term: 1 leader_uuid: "724be6c83e77415d8526beeb96bece87" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "724be6c83e77415d8526beeb96bece87" member_type: VOTER last_known_addr { host: "127.1.135.254" port: 44847 } } }
I20260923 22:16:37.507107 23592 sys_catalog.cc:458] T 00000000000000000000000000000000 P 724be6c83e77415d8526beeb96bece87 [sys.catalog]: This master's current role is: LEADER
W20260923 22:16:37.507304 23606 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 724be6c83e77415d8526beeb96bece87: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260923 22:16:37.507364 23606 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260923 22:16:37.507876 23599 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260923 22:16:37.508160 23599 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260923 22:16:37.509635 23599 catalog_manager.cc:1347] Generated new cluster ID: 4dcd9457308d46b492574298f8fc2c42
I20260923 22:16:37.509685 23599 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260923 22:16:37.515498 23588 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 22:16:37.507966 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33427 (local address 127.1.135.254:44847)
0923 22:16:37.508006 (+ 40us) server_negotiation.cc:206] Beginning negotiation
0923 22:16:37.508008 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0923 22:16:37.512273 (+ 4265us) server_negotiation.cc:407] Connection header received
0923 22:16:37.512336 (+ 63us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 22:16:37.512337 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 22:16:37.512347 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 22:16:37.512404 (+ 57us) server_negotiation.cc:226] Negotiated authn=SASL
0923 22:16:37.512530 (+ 126us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 22:16:37.513236 (+ 706us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 22:16:37.514261 (+ 1025us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 22:16:37.514542 (+ 281us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 22:16:37.514600 (+ 58us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 22:16:37.515266 (+ 666us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0923 22:16:37.515267 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0923 22:16:37.515267 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0923 22:16:37.515271 (+ 4us) server_negotiation.cc:922] calling sasl_server_start()
0923 22:16:37.515284 (+ 13us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0923 22:16:37.515288 (+ 4us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0923 22:16:37.515337 (+ 49us) server_negotiation.cc:1035] Waiting for connection context
0923 22:16:37.515408 (+ 71us) server_negotiation.cc:299] Negotiation successful
0923 22:16:37.515429 (+ 21us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":21}
I20260923 22:16:37.532346 23599 catalog_manager.cc:1370] Generated new certificate authority record
I20260923 22:16:37.533027 23599 catalog_manager.cc:1504] Loading token signing keys...
I20260923 22:16:37.539541 23599 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 724be6c83e77415d8526beeb96bece87: Generated new TSK 0
I20260923 22:16:37.539745 23599 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260923 22:16:37.550081 1567 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskRiQe_a/build/release/bin/kudu with pid 23517
2026-09-23T22:16:37Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/16 (1315 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/17
Loading random data
Initializing database '/tmp/dist-test-taskRiQe_a/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1790201539513534-1567-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Sep 23 22:16:37 dist-test-slave-s2fj krb5kdc[23616](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Sep 23 22:16:37 dist-test-slave-s2fj krb5kdc[23616](info): set up 2 sockets
Sep 23 22:16:37 dist-test-slave-s2fj krb5kdc[23616](info): commencing operation
krb5kdc: starting...
W20260923 22:16:38.592254 1567 mini_kdc.cc:121] Time spent starting KDC: real 1.021s user 0.000s sys 0.006s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskRiQe_a/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1790201539513534-1567-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskRiQe_a/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1790201539513534-1567-0/krb5kdc/test-user.keytab.
Sep 23 22:16:38 dist-test-slave-s2fj krb5kdc[23616](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790201798, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-09-23T22:16:38Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-09-23T22:16:38Z Disabled control of system clock
WARNING: no policy specified for kudu/127.1.135.254@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.1.135.254@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.1.135.254 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskRiQe_a/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1790201539513534-1567-0/krb5kdc/kudu_127.1.135.254.keytab.
Entry for principal kudu/127.1.135.254 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskRiQe_a/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1790201539513534-1567-0/krb5kdc/kudu_127.1.135.254.keytab.
I20260923 22:16:38.711436 1567 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskRiQe_a/build/release/bin/kudu
/tmp/dist-test-taskRiQe_a/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskRiQe_a/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1790201539513534-1567-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskRiQe_a/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1790201539513534-1567-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskRiQe_a/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1790201539513534-1567-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskRiQe_a/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1790201539513534-1567-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.1.135.254:38249
--webserver_interface=127.1.135.254
--webserver_port=0
--keytab_file=/tmp/dist-test-taskRiQe_a/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1790201539513534-1567-0/krb5kdc/kudu_127.1.135.254.keytab
--principal=kudu/127.1.135.254
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.1.135.212:45321
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.1.135.254:38249
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=disabled
--rpc_encrypt_loopback_connections=false with env {KRB5CCNAME=/tmp/dist-test-taskRiQe_a/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1790201539513534-1567-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskRiQe_a/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1790201539513534-1567-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskRiQe_a/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1790201539513534-1567-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260923 22:16:38.792361 23629 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260923 22:16:38.792666 23629 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260923 22:16:38.792724 23629 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260923 22:16:38.793926 23629 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260923 22:16:38.794013 23629 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260923 22:16:38.794039 23629 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260923 22:16:38.794059 23629 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260923 22:16:38.794090 23629 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260923 22:16:38.795645 23629 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.1.135.212:45321
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskRiQe_a/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1790201539513534-1567-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskRiQe_a/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1790201539513534-1567-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.1.135.254:38249
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskRiQe_a/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1790201539513534-1567-0/krb5kdc/kudu_127.1.135.254.keytab
--principal=kudu/127.1.135.254
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.1.135.254:38249
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskRiQe_a/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1790201539513534-1567-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.1.135.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.23629
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskRiQe_a/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1790201539513534-1567-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 23 Sep 2026 20:53:37 UTC on ccycloud.quasar-rkviyx.root.comops.site
build id 4668083
I20260923 22:16:38.795948 23629 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260923 22:16:38.796207 23629 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260923 22:16:38.801433 23635 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260923 22:16:38.801438 23637 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260923 22:16:38.801446 23634 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260923 22:16:38.801523 23629 server_base.cc:1034] running on GCE node
I20260923 22:16:38.801936 23629 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260923 22:16:38.802212 23629 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260923 22:16:38.803395 23629 hybrid_clock.cc:648] HybridClock initialized: now 1790201798803389 us; error 36 us; skew 500 ppm
Sep 23 22:16:38 dist-test-slave-s2fj krb5kdc[23616](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790201798, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.135.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260923 22:16:38.812383 23629 init.cc:377] Logged in from keytab as kudu/127.1.135.254@KRBTEST.COM (short username kudu)
I20260923 22:16:38.812857 23629 webserver.cc:460] Webserver started at http://127.1.135.254:45369/ using document root <none> and password file <none>
I20260923 22:16:38.813095 23629 fs_manager.cc:359] Metadata directory not provided
I20260923 22:16:38.813175 23629 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260923 22:16:38.813329 23629 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260923 22:16:38.814440 23629 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskRiQe_a/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1790201539513534-1567-0/minicluster-data/master-0/data/instance:
uuid: "82eeda873f304345ab4e01cf6dd4fb38"
format_stamp: "Formatted at 2026-09-23 22:16:38 on dist-test-slave-s2fj"
server_key: "2ff62f3bb353e0911aa5dbd3cd93f9a1"
server_key_iv: "d6de42035d3be9c06871b6e98b112bbf"
server_key_version: "encryptionkey@0"
I20260923 22:16:38.814810 23629 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskRiQe_a/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1790201539513534-1567-0/minicluster-data/master-0/wal/instance:
uuid: "82eeda873f304345ab4e01cf6dd4fb38"
format_stamp: "Formatted at 2026-09-23 22:16:38 on dist-test-slave-s2fj"
server_key: "2ff62f3bb353e0911aa5dbd3cd93f9a1"
server_key_iv: "d6de42035d3be9c06871b6e98b112bbf"
server_key_version: "encryptionkey@0"
I20260923 22:16:38.816226 23629 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260923 22:16:38.816949 23644 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260923 22:16:38.817137 23629 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260923 22:16:38.817238 23629 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskRiQe_a/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1790201539513534-1567-0/minicluster-data/master-0/data,/tmp/dist-test-taskRiQe_a/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1790201539513534-1567-0/minicluster-data/master-0/wal
uuid: "82eeda873f304345ab4e01cf6dd4fb38"
format_stamp: "Formatted at 2026-09-23 22:16:38 on dist-test-slave-s2fj"
server_key: "2ff62f3bb353e0911aa5dbd3cd93f9a1"
server_key_iv: "d6de42035d3be9c06871b6e98b112bbf"
server_key_version: "encryptionkey@0"
I20260923 22:16:38.817323 23629 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskRiQe_a/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1790201539513534-1567-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskRiQe_a/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1790201539513534-1567-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskRiQe_a/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1790201539513534-1567-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260923 22:16:38.820847 23629 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260923 22:16:38.822778 23629 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260923 22:16:38.822944 23629 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260923 22:16:38.827090 23629 rpc_server.cc:307] RPC server started. Bound to: 127.1.135.254:38249
I20260923 22:16:38.827122 23696 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.1.135.254:38249 every 8 connection(s)
I20260923 22:16:38.827469 23629 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskRiQe_a/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1790201539513534-1567-0/minicluster-data/master-0/data/info.pb
I20260923 22:16:38.828548 23697 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260923 22:16:38.830821 23697 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 82eeda873f304345ab4e01cf6dd4fb38: Bootstrap starting.
I20260923 22:16:38.831425 23697 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 82eeda873f304345ab4e01cf6dd4fb38: Neither blocks nor log segments found. Creating new log.
I20260923 22:16:38.831708 23697 log.cc:824] T 00000000000000000000000000000000 P 82eeda873f304345ab4e01cf6dd4fb38: Log is configured to *not* fsync() on all Append() calls
I20260923 22:16:38.832486 23697 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 82eeda873f304345ab4e01cf6dd4fb38: No bootstrap required, opened a new log
I20260923 22:16:38.834081 23697 raft_consensus.cc:348] T 00000000000000000000000000000000 P 82eeda873f304345ab4e01cf6dd4fb38 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "82eeda873f304345ab4e01cf6dd4fb38" member_type: VOTER last_known_addr { host: "127.1.135.254" port: 38249 } }
I20260923 22:16:38.834240 23697 raft_consensus.cc:374] T 00000000000000000000000000000000 P 82eeda873f304345ab4e01cf6dd4fb38 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260923 22:16:38.834264 23697 raft_consensus.cc:729] T 00000000000000000000000000000000 P 82eeda873f304345ab4e01cf6dd4fb38 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 82eeda873f304345ab4e01cf6dd4fb38, State: Initialized, Role: FOLLOWER
I20260923 22:16:38.834374 23697 consensus_queue.cc:260] T 00000000000000000000000000000000 P 82eeda873f304345ab4e01cf6dd4fb38 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "82eeda873f304345ab4e01cf6dd4fb38" member_type: VOTER last_known_addr { host: "127.1.135.254" port: 38249 } }
I20260923 22:16:38.834446 23697 raft_consensus.cc:388] T 00000000000000000000000000000000 P 82eeda873f304345ab4e01cf6dd4fb38 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260923 22:16:38.834484 23697 raft_consensus.cc:482] T 00000000000000000000000000000000 P 82eeda873f304345ab4e01cf6dd4fb38 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260923 22:16:38.834522 23697 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 82eeda873f304345ab4e01cf6dd4fb38 [term 0 FOLLOWER]: Advancing to term 1
I20260923 22:16:38.835405 23697 raft_consensus.cc:504] T 00000000000000000000000000000000 P 82eeda873f304345ab4e01cf6dd4fb38 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "82eeda873f304345ab4e01cf6dd4fb38" member_type: VOTER last_known_addr { host: "127.1.135.254" port: 38249 } }
I20260923 22:16:38.835523 23697 leader_election.cc:302] T 00000000000000000000000000000000 P 82eeda873f304345ab4e01cf6dd4fb38 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 82eeda873f304345ab4e01cf6dd4fb38; no voters:
I20260923 22:16:38.835713 23697 leader_election.cc:288] T 00000000000000000000000000000000 P 82eeda873f304345ab4e01cf6dd4fb38 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260923 22:16:38.835760 23700 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 82eeda873f304345ab4e01cf6dd4fb38 [term 1 FOLLOWER]: Leader election won for term 1
I20260923 22:16:38.835954 23700 raft_consensus.cc:686] T 00000000000000000000000000000000 P 82eeda873f304345ab4e01cf6dd4fb38 [term 1 LEADER]: Becoming Leader. State: Replica: 82eeda873f304345ab4e01cf6dd4fb38, State: Running, Role: LEADER
I20260923 22:16:38.835988 23697 sys_catalog.cc:564] T 00000000000000000000000000000000 P 82eeda873f304345ab4e01cf6dd4fb38 [sys.catalog]: configured and running, proceeding with master startup.
I20260923 22:16:38.836120 23700 consensus_queue.cc:237] T 00000000000000000000000000000000 P 82eeda873f304345ab4e01cf6dd4fb38 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "82eeda873f304345ab4e01cf6dd4fb38" member_type: VOTER last_known_addr { host: "127.1.135.254" port: 38249 } }
I20260923 22:16:38.836362 1567 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskRiQe_a/build/release/bin/kudu as pid 23629
I20260923 22:16:38.836463 1567 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskRiQe_a/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1790201539513534-1567-0/minicluster-data/master-0/wal/instance
I20260923 22:16:38.836701 1567 external_mini_cluster.cc:1428] Setting key 05dc05119979cabb308ff1f9e7b9d38b
I20260923 22:16:38.836732 23702 sys_catalog.cc:455] T 00000000000000000000000000000000 P 82eeda873f304345ab4e01cf6dd4fb38 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 82eeda873f304345ab4e01cf6dd4fb38. Latest consensus state: current_term: 1 leader_uuid: "82eeda873f304345ab4e01cf6dd4fb38" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "82eeda873f304345ab4e01cf6dd4fb38" member_type: VOTER last_known_addr { host: "127.1.135.254" port: 38249 } } }
I20260923 22:16:38.836802 23702 sys_catalog.cc:458] T 00000000000000000000000000000000 P 82eeda873f304345ab4e01cf6dd4fb38 [sys.catalog]: This master's current role is: LEADER
I20260923 22:16:38.837144 23701 sys_catalog.cc:455] T 00000000000000000000000000000000 P 82eeda873f304345ab4e01cf6dd4fb38 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "82eeda873f304345ab4e01cf6dd4fb38" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "82eeda873f304345ab4e01cf6dd4fb38" member_type: VOTER last_known_addr { host: "127.1.135.254" port: 38249 } } }
I20260923 22:16:38.837175 23709 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260923 22:16:38.837800 23709 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260923 22:16:38.838914 23701 sys_catalog.cc:458] T 00000000000000000000000000000000 P 82eeda873f304345ab4e01cf6dd4fb38 [sys.catalog]: This master's current role is: LEADER
I20260923 22:16:38.839352 23717 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 22:16:38.837404 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36557 (local address 127.1.135.254:38249)
0923 22:16:38.837562 (+ 158us) server_negotiation.cc:206] Beginning negotiation
0923 22:16:38.837573 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0923 22:16:38.837609 (+ 36us) server_negotiation.cc:407] Connection header received
0923 22:16:38.837738 (+ 129us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 22:16:38.837746 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 22:16:38.837780 (+ 34us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 22:16:38.838253 (+ 473us) server_negotiation.cc:226] Negotiated authn=SASL
0923 22:16:38.838763 (+ 510us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0923 22:16:38.838766 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0923 22:16:38.838767 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0923 22:16:38.838770 (+ 3us) server_negotiation.cc:922] calling sasl_server_start()
0923 22:16:38.838810 (+ 40us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0923 22:16:38.838825 (+ 15us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0923 22:16:38.839092 (+ 267us) server_negotiation.cc:1035] Waiting for connection context
0923 22:16:38.839124 (+ 32us) server_negotiation.cc:299] Negotiation successful
0923 22:16:38.839166 (+ 42us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":88,"thread_start_us":57,"threads_started":1}
I20260923 22:16:38.839494 23709 catalog_manager.cc:1347] Generated new cluster ID: 4b440d4c7dd244f8b3d9fb54d207e2b7
I20260923 22:16:38.839579 23709 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260923 22:16:38.840471 1567 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260923 22:16:38.843114 23717 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 22:16:38.841998 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39433 (local address 127.1.135.254:38249)
0923 22:16:38.842040 (+ 42us) server_negotiation.cc:206] Beginning negotiation
0923 22:16:38.842042 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0923 22:16:38.842185 (+ 143us) server_negotiation.cc:407] Connection header received
0923 22:16:38.842249 (+ 64us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 22:16:38.842250 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 22:16:38.842256 (+ 6us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 22:16:38.842306 (+ 50us) server_negotiation.cc:226] Negotiated authn=SASL
0923 22:16:38.842898 (+ 592us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0923 22:16:38.842899 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0923 22:16:38.842899 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0923 22:16:38.842903 (+ 4us) server_negotiation.cc:922] calling sasl_server_start()
0923 22:16:38.842917 (+ 14us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0923 22:16:38.842921 (+ 4us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0923 22:16:38.842974 (+ 53us) server_negotiation.cc:1035] Waiting for connection context
0923 22:16:38.843045 (+ 71us) server_negotiation.cc:299] Negotiation successful
0923 22:16:38.843057 (+ 12us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":24}
I20260923 22:16:38.853452 23709 catalog_manager.cc:1370] Generated new certificate authority record
I20260923 22:16:38.853626 23709 catalog_manager.cc:1504] Loading token signing keys...
I20260923 22:16:38.856528 23709 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 82eeda873f304345ab4e01cf6dd4fb38: Generated new TSK 0
I20260923 22:16:38.856701 23709 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260923 22:16:38.858332 1567 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskRiQe_a/build/release/bin/kudu with pid 23629
2026-09-23T22:16:38Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/17 (1305 ms)
[----------] 18 tests from AuthTokenIssuingTest (23300 ms total)
[----------] 2 tests from ConnectToFollowerMasterTest
[ RUN ] ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys/0
Loading random data
Initializing database '/tmp/dist-test-taskRiQe_a/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790201539513534-1567-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Sep 23 22:16:38 dist-test-slave-s2fj krb5kdc[23728](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Sep 23 22:16:38 dist-test-slave-s2fj krb5kdc[23728](info): set up 2 sockets
Sep 23 22:16:38 dist-test-slave-s2fj krb5kdc[23728](info): commencing operation
krb5kdc: starting...
W20260923 22:16:39.880539 1567 mini_kdc.cc:121] Time spent starting KDC: real 1.005s user 0.000s sys 0.006s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskRiQe_a/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790201539513534-1567-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskRiQe_a/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790201539513534-1567-0/krb5kdc/test-user.keytab.
Sep 23 22:16:39 dist-test-slave-s2fj krb5kdc[23728](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790201799, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-09-23T22:16:39Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-09-23T22:16:39Z Disabled control of system clock
WARNING: no policy specified for kudu/127.1.135.254@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.1.135.254@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.1.135.254 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskRiQe_a/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790201539513534-1567-0/krb5kdc/kudu_127.1.135.254.keytab.
Entry for principal kudu/127.1.135.254 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskRiQe_a/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790201539513534-1567-0/krb5kdc/kudu_127.1.135.254.keytab.
I20260923 22:16:40.002983 1567 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskRiQe_a/build/release/bin/kudu
/tmp/dist-test-taskRiQe_a/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskRiQe_a/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790201539513534-1567-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskRiQe_a/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790201539513534-1567-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskRiQe_a/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790201539513534-1567-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskRiQe_a/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790201539513534-1567-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.1.135.254:43661
--webserver_interface=127.1.135.254
--webserver_port=0
--keytab_file=/tmp/dist-test-taskRiQe_a/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790201539513534-1567-0/krb5kdc/kudu_127.1.135.254.keytab
--principal=kudu/127.1.135.254
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.1.135.212:36445
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.1.135.254:43661,127.1.135.253:36357,127.1.135.252:43047
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--enable_leader_failure_detection=false
--rpc_authentication=required
--rpc_encryption=optional with env {KRB5CCNAME=/tmp/dist-test-taskRiQe_a/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790201539513534-1567-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskRiQe_a/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790201539513534-1567-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskRiQe_a/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790201539513534-1567-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260923 22:16:40.085512 23741 flags.cc:425] Enabled unsafe flag: --enable_leader_failure_detection=false
W20260923 22:16:40.085804 23741 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260923 22:16:40.085836 23741 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260923 22:16:40.085884 23741 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260923 22:16:40.087071 23741 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260923 22:16:40.087157 23741 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260923 22:16:40.087184 23741 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260923 22:16:40.087212 23741 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260923 22:16:40.087244 23741 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260923 22:16:40.088814 23741 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.1.135.212:36445
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--enable_leader_failure_detection=false
--fs_data_dirs=/tmp/dist-test-taskRiQe_a/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790201539513534-1567-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskRiQe_a/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790201539513534-1567-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.1.135.254:43661,127.1.135.253:36357,127.1.135.252:43047
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskRiQe_a/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790201539513534-1567-0/krb5kdc/kudu_127.1.135.254.keytab
--principal=kudu/127.1.135.254
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.1.135.254:43661
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskRiQe_a/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790201539513534-1567-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.1.135.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.23741
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskRiQe_a/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790201539513534-1567-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 23 Sep 2026 20:53:37 UTC on ccycloud.quasar-rkviyx.root.comops.site
build id 4668083
I20260923 22:16:40.089126 23741 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260923 22:16:40.089371 23741 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260923 22:16:40.094305 23749 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260923 22:16:40.094298 23747 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260923 22:16:40.094530 23741 server_base.cc:1034] running on GCE node
W20260923 22:16:40.094298 23746 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260923 22:16:40.094905 23741 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260923 22:16:40.095220 23741 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260923 22:16:40.096393 23741 hybrid_clock.cc:648] HybridClock initialized: now 1790201800096365 us; error 46 us; skew 500 ppm
Sep 23 22:16:40 dist-test-slave-s2fj krb5kdc[23728](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790201800, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.135.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260923 22:16:40.098881 23741 init.cc:377] Logged in from keytab as kudu/127.1.135.254@KRBTEST.COM (short username kudu)
I20260923 22:16:40.099357 23741 webserver.cc:460] Webserver started at http://127.1.135.254:38747/ using document root <none> and password file <none>
I20260923 22:16:40.099548 23741 fs_manager.cc:359] Metadata directory not provided
I20260923 22:16:40.099591 23741 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260923 22:16:40.099679 23741 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260923 22:16:40.100785 23741 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskRiQe_a/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790201539513534-1567-0/minicluster-data/master-0/data/instance:
uuid: "a663d0fc71f24f16811ad260612eced4"
format_stamp: "Formatted at 2026-09-23 22:16:40 on dist-test-slave-s2fj"
server_key: "cbaea07b0e107b44649db6c16b32b3f9"
server_key_iv: "a7db6ea16dc220640468a5590381d0b5"
server_key_version: "encryptionkey@0"
I20260923 22:16:40.101096 23741 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskRiQe_a/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790201539513534-1567-0/minicluster-data/master-0/wal/instance:
uuid: "a663d0fc71f24f16811ad260612eced4"
format_stamp: "Formatted at 2026-09-23 22:16:40 on dist-test-slave-s2fj"
server_key: "cbaea07b0e107b44649db6c16b32b3f9"
server_key_iv: "a7db6ea16dc220640468a5590381d0b5"
server_key_version: "encryptionkey@0"
I20260923 22:16:40.102384 23741 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.000s
I20260923 22:16:40.103075 23756 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260923 22:16:40.103279 23741 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260923 22:16:40.103368 23741 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskRiQe_a/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790201539513534-1567-0/minicluster-data/master-0/data,/tmp/dist-test-taskRiQe_a/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790201539513534-1567-0/minicluster-data/master-0/wal
uuid: "a663d0fc71f24f16811ad260612eced4"
format_stamp: "Formatted at 2026-09-23 22:16:40 on dist-test-slave-s2fj"
server_key: "cbaea07b0e107b44649db6c16b32b3f9"
server_key_iv: "a7db6ea16dc220640468a5590381d0b5"
server_key_version: "encryptionkey@0"
I20260923 22:16:40.103425 23741 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskRiQe_a/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790201539513534-1567-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskRiQe_a/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790201539513534-1567-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskRiQe_a/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790201539513534-1567-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260923 22:16:40.124008 23741 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260923 22:16:40.126099 23741 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260923 22:16:40.126219 23741 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260923 22:16:40.130188 23741 rpc_server.cc:307] RPC server started. Bound to: 127.1.135.254:43661
I20260923 22:16:40.130293 23808 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.1.135.254:43661 every 8 connection(s)
I20260923 22:16:40.130638 23741 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskRiQe_a/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790201539513534-1567-0/minicluster-data/master-0/data/info.pb
I20260923 22:16:40.131729 23809 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260923 22:16:40.133376 23809 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.1.135.254" port: 43661 } has no permanent_uuid. Determining permanent_uuid...
I20260923 22:16:40.138334 1567 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskRiQe_a/build/release/bin/kudu as pid 23741
I20260923 22:16:40.138458 1567 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskRiQe_a/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790201539513534-1567-0/minicluster-data/master-0/wal/instance
I20260923 22:16:40.138674 1567 external_mini_cluster.cc:1428] Setting key e1848a51243a516e4eb79ceb411899d3
Sep 23 22:16:40 dist-test-slave-s2fj krb5kdc[23728](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790201800, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.135.254@KRBTEST.COM for kudu/127.1.135.254@KRBTEST.COM
I20260923 22:16:40.143823 23812 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 22:16:40.134222 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45076 (local address 127.1.135.254:43661)
0923 22:16:40.135008 (+ 786us) server_negotiation.cc:206] Beginning negotiation
0923 22:16:40.135012 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0923 22:16:40.135023 (+ 11us) server_negotiation.cc:407] Connection header received
0923 22:16:40.135083 (+ 60us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 22:16:40.135085 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 22:16:40.135100 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 22:16:40.135162 (+ 62us) server_negotiation.cc:226] Negotiated authn=SASL
0923 22:16:40.135759 (+ 597us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 22:16:40.136605 (+ 846us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 22:16:40.137510 (+ 905us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 22:16:40.137826 (+ 316us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 22:16:40.137905 (+ 79us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 22:16:40.140351 (+ 2446us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0923 22:16:40.140359 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0923 22:16:40.140359 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0923 22:16:40.140368 (+ 9us) server_negotiation.cc:922] calling sasl_server_start()
0923 22:16:40.142140 (+ 1772us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0923 22:16:40.142642 (+ 502us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0923 22:16:40.142645 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0923 22:16:40.142645 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0923 22:16:40.142726 (+ 81us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0923 22:16:40.142990 (+ 264us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0923 22:16:40.142991 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0923 22:16:40.142991 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0923 22:16:40.143211 (+ 220us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0923 22:16:40.143275 (+ 64us) server_negotiation.cc:1035] Waiting for connection context
0923 22:16:40.143533 (+ 258us) server_negotiation.cc:299] Negotiation successful
0923 22:16:40.143652 (+ 119us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":4,"server-negotiator.queue_time_us":766,"thread_start_us":77,"threads_started":1}
I20260923 22:16:40.143934 23811 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 22:16:40.134182 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.135.254:43661 (local address 127.0.0.1:45076)
0923 22:16:40.134364 (+ 182us) negotiation.cc:107] Waiting for socket to connect
0923 22:16:40.134400 (+ 36us) client_negotiation.cc:174] Beginning negotiation
0923 22:16:40.134875 (+ 475us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0923 22:16:40.135216 (+ 341us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0923 22:16:40.135218 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0923 22:16:40.135494 (+ 276us) client_negotiation.cc:189] Negotiated authn=SASL
0923 22:16:40.135661 (+ 167us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 22:16:40.135665 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 22:16:40.136725 (+ 1060us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 22:16:40.136726 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 22:16:40.137430 (+ 704us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 22:16:40.137432 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 22:16:40.137919 (+ 487us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 22:16:40.137919 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 22:16:40.137972 (+ 53us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 22:16:40.138697 (+ 725us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0923 22:16:40.138726 (+ 29us) client_negotiation.cc:656] calling sasl_client_start()
0923 22:16:40.140261 (+ 1535us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0923 22:16:40.142235 (+ 1974us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0923 22:16:40.142243 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0923 22:16:40.142246 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0923 22:16:40.142554 (+ 308us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0923 22:16:40.142800 (+ 246us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0923 22:16:40.142801 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0923 22:16:40.142801 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0923 22:16:40.142920 (+ 119us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0923 22:16:40.143288 (+ 368us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0923 22:16:40.143296 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0923 22:16:40.143383 (+ 87us) client_negotiation.cc:769] Sending connection context
0923 22:16:40.143500 (+ 117us) client_negotiation.cc:240] Negotiation successful
0923 22:16:40.143746 (+ 246us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":140,"mutex_wait_us":71,"thread_start_us":96,"threads_started":1}
I20260923 22:16:40.144797 23809 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.1.135.253" port: 36357 } has no permanent_uuid. Determining permanent_uuid...
I20260923 22:16:40.145202 23811 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 22:16:40.145057 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.135.253:36357 (local address 127.0.0.1:33302)
0923 22:16:40.145111 (+ 54us) negotiation.cc:107] Waiting for socket to connect
0923 22:16:40.145168 (+ 57us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.135.253:36357: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":34}
W20260923 22:16:40.145364 23758 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.1.135.253:36357: connect: Connection refused (error 111)
I20260923 22:16:40.145843 23811 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 22:16:40.145726 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.135.253:36357 (local address 127.0.0.1:33312)
0923 22:16:40.145779 (+ 53us) negotiation.cc:107] Waiting for socket to connect
0923 22:16:40.145811 (+ 32us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.135.253:36357: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":35}
W20260923 22:16:40.145957 23809 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.1.135.253:36357: Network error: Client connection negotiation failed: client connection to 127.1.135.253:36357: connect: Connection refused (error 111)
WARNING: no policy specified for kudu/127.1.135.253@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.1.135.253@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.1.135.253 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskRiQe_a/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790201539513534-1567-0/krb5kdc/kudu_127.1.135.253.keytab.
Entry for principal kudu/127.1.135.253 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskRiQe_a/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790201539513534-1567-0/krb5kdc/kudu_127.1.135.253.keytab.
I20260923 22:16:40.166848 1567 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskRiQe_a/build/release/bin/kudu
/tmp/dist-test-taskRiQe_a/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskRiQe_a/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790201539513534-1567-0/minicluster-data/master-1/wal
--fs_data_dirs=/tmp/dist-test-taskRiQe_a/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790201539513534-1567-0/minicluster-data/master-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskRiQe_a/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790201539513534-1567-0/minicluster-data/master-1/logs
--server_dump_info_path=/tmp/dist-test-taskRiQe_a/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790201539513534-1567-0/minicluster-data/master-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.1.135.253:36357
--webserver_interface=127.1.135.253
--webserver_port=0
--keytab_file=/tmp/dist-test-taskRiQe_a/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790201539513534-1567-0/krb5kdc/kudu_127.1.135.253.keytab
--principal=kudu/127.1.135.253
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.1.135.212:36445
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.1.135.254:43661,127.1.135.253:36357,127.1.135.252:43047
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--enable_leader_failure_detection=false
--rpc_authentication=required
--rpc_encryption=optional with env {KRB5CCNAME=/tmp/dist-test-taskRiQe_a/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790201539513534-1567-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskRiQe_a/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790201539513534-1567-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskRiQe_a/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790201539513534-1567-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260923 22:16:40.195222 23809 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.1.135.253" port: 36357 } attempt: 1
I20260923 22:16:40.195835 23811 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 22:16:40.195677 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.135.253:36357 (local address 127.0.0.1:33314)
0923 22:16:40.195746 (+ 69us) negotiation.cc:107] Waiting for socket to connect
0923 22:16:40.195798 (+ 52us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.135.253:36357: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":43}
I20260923 22:16:40.196276 23811 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 22:16:40.196178 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.135.253:36357 (local address 127.0.0.1:33330)
0923 22:16:40.196225 (+ 47us) negotiation.cc:107] Waiting for socket to connect
0923 22:16:40.196254 (+ 29us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.135.253:36357: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":32}
W20260923 22:16:40.196376 23809 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.1.135.253:36357: Network error: Client connection negotiation failed: client connection to 127.1.135.253:36357: connect: Connection refused (error 111)
W20260923 22:16:40.247556 23816 flags.cc:425] Enabled unsafe flag: --enable_leader_failure_detection=false
W20260923 22:16:40.247740 23816 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260923 22:16:40.247756 23816 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260923 22:16:40.247773 23816 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260923 22:16:40.248792 23816 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260923 22:16:40.248827 23816 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260923 22:16:40.248842 23816 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260923 22:16:40.248853 23816 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260923 22:16:40.248864 23816 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260923 22:16:40.250203 23816 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.1.135.212:36445
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--enable_leader_failure_detection=false
--fs_data_dirs=/tmp/dist-test-taskRiQe_a/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790201539513534-1567-0/minicluster-data/master-1/data
--fs_wal_dir=/tmp/dist-test-taskRiQe_a/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790201539513534-1567-0/minicluster-data/master-1/wal
--ipki_ca_key_size=768
--master_addresses=127.1.135.254:43661,127.1.135.253:36357,127.1.135.252:43047
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskRiQe_a/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790201539513534-1567-0/krb5kdc/kudu_127.1.135.253.keytab
--principal=kudu/127.1.135.253
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.1.135.253:36357
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskRiQe_a/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790201539513534-1567-0/minicluster-data/master-1/data/info.pb
--webserver_interface=127.1.135.253
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.23816
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskRiQe_a/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790201539513534-1567-0/minicluster-data/master-1/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 23 Sep 2026 20:53:37 UTC on ccycloud.quasar-rkviyx.root.comops.site
build id 4668083
I20260923 22:16:40.250375 23816 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260923 22:16:40.250519 23816 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260923 22:16:40.254976 23824 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260923 22:16:40.254968 23821 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260923 22:16:40.255028 23822 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260923 22:16:40.255148 23816 server_base.cc:1034] running on GCE node
I20260923 22:16:40.255340 23816 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260923 22:16:40.255612 23816 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260923 22:16:40.256745 23816 hybrid_clock.cc:648] HybridClock initialized: now 1790201800256732 us; error 28 us; skew 500 ppm
Sep 23 22:16:40 dist-test-slave-s2fj krb5kdc[23728](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790201800, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.135.253@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260923 22:16:40.258982 23816 init.cc:377] Logged in from keytab as kudu/127.1.135.253@KRBTEST.COM (short username kudu)
I20260923 22:16:40.259368 23816 webserver.cc:460] Webserver started at http://127.1.135.253:46205/ using document root <none> and password file <none>
I20260923 22:16:40.259582 23816 fs_manager.cc:359] Metadata directory not provided
I20260923 22:16:40.259624 23816 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260923 22:16:40.259776 23816 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260923 22:16:40.260797 23816 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskRiQe_a/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790201539513534-1567-0/minicluster-data/master-1/data/instance:
uuid: "5a3ce1613d3b498d884fe571f308e48a"
format_stamp: "Formatted at 2026-09-23 22:16:40 on dist-test-slave-s2fj"
server_key: "0437bc7868f0b33e2bd3812e5a1b9647"
server_key_iv: "fa3b60bab6a51d9c76a3b81a219491d5"
server_key_version: "encryptionkey@0"
I20260923 22:16:40.261134 23816 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskRiQe_a/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790201539513534-1567-0/minicluster-data/master-1/wal/instance:
uuid: "5a3ce1613d3b498d884fe571f308e48a"
format_stamp: "Formatted at 2026-09-23 22:16:40 on dist-test-slave-s2fj"
server_key: "0437bc7868f0b33e2bd3812e5a1b9647"
server_key_iv: "fa3b60bab6a51d9c76a3b81a219491d5"
server_key_version: "encryptionkey@0"
I20260923 22:16:40.262404 23816 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.000s
I20260923 22:16:40.263098 23831 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260923 22:16:40.263263 23816 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260923 22:16:40.263362 23816 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskRiQe_a/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790201539513534-1567-0/minicluster-data/master-1/data,/tmp/dist-test-taskRiQe_a/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790201539513534-1567-0/minicluster-data/master-1/wal
uuid: "5a3ce1613d3b498d884fe571f308e48a"
format_stamp: "Formatted at 2026-09-23 22:16:40 on dist-test-slave-s2fj"
server_key: "0437bc7868f0b33e2bd3812e5a1b9647"
server_key_iv: "fa3b60bab6a51d9c76a3b81a219491d5"
server_key_version: "encryptionkey@0"
I20260923 22:16:40.263438 23816 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskRiQe_a/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790201539513534-1567-0/minicluster-data/master-1/wal
metadata directory: /tmp/dist-test-taskRiQe_a/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790201539513534-1567-0/minicluster-data/master-1/wal
1 data directories: /tmp/dist-test-taskRiQe_a/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790201539513534-1567-0/minicluster-data/master-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260923 22:16:40.264513 23809 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.1.135.253" port: 36357 } attempt: 2
I20260923 22:16:40.264886 23811 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 22:16:40.264773 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.135.253:36357 (local address 127.0.0.1:33332)
0923 22:16:40.264826 (+ 53us) negotiation.cc:107] Waiting for socket to connect
0923 22:16:40.264859 (+ 33us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.135.253:36357: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":33}
I20260923 22:16:40.265165 23811 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 22:16:40.265078 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.135.253:36357 (local address 127.0.0.1:33344)
0923 22:16:40.265119 (+ 41us) negotiation.cc:107] Waiting for socket to connect
0923 22:16:40.265142 (+ 23us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.135.253:36357: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":29}
W20260923 22:16:40.265242 23809 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.1.135.253:36357: Network error: Client connection negotiation failed: client connection to 127.1.135.253:36357: connect: Connection refused (error 111)
I20260923 22:16:40.301324 23816 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260923 22:16:40.303112 23816 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260923 22:16:40.303277 23816 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260923 22:16:40.307134 23816 rpc_server.cc:307] RPC server started. Bound to: 127.1.135.253:36357
I20260923 22:16:40.307207 23883 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.1.135.253:36357 every 8 connection(s)
I20260923 22:16:40.307530 23816 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskRiQe_a/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790201539513534-1567-0/minicluster-data/master-1/data/info.pb
I20260923 22:16:40.308525 23884 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260923 22:16:40.310019 23884 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.1.135.254" port: 43661 } has no permanent_uuid. Determining permanent_uuid...
I20260923 22:16:40.311728 1567 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskRiQe_a/build/release/bin/kudu as pid 23816
I20260923 22:16:40.311852 1567 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskRiQe_a/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790201539513534-1567-0/minicluster-data/master-1/wal/instance
I20260923 22:16:40.312039 1567 external_mini_cluster.cc:1428] Setting key 2e1d965242da991401f9ab047031bc6d
Sep 23 22:16:40 dist-test-slave-s2fj krb5kdc[23728](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790201800, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.135.253@KRBTEST.COM for kudu/127.1.135.254@KRBTEST.COM
I20260923 22:16:40.320109 23812 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 22:16:40.310730 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45092 (local address 127.1.135.254:43661)
0923 22:16:40.310795 (+ 65us) server_negotiation.cc:206] Beginning negotiation
0923 22:16:40.310797 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0923 22:16:40.311031 (+ 234us) server_negotiation.cc:407] Connection header received
0923 22:16:40.311457 (+ 426us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 22:16:40.311458 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 22:16:40.311469 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 22:16:40.311519 (+ 50us) server_negotiation.cc:226] Negotiated authn=SASL
0923 22:16:40.312287 (+ 768us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 22:16:40.313069 (+ 782us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 22:16:40.314124 (+ 1055us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 22:16:40.314362 (+ 238us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 22:16:40.314432 (+ 70us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 22:16:40.317057 (+ 2625us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0923 22:16:40.317058 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0923 22:16:40.317058 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0923 22:16:40.317066 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0923 22:16:40.318524 (+ 1458us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0923 22:16:40.319064 (+ 540us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0923 22:16:40.319064 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0923 22:16:40.319065 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0923 22:16:40.319128 (+ 63us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0923 22:16:40.319423 (+ 295us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0923 22:16:40.319423 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0923 22:16:40.319423 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0923 22:16:40.319525 (+ 102us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0923 22:16:40.319574 (+ 49us) server_negotiation.cc:1035] Waiting for connection context
0923 22:16:40.319928 (+ 354us) server_negotiation.cc:299] Negotiation successful
0923 22:16:40.319992 (+ 64us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":36}
WARNING: no policy specified for kudu/127.1.135.252@KRBTEST.COM; defaulting to no policy
I20260923 22:16:40.320369 23886 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 22:16:40.310724 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.135.254:43661 (local address 127.0.0.1:45092)
0923 22:16:40.310898 (+ 174us) negotiation.cc:107] Waiting for socket to connect
0923 22:16:40.310939 (+ 41us) client_negotiation.cc:174] Beginning negotiation
0923 22:16:40.311382 (+ 443us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0923 22:16:40.311580 (+ 198us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0923 22:16:40.311583 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0923 22:16:40.311846 (+ 263us) client_negotiation.cc:189] Negotiated authn=SASL
0923 22:16:40.312180 (+ 334us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 22:16:40.312186 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 22:16:40.313185 (+ 999us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 22:16:40.313186 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 22:16:40.313986 (+ 800us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 22:16:40.313988 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 22:16:40.314481 (+ 493us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 22:16:40.314483 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 22:16:40.314617 (+ 134us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 22:16:40.315142 (+ 525us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0923 22:16:40.315150 (+ 8us) client_negotiation.cc:656] calling sasl_client_start()
0923 22:16:40.316923 (+ 1773us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0923 22:16:40.318618 (+ 1695us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0923 22:16:40.318626 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0923 22:16:40.318628 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0923 22:16:40.318975 (+ 347us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0923 22:16:40.319204 (+ 229us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0923 22:16:40.319205 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0923 22:16:40.319206 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0923 22:16:40.319349 (+ 143us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0923 22:16:40.319594 (+ 245us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0923 22:16:40.319600 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0923 22:16:40.319774 (+ 174us) client_negotiation.cc:769] Sending connection context
0923 22:16:40.319914 (+ 140us) client_negotiation.cc:240] Negotiation successful
0923 22:16:40.320135 (+ 221us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":131,"thread_start_us":85,"threads_started":1}
I20260923 22:16:40.320915 23884 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.1.135.253" port: 36357 } has no permanent_uuid. Determining permanent_uuid...
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.1.135.252@KRBTEST.COM" created.
Sep 23 22:16:40 dist-test-slave-s2fj krb5kdc[23728](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790201800, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.135.253@KRBTEST.COM for kudu/127.1.135.253@KRBTEST.COM
I20260923 22:16:40.328727 23888 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 22:16:40.321273 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33350 (local address 127.1.135.253:36357)
0923 22:16:40.321445 (+ 172us) server_negotiation.cc:206] Beginning negotiation
0923 22:16:40.321449 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0923 22:16:40.321464 (+ 15us) server_negotiation.cc:407] Connection header received
0923 22:16:40.321481 (+ 17us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 22:16:40.321482 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 22:16:40.321497 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 22:16:40.321593 (+ 96us) server_negotiation.cc:226] Negotiated authn=SASL
0923 22:16:40.321947 (+ 354us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 22:16:40.322744 (+ 797us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 22:16:40.323717 (+ 973us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 22:16:40.324044 (+ 327us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 22:16:40.324157 (+ 113us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 22:16:40.325835 (+ 1678us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0923 22:16:40.325841 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0923 22:16:40.325842 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0923 22:16:40.325847 (+ 5us) server_negotiation.cc:922] calling sasl_server_start()
0923 22:16:40.327371 (+ 1524us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0923 22:16:40.327789 (+ 418us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0923 22:16:40.327791 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0923 22:16:40.327791 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0923 22:16:40.327840 (+ 49us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0923 22:16:40.328041 (+ 201us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0923 22:16:40.328042 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0923 22:16:40.328042 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0923 22:16:40.328228 (+ 186us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0923 22:16:40.328293 (+ 65us) server_negotiation.cc:1035] Waiting for connection context
0923 22:16:40.328468 (+ 175us) server_negotiation.cc:299] Negotiation successful
0923 22:16:40.328613 (+ 145us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":74,"server-negotiator.queue_time_us":146,"thread_start_us":92,"threads_started":1}
I20260923 22:16:40.328749 23886 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 22:16:40.321197 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.135.253:36357 (local address 127.0.0.1:33350)
0923 22:16:40.321248 (+ 51us) negotiation.cc:107] Waiting for socket to connect
0923 22:16:40.321265 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0923 22:16:40.321297 (+ 32us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0923 22:16:40.321593 (+ 296us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0923 22:16:40.321594 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0923 22:16:40.321803 (+ 209us) client_negotiation.cc:189] Negotiated authn=SASL
0923 22:16:40.321855 (+ 52us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 22:16:40.321857 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 22:16:40.322833 (+ 976us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 22:16:40.322834 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 22:16:40.323624 (+ 790us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 22:16:40.323627 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 22:16:40.324156 (+ 529us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 22:16:40.324157 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 22:16:40.324221 (+ 64us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 22:16:40.324623 (+ 402us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0923 22:16:40.324626 (+ 3us) client_negotiation.cc:656] calling sasl_client_start()
0923 22:16:40.325766 (+ 1140us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0923 22:16:40.327459 (+ 1693us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0923 22:16:40.327460 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0923 22:16:40.327461 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0923 22:16:40.327700 (+ 239us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0923 22:16:40.327905 (+ 205us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0923 22:16:40.327905 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0923 22:16:40.327906 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0923 22:16:40.327976 (+ 70us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0923 22:16:40.328302 (+ 326us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0923 22:16:40.328303 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0923 22:16:40.328355 (+ 52us) client_negotiation.cc:769] Sending connection context
0923 22:16:40.328447 (+ 92us) client_negotiation.cc:240] Negotiation successful
0923 22:16:40.328569 (+ 122us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":34}
I20260923 22:16:40.329430 23884 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.1.135.252" port: 43047 } has no permanent_uuid. Determining permanent_uuid...
I20260923 22:16:40.329797 23886 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 22:16:40.329667 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.135.252:43047 (local address 127.0.0.1:45764)
0923 22:16:40.329720 (+ 53us) negotiation.cc:107] Waiting for socket to connect
0923 22:16:40.329773 (+ 53us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.135.252:43047: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":35}
W20260923 22:16:40.329943 23835 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.1.135.252:43047: connect: Connection refused (error 111)
I20260923 22:16:40.330394 23886 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 22:16:40.330287 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.135.252:43047 (local address 127.0.0.1:45766)
0923 22:16:40.330343 (+ 56us) negotiation.cc:107] Waiting for socket to connect
0923 22:16:40.330372 (+ 29us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.135.252:43047: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":40}
W20260923 22:16:40.330520 23884 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.1.135.252:43047: Network error: Client connection negotiation failed: client connection to 127.1.135.252:43047: connect: Connection refused (error 111)
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.1.135.252 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskRiQe_a/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790201539513534-1567-0/krb5kdc/kudu_127.1.135.252.keytab.
Entry for principal kudu/127.1.135.252 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskRiQe_a/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790201539513534-1567-0/krb5kdc/kudu_127.1.135.252.keytab.
I20260923 22:16:40.340001 1567 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskRiQe_a/build/release/bin/kudu
/tmp/dist-test-taskRiQe_a/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskRiQe_a/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790201539513534-1567-0/minicluster-data/master-2/wal
--fs_data_dirs=/tmp/dist-test-taskRiQe_a/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790201539513534-1567-0/minicluster-data/master-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskRiQe_a/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790201539513534-1567-0/minicluster-data/master-2/logs
--server_dump_info_path=/tmp/dist-test-taskRiQe_a/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790201539513534-1567-0/minicluster-data/master-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.1.135.252:43047
--webserver_interface=127.1.135.252
--webserver_port=0
--keytab_file=/tmp/dist-test-taskRiQe_a/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790201539513534-1567-0/krb5kdc/kudu_127.1.135.252.keytab
--principal=kudu/127.1.135.252
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.1.135.212:36445
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.1.135.254:43661,127.1.135.253:36357,127.1.135.252:43047
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--enable_leader_failure_detection=false
--rpc_authentication=required
--rpc_encryption=optional with env {KRB5CCNAME=/tmp/dist-test-taskRiQe_a/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790201539513534-1567-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskRiQe_a/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790201539513534-1567-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskRiQe_a/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790201539513534-1567-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260923 22:16:40.356374 23809 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.1.135.253" port: 36357 } attempt: 3
Sep 23 22:16:40 dist-test-slave-s2fj krb5kdc[23728](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790201800, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.135.254@KRBTEST.COM for kudu/127.1.135.253@KRBTEST.COM
I20260923 22:16:40.363075 23811 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 22:16:40.356695 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.135.253:36357 (local address 127.0.0.1:33362)
0923 22:16:40.356758 (+ 63us) negotiation.cc:107] Waiting for socket to connect
0923 22:16:40.356776 (+ 18us) client_negotiation.cc:174] Beginning negotiation
0923 22:16:40.356836 (+ 60us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0923 22:16:40.356987 (+ 151us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0923 22:16:40.356988 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0923 22:16:40.357149 (+ 161us) client_negotiation.cc:189] Negotiated authn=SASL
0923 22:16:40.357213 (+ 64us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 22:16:40.357215 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 22:16:40.357882 (+ 667us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 22:16:40.357883 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 22:16:40.358411 (+ 528us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 22:16:40.358413 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 22:16:40.358806 (+ 393us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 22:16:40.358807 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 22:16:40.358885 (+ 78us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 22:16:40.359364 (+ 479us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0923 22:16:40.359369 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0923 22:16:40.360547 (+ 1178us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0923 22:16:40.362078 (+ 1531us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0923 22:16:40.362079 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0923 22:16:40.362079 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0923 22:16:40.362248 (+ 169us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0923 22:16:40.362448 (+ 200us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0923 22:16:40.362448 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0923 22:16:40.362449 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0923 22:16:40.362506 (+ 57us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0923 22:16:40.362760 (+ 254us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0923 22:16:40.362761 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0923 22:16:40.362804 (+ 43us) client_negotiation.cc:769] Sending connection context
0923 22:16:40.362889 (+ 85us) client_negotiation.cc:240] Negotiation successful
0923 22:16:40.362982 (+ 93us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":41}
I20260923 22:16:40.363101 23888 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 22:16:40.356789 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33362 (local address 127.1.135.253:36357)
0923 22:16:40.356858 (+ 69us) server_negotiation.cc:206] Beginning negotiation
0923 22:16:40.356860 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0923 22:16:40.356872 (+ 12us) server_negotiation.cc:407] Connection header received
0923 22:16:40.356898 (+ 26us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 22:16:40.356899 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 22:16:40.356909 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 22:16:40.356981 (+ 72us) server_negotiation.cc:226] Negotiated authn=SASL
0923 22:16:40.357297 (+ 316us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 22:16:40.357799 (+ 502us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 22:16:40.358503 (+ 704us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 22:16:40.358730 (+ 227us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 22:16:40.358809 (+ 79us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 22:16:40.360648 (+ 1839us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0923 22:16:40.360654 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0923 22:16:40.360654 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0923 22:16:40.360661 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0923 22:16:40.361985 (+ 1324us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0923 22:16:40.362320 (+ 335us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0923 22:16:40.362322 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0923 22:16:40.362322 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0923 22:16:40.362380 (+ 58us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0923 22:16:40.362575 (+ 195us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0923 22:16:40.362576 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0923 22:16:40.362577 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0923 22:16:40.362691 (+ 114us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0923 22:16:40.362769 (+ 78us) server_negotiation.cc:1035] Waiting for connection context
0923 22:16:40.362926 (+ 157us) server_negotiation.cc:299] Negotiation successful
0923 22:16:40.363020 (+ 94us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":41}
I20260923 22:16:40.363471 23809 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.1.135.252" port: 43047 } has no permanent_uuid. Determining permanent_uuid...
I20260923 22:16:40.363814 23811 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 22:16:40.363714 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.135.252:43047 (local address 127.0.0.1:45776)
0923 22:16:40.363761 (+ 47us) negotiation.cc:107] Waiting for socket to connect
0923 22:16:40.363788 (+ 27us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.135.252:43047: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":31}
I20260923 22:16:40.364228 23811 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 22:16:40.364158 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.135.252:43047 (local address 127.0.0.1:45788)
0923 22:16:40.364191 (+ 33us) negotiation.cc:107] Waiting for socket to connect
0923 22:16:40.364212 (+ 21us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.135.252:43047: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":22}
W20260923 22:16:40.364326 23809 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.1.135.252:43047: Network error: Client connection negotiation failed: client connection to 127.1.135.252:43047: connect: Connection refused (error 111)
I20260923 22:16:40.379752 23884 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.1.135.252" port: 43047 } attempt: 1
I20260923 22:16:40.380295 23886 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 22:16:40.380170 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.135.252:43047 (local address 127.0.0.1:45802)
0923 22:16:40.380216 (+ 46us) negotiation.cc:107] Waiting for socket to connect
0923 22:16:40.380261 (+ 45us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.135.252:43047: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":23}
I20260923 22:16:40.380637 23886 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 22:16:40.380539 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.135.252:43047 (local address 127.0.0.1:45804)
0923 22:16:40.380591 (+ 52us) negotiation.cc:107] Waiting for socket to connect
0923 22:16:40.380618 (+ 27us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.135.252:43047: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":39}
W20260923 22:16:40.380735 23884 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.1.135.252:43047: Network error: Client connection negotiation failed: client connection to 127.1.135.252:43047: connect: Connection refused (error 111)
I20260923 22:16:40.395469 23809 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.1.135.252" port: 43047 } attempt: 1
I20260923 22:16:40.395888 23811 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 22:16:40.395766 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.135.252:43047 (local address 127.0.0.1:45808)
0923 22:16:40.395825 (+ 59us) negotiation.cc:107] Waiting for socket to connect
0923 22:16:40.395862 (+ 37us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.135.252:43047: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":38}
I20260923 22:16:40.396263 23811 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 22:16:40.396165 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.135.252:43047 (local address 127.0.0.1:45820)
0923 22:16:40.396211 (+ 46us) negotiation.cc:107] Waiting for socket to connect
0923 22:16:40.396238 (+ 27us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.135.252:43047: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":31}
W20260923 22:16:40.396358 23809 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.1.135.252:43047: Network error: Client connection negotiation failed: client connection to 127.1.135.252:43047: connect: Connection refused (error 111)
W20260923 22:16:40.421787 23891 flags.cc:425] Enabled unsafe flag: --enable_leader_failure_detection=false
W20260923 22:16:40.422011 23891 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260923 22:16:40.422029 23891 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260923 22:16:40.422083 23891 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260923 22:16:40.423200 23891 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260923 22:16:40.423241 23891 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260923 22:16:40.423255 23891 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260923 22:16:40.423308 23891 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260923 22:16:40.423332 23891 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260923 22:16:40.424933 23891 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.1.135.212:36445
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--enable_leader_failure_detection=false
--fs_data_dirs=/tmp/dist-test-taskRiQe_a/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790201539513534-1567-0/minicluster-data/master-2/data
--fs_wal_dir=/tmp/dist-test-taskRiQe_a/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790201539513534-1567-0/minicluster-data/master-2/wal
--ipki_ca_key_size=768
--master_addresses=127.1.135.254:43661,127.1.135.253:36357,127.1.135.252:43047
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskRiQe_a/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790201539513534-1567-0/krb5kdc/kudu_127.1.135.252.keytab
--principal=kudu/127.1.135.252
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.1.135.252:43047
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskRiQe_a/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790201539513534-1567-0/minicluster-data/master-2/data/info.pb
--webserver_interface=127.1.135.252
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.23891
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskRiQe_a/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790201539513534-1567-0/minicluster-data/master-2/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 23 Sep 2026 20:53:37 UTC on ccycloud.quasar-rkviyx.root.comops.site
build id 4668083
I20260923 22:16:40.425190 23891 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260923 22:16:40.425414 23891 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260923 22:16:40.430130 23899 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260923 22:16:40.430138 23896 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260923 22:16:40.430150 23897 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260923 22:16:40.430410 23891 server_base.cc:1034] running on GCE node
I20260923 22:16:40.430646 23891 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260923 22:16:40.430960 23891 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260923 22:16:40.432129 23891 hybrid_clock.cc:648] HybridClock initialized: now 1790201800432112 us; error 32 us; skew 500 ppm
Sep 23 22:16:40 dist-test-slave-s2fj krb5kdc[23728](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790201800, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.135.252@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260923 22:16:40.435115 23891 init.cc:377] Logged in from keytab as kudu/127.1.135.252@KRBTEST.COM (short username kudu)
I20260923 22:16:40.435667 23891 webserver.cc:460] Webserver started at http://127.1.135.252:41481/ using document root <none> and password file <none>
I20260923 22:16:40.435964 23891 fs_manager.cc:359] Metadata directory not provided
I20260923 22:16:40.436045 23891 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260923 22:16:40.436224 23891 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260923 22:16:40.437968 23891 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskRiQe_a/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790201539513534-1567-0/minicluster-data/master-2/data/instance:
uuid: "33bbb8df312f4ee7a608c9edb5621a4b"
format_stamp: "Formatted at 2026-09-23 22:16:40 on dist-test-slave-s2fj"
server_key: "6615c55b22d33bad185ec88144f7e567"
server_key_iv: "aa410564fe696b544507c30647f1f553"
server_key_version: "encryptionkey@0"
I20260923 22:16:40.438405 23891 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskRiQe_a/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790201539513534-1567-0/minicluster-data/master-2/wal/instance:
uuid: "33bbb8df312f4ee7a608c9edb5621a4b"
format_stamp: "Formatted at 2026-09-23 22:16:40 on dist-test-slave-s2fj"
server_key: "6615c55b22d33bad185ec88144f7e567"
server_key_iv: "aa410564fe696b544507c30647f1f553"
server_key_version: "encryptionkey@0"
I20260923 22:16:40.439838 23891 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.001s sys 0.000s
I20260923 22:16:40.441098 23906 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260923 22:16:40.441324 23891 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260923 22:16:40.441419 23891 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskRiQe_a/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790201539513534-1567-0/minicluster-data/master-2/data,/tmp/dist-test-taskRiQe_a/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790201539513534-1567-0/minicluster-data/master-2/wal
uuid: "33bbb8df312f4ee7a608c9edb5621a4b"
format_stamp: "Formatted at 2026-09-23 22:16:40 on dist-test-slave-s2fj"
server_key: "6615c55b22d33bad185ec88144f7e567"
server_key_iv: "aa410564fe696b544507c30647f1f553"
server_key_version: "encryptionkey@0"
I20260923 22:16:40.441493 23891 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskRiQe_a/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790201539513534-1567-0/minicluster-data/master-2/wal
metadata directory: /tmp/dist-test-taskRiQe_a/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790201539513534-1567-0/minicluster-data/master-2/wal
1 data directories: /tmp/dist-test-taskRiQe_a/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790201539513534-1567-0/minicluster-data/master-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260923 22:16:40.448855 23884 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.1.135.252" port: 43047 } attempt: 2
I20260923 22:16:40.449326 23886 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 22:16:40.449193 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.135.252:43047 (local address 127.0.0.1:45832)
0923 22:16:40.449253 (+ 60us) negotiation.cc:107] Waiting for socket to connect
0923 22:16:40.449282 (+ 29us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.135.252:43047: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":40}
I20260923 22:16:40.449717 23886 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 22:16:40.449629 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.135.252:43047 (local address 127.0.0.1:45842)
0923 22:16:40.449664 (+ 35us) negotiation.cc:107] Waiting for socket to connect
0923 22:16:40.449686 (+ 22us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.135.252:43047: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":21}
W20260923 22:16:40.449837 23884 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.1.135.252:43047: Network error: Client connection negotiation failed: client connection to 127.1.135.252:43047: connect: Connection refused (error 111)
I20260923 22:16:40.458504 23891 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260923 22:16:40.461511 23891 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260923 22:16:40.461680 23891 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260923 22:16:40.469182 23891 rpc_server.cc:307] RPC server started. Bound to: 127.1.135.252:43047
I20260923 22:16:40.469216 23958 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.1.135.252:43047 every 8 connection(s)
I20260923 22:16:40.469674 23891 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskRiQe_a/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790201539513534-1567-0/minicluster-data/master-2/data/info.pb
I20260923 22:16:40.470731 23959 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260923 22:16:40.471522 23809 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.1.135.252" port: 43047 } attempt: 2
I20260923 22:16:40.472684 23959 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.1.135.254" port: 43661 } has no permanent_uuid. Determining permanent_uuid...
I20260923 22:16:40.477515 1567 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskRiQe_a/build/release/bin/kudu as pid 23891
I20260923 22:16:40.477607 1567 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskRiQe_a/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790201539513534-1567-0/minicluster-data/master-2/wal/instance
I20260923 22:16:40.477751 1567 external_mini_cluster.cc:1428] Setting key 4c3fef7108f911873274e2ab6eddcf4d
Sep 23 22:16:40 dist-test-slave-s2fj krb5kdc[23728](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790201800, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.135.254@KRBTEST.COM for kudu/127.1.135.252@KRBTEST.COM
Sep 23 22:16:40 dist-test-slave-s2fj krb5kdc[23728](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790201800, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.135.252@KRBTEST.COM for kudu/127.1.135.254@KRBTEST.COM
I20260923 22:16:40.482164 23811 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 22:16:40.471894 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.135.252:43047 (local address 127.0.0.1:45856)
0923 22:16:40.471943 (+ 49us) negotiation.cc:107] Waiting for socket to connect
0923 22:16:40.471956 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0923 22:16:40.472020 (+ 64us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0923 22:16:40.473188 (+ 1168us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0923 22:16:40.473189 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0923 22:16:40.473372 (+ 183us) client_negotiation.cc:189] Negotiated authn=SASL
0923 22:16:40.473443 (+ 71us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 22:16:40.473446 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 22:16:40.474784 (+ 1338us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 22:16:40.474785 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 22:16:40.475429 (+ 644us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 22:16:40.475433 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 22:16:40.476243 (+ 810us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 22:16:40.476244 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 22:16:40.476300 (+ 56us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 22:16:40.476771 (+ 471us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0923 22:16:40.476776 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0923 22:16:40.478208 (+ 1432us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0923 22:16:40.480439 (+ 2231us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0923 22:16:40.480440 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0923 22:16:40.480441 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0923 22:16:40.480870 (+ 429us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0923 22:16:40.481181 (+ 311us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0923 22:16:40.481182 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0923 22:16:40.481182 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0923 22:16:40.481249 (+ 67us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0923 22:16:40.481679 (+ 430us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0923 22:16:40.481680 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0923 22:16:40.481726 (+ 46us) client_negotiation.cc:769] Sending connection context
0923 22:16:40.481818 (+ 92us) client_negotiation.cc:240] Negotiation successful
0923 22:16:40.482036 (+ 218us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":26}
I20260923 22:16:40.482326 23812 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 22:16:40.473519 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45102 (local address 127.1.135.254:43661)
0923 22:16:40.473565 (+ 46us) server_negotiation.cc:206] Beginning negotiation
0923 22:16:40.473567 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0923 22:16:40.473744 (+ 177us) server_negotiation.cc:407] Connection header received
0923 22:16:40.473779 (+ 35us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 22:16:40.473780 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 22:16:40.473790 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 22:16:40.473841 (+ 51us) server_negotiation.cc:226] Negotiated authn=SASL
0923 22:16:40.474437 (+ 596us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 22:16:40.475152 (+ 715us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 22:16:40.476220 (+ 1068us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 22:16:40.476417 (+ 197us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 22:16:40.476450 (+ 33us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 22:16:40.479089 (+ 2639us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0923 22:16:40.479091 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0923 22:16:40.479091 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0923 22:16:40.479099 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0923 22:16:40.481038 (+ 1939us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0923 22:16:40.481440 (+ 402us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0923 22:16:40.481441 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0923 22:16:40.481441 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0923 22:16:40.481495 (+ 54us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0923 22:16:40.481690 (+ 195us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0923 22:16:40.481691 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0923 22:16:40.481692 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0923 22:16:40.481823 (+ 131us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0923 22:16:40.481962 (+ 139us) server_negotiation.cc:1035] Waiting for connection context
0923 22:16:40.482152 (+ 190us) server_negotiation.cc:299] Negotiation successful
0923 22:16:40.482228 (+ 76us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":24}
I20260923 22:16:40.482486 23961 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 22:16:40.472125 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45856 (local address 127.1.135.252:43047)
0923 22:16:40.472310 (+ 185us) server_negotiation.cc:206] Beginning negotiation
0923 22:16:40.472331 (+ 21us) server_negotiation.cc:399] Waiting for connection header
0923 22:16:40.472357 (+ 26us) server_negotiation.cc:407] Connection header received
0923 22:16:40.472866 (+ 509us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 22:16:40.472895 (+ 29us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 22:16:40.473010 (+ 115us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 22:16:40.473188 (+ 178us) server_negotiation.cc:226] Negotiated authn=SASL
0923 22:16:40.473536 (+ 348us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 22:16:40.474694 (+ 1158us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 22:16:40.475531 (+ 837us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 22:16:40.476014 (+ 483us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 22:16:40.476224 (+ 210us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 22:16:40.478292 (+ 2068us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0923 22:16:40.478299 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0923 22:16:40.478299 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0923 22:16:40.478306 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0923 22:16:40.480352 (+ 2046us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0923 22:16:40.480993 (+ 641us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0923 22:16:40.480996 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0923 22:16:40.480997 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0923 22:16:40.481103 (+ 106us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0923 22:16:40.481302 (+ 199us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0923 22:16:40.481303 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0923 22:16:40.481304 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0923 22:16:40.481600 (+ 296us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0923 22:16:40.481981 (+ 381us) server_negotiation.cc:1035] Waiting for connection context
0923 22:16:40.482060 (+ 79us) server_negotiation.cc:299] Negotiation successful
0923 22:16:40.482235 (+ 175us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":23,"server-negotiator.queue_time_us":133,"thread_start_us":80,"threads_started":1}
I20260923 22:16:40.482852 23962 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 22:16:40.473446 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.135.254:43661 (local address 127.0.0.1:45102)
0923 22:16:40.473657 (+ 211us) negotiation.cc:107] Waiting for socket to connect
0923 22:16:40.473682 (+ 25us) client_negotiation.cc:174] Beginning negotiation
0923 22:16:40.473736 (+ 54us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0923 22:16:40.473889 (+ 153us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0923 22:16:40.473892 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0923 22:16:40.474211 (+ 319us) client_negotiation.cc:189] Negotiated authn=SASL
0923 22:16:40.474346 (+ 135us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 22:16:40.474351 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 22:16:40.475247 (+ 896us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 22:16:40.475248 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 22:16:40.476051 (+ 803us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 22:16:40.476053 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 22:16:40.476565 (+ 512us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 22:16:40.476566 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 22:16:40.476640 (+ 74us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 22:16:40.477002 (+ 362us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0923 22:16:40.477009 (+ 7us) client_negotiation.cc:656] calling sasl_client_start()
0923 22:16:40.478999 (+ 1990us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0923 22:16:40.481106 (+ 2107us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0923 22:16:40.481112 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0923 22:16:40.481114 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0923 22:16:40.481358 (+ 244us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0923 22:16:40.481570 (+ 212us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0923 22:16:40.481571 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0923 22:16:40.481572 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0923 22:16:40.481632 (+ 60us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0923 22:16:40.481893 (+ 261us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0923 22:16:40.481900 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0923 22:16:40.481983 (+ 83us) client_negotiation.cc:769] Sending connection context
0923 22:16:40.482107 (+ 124us) client_negotiation.cc:240] Negotiation successful
0923 22:16:40.482789 (+ 682us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":191,"mutex_wait_us":582,"thread_start_us":93,"threads_started":1}
I20260923 22:16:40.483316 23959 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.1.135.253" port: 36357 } has no permanent_uuid. Determining permanent_uuid...
I20260923 22:16:40.485100 23809 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P a663d0fc71f24f16811ad260612eced4: Bootstrap starting.
I20260923 22:16:40.485836 23809 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P a663d0fc71f24f16811ad260612eced4: Neither blocks nor log segments found. Creating new log.
Sep 23 22:16:40 dist-test-slave-s2fj krb5kdc[23728](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790201799, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.1.135.252@KRBTEST.COM
I20260923 22:16:40.486493 23809 log.cc:824] T 00000000000000000000000000000000 P a663d0fc71f24f16811ad260612eced4: Log is configured to *not* fsync() on all Append() calls
I20260923 22:16:40.487397 23809 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P a663d0fc71f24f16811ad260612eced4: No bootstrap required, opened a new log
Sep 23 22:16:40 dist-test-slave-s2fj krb5kdc[23728](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790201800, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.135.252@KRBTEST.COM for kudu/127.1.135.253@KRBTEST.COM
I20260923 22:16:40.489681 23809 raft_consensus.cc:348] T 00000000000000000000000000000000 P a663d0fc71f24f16811ad260612eced4 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a663d0fc71f24f16811ad260612eced4" member_type: VOTER last_known_addr { host: "127.1.135.254" port: 43661 } } peers { permanent_uuid: "5a3ce1613d3b498d884fe571f308e48a" member_type: VOTER last_known_addr { host: "127.1.135.253" port: 36357 } } peers { permanent_uuid: "33bbb8df312f4ee7a608c9edb5621a4b" member_type: VOTER last_known_addr { host: "127.1.135.252" port: 43047 } }
I20260923 22:16:40.489853 23809 raft_consensus.cc:729] T 00000000000000000000000000000000 P a663d0fc71f24f16811ad260612eced4 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: a663d0fc71f24f16811ad260612eced4, State: Initialized, Role: FOLLOWER
I20260923 22:16:40.489956 23809 consensus_queue.cc:260] T 00000000000000000000000000000000 P a663d0fc71f24f16811ad260612eced4 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a663d0fc71f24f16811ad260612eced4" member_type: VOTER last_known_addr { host: "127.1.135.254" port: 43661 } } peers { permanent_uuid: "5a3ce1613d3b498d884fe571f308e48a" member_type: VOTER last_known_addr { host: "127.1.135.253" port: 36357 } } peers { permanent_uuid: "33bbb8df312f4ee7a608c9edb5621a4b" member_type: VOTER last_known_addr { host: "127.1.135.252" port: 43047 } }
I20260923 22:16:40.490311 23809 sys_catalog.cc:564] T 00000000000000000000000000000000 P a663d0fc71f24f16811ad260612eced4 [sys.catalog]: configured and running, proceeding with master startup.
I20260923 22:16:40.490739 23964 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 22:16:40.478704 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45872 (local address 127.1.135.252:43047)
0923 22:16:40.478834 (+ 130us) server_negotiation.cc:206] Beginning negotiation
0923 22:16:40.478836 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0923 22:16:40.478844 (+ 8us) server_negotiation.cc:407] Connection header received
0923 22:16:40.478863 (+ 19us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 22:16:40.478863 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 22:16:40.478874 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 22:16:40.478954 (+ 80us) server_negotiation.cc:226] Negotiated authn=SASL
0923 22:16:40.479893 (+ 939us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 22:16:40.480857 (+ 964us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 22:16:40.482179 (+ 1322us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 22:16:40.483528 (+ 1349us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 22:16:40.483596 (+ 68us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 22:16:40.486492 (+ 2896us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0923 22:16:40.486493 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0923 22:16:40.486494 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0923 22:16:40.486500 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0923 22:16:40.489158 (+ 2658us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0923 22:16:40.489520 (+ 362us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0923 22:16:40.489521 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0923 22:16:40.489522 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0923 22:16:40.489582 (+ 60us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0923 22:16:40.489771 (+ 189us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0923 22:16:40.489772 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0923 22:16:40.489772 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0923 22:16:40.489934 (+ 162us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0923 22:16:40.490448 (+ 514us) server_negotiation.cc:1035] Waiting for connection context
0923 22:16:40.490504 (+ 56us) server_negotiation.cc:299] Negotiation successful
0923 22:16:40.490587 (+ 83us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":108,"thread_start_us":72,"threads_started":1}
I20260923 22:16:40.491269 23888 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 22:16:40.483707 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33366 (local address 127.1.135.253:36357)
0923 22:16:40.483749 (+ 42us) server_negotiation.cc:206] Beginning negotiation
0923 22:16:40.483751 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0923 22:16:40.483758 (+ 7us) server_negotiation.cc:407] Connection header received
0923 22:16:40.483781 (+ 23us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 22:16:40.483782 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 22:16:40.483792 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 22:16:40.483839 (+ 47us) server_negotiation.cc:226] Negotiated authn=SASL
0923 22:16:40.484250 (+ 411us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 22:16:40.484709 (+ 459us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 22:16:40.485588 (+ 879us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 22:16:40.485799 (+ 211us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 22:16:40.485892 (+ 93us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 22:16:40.487920 (+ 2028us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0923 22:16:40.487922 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0923 22:16:40.487922 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0923 22:16:40.487929 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0923 22:16:40.489695 (+ 1766us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0923 22:16:40.490132 (+ 437us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0923 22:16:40.490134 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0923 22:16:40.490134 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0923 22:16:40.490192 (+ 58us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0923 22:16:40.490404 (+ 212us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0923 22:16:40.490405 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0923 22:16:40.490405 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0923 22:16:40.490536 (+ 131us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0923 22:16:40.490610 (+ 74us) server_negotiation.cc:1035] Waiting for connection context
0923 22:16:40.491081 (+ 471us) server_negotiation.cc:299] Negotiation successful
0923 22:16:40.491165 (+ 84us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":22}
I20260923 22:16:40.491665 23962 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 22:16:40.483601 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.135.253:36357 (local address 127.0.0.1:33366)
0923 22:16:40.483635 (+ 34us) negotiation.cc:107] Waiting for socket to connect
0923 22:16:40.483649 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0923 22:16:40.483688 (+ 39us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0923 22:16:40.483887 (+ 199us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0923 22:16:40.483887 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0923 22:16:40.484044 (+ 157us) client_negotiation.cc:189] Negotiated authn=SASL
0923 22:16:40.484144 (+ 100us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 22:16:40.484146 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 22:16:40.484791 (+ 645us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 22:16:40.484793 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 22:16:40.485487 (+ 694us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 22:16:40.485489 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 22:16:40.485889 (+ 400us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 22:16:40.485890 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 22:16:40.485962 (+ 72us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 22:16:40.486374 (+ 412us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0923 22:16:40.486378 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0923 22:16:40.487814 (+ 1436us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0923 22:16:40.489785 (+ 1971us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0923 22:16:40.489786 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0923 22:16:40.489786 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0923 22:16:40.490044 (+ 258us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0923 22:16:40.490258 (+ 214us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0923 22:16:40.490259 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0923 22:16:40.490259 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0923 22:16:40.490336 (+ 77us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0923 22:16:40.490603 (+ 267us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0923 22:16:40.490604 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0923 22:16:40.490643 (+ 39us) client_negotiation.cc:769] Sending connection context
0923 22:16:40.490709 (+ 66us) client_negotiation.cc:240] Negotiation successful
0923 22:16:40.491574 (+ 865us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":18}
I20260923 22:16:40.492347 23959 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.1.135.252" port: 43047 } has no permanent_uuid. Determining permanent_uuid...
I20260923 22:16:40.491904 23966 sys_catalog.cc:455] T 00000000000000000000000000000000 P a663d0fc71f24f16811ad260612eced4 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a663d0fc71f24f16811ad260612eced4" member_type: VOTER last_known_addr { host: "127.1.135.254" port: 43661 } } peers { permanent_uuid: "5a3ce1613d3b498d884fe571f308e48a" member_type: VOTER last_known_addr { host: "127.1.135.253" port: 36357 } } peers { permanent_uuid: "33bbb8df312f4ee7a608c9edb5621a4b" member_type: VOTER last_known_addr { host: "127.1.135.252" port: 43047 } } }
I20260923 22:16:40.492825 23966 sys_catalog.cc:458] T 00000000000000000000000000000000 P a663d0fc71f24f16811ad260612eced4 [sys.catalog]: This master's current role is: FOLLOWER
W20260923 22:16:40.492281 23977 catalog_manager.cc:1558] T 00000000000000000000000000000000 P a663d0fc71f24f16811ad260612eced4: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260923 22:16:40.493026 23977 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260923 22:16:40.493691 23741 master_runner.cc:186] Error getting master registration for 127.1.135.253:36357: OK, instance_id { permanent_uuid: "5a3ce1613d3b498d884fe571f308e48a" instance_seqno: 1790201800306106 } error { code: CATALOG_MANAGER_NOT_INITIALIZED status { code: SERVICE_UNAVAILABLE message: "Catalog manager is not initialized. State: Starting" } }
Sep 23 22:16:40 dist-test-slave-s2fj krb5kdc[23728](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790201799, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.1.135.254@KRBTEST.COM
I20260923 22:16:40.496683 23741 master_runner.cc:418] Couldn't verify the masters in the cluster. Trying again...
Sep 23 22:16:40 dist-test-slave-s2fj krb5kdc[23728](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790201800, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.135.252@KRBTEST.COM for kudu/127.1.135.252@KRBTEST.COM
I20260923 22:16:40.500385 23812 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 22:16:40.491478 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45114 (local address 127.1.135.254:43661)
0923 22:16:40.491510 (+ 32us) server_negotiation.cc:206] Beginning negotiation
0923 22:16:40.491512 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0923 22:16:40.491521 (+ 9us) server_negotiation.cc:407] Connection header received
0923 22:16:40.491542 (+ 21us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 22:16:40.491543 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 22:16:40.491552 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 22:16:40.491603 (+ 51us) server_negotiation.cc:226] Negotiated authn=SASL
0923 22:16:40.492500 (+ 897us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 22:16:40.493173 (+ 673us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 22:16:40.494095 (+ 922us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 22:16:40.494358 (+ 263us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 22:16:40.494416 (+ 58us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 22:16:40.497249 (+ 2833us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0923 22:16:40.497251 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0923 22:16:40.497251 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0923 22:16:40.497258 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0923 22:16:40.498946 (+ 1688us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0923 22:16:40.499332 (+ 386us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0923 22:16:40.499333 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0923 22:16:40.499334 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0923 22:16:40.499385 (+ 51us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0923 22:16:40.499587 (+ 202us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0923 22:16:40.499588 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0923 22:16:40.499589 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0923 22:16:40.499708 (+ 119us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0923 22:16:40.499830 (+ 122us) server_negotiation.cc:1035] Waiting for connection context
0923 22:16:40.500165 (+ 335us) server_negotiation.cc:299] Negotiation successful
0923 22:16:40.500252 (+ 87us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":17}
I20260923 22:16:40.504160 23962 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 22:16:40.492598 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.135.252:43047 (local address 127.0.0.1:45878)
0923 22:16:40.492663 (+ 65us) negotiation.cc:107] Waiting for socket to connect
0923 22:16:40.492671 (+ 8us) client_negotiation.cc:174] Beginning negotiation
0923 22:16:40.492703 (+ 32us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0923 22:16:40.493343 (+ 640us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0923 22:16:40.493344 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0923 22:16:40.493486 (+ 142us) client_negotiation.cc:189] Negotiated authn=SASL
0923 22:16:40.493515 (+ 29us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 22:16:40.493516 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 22:16:40.494390 (+ 874us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 22:16:40.494391 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 22:16:40.495167 (+ 776us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 22:16:40.495169 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 22:16:40.495653 (+ 484us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 22:16:40.495654 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 22:16:40.495698 (+ 44us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 22:16:40.496772 (+ 1074us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0923 22:16:40.496777 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0923 22:16:40.498111 (+ 1334us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0923 22:16:40.502819 (+ 4708us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0923 22:16:40.502819 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0923 22:16:40.502820 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0923 22:16:40.503008 (+ 188us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0923 22:16:40.503194 (+ 186us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0923 22:16:40.503195 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0923 22:16:40.503196 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0923 22:16:40.503250 (+ 54us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0923 22:16:40.503584 (+ 334us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0923 22:16:40.503585 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0923 22:16:40.503621 (+ 36us) client_negotiation.cc:769] Sending connection context
0923 22:16:40.503920 (+ 299us) client_negotiation.cc:240] Negotiation successful
0923 22:16:40.504016 (+ 96us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":51}
I20260923 22:16:40.504171 23964 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 22:16:40.492695 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45878 (local address 127.1.135.252:43047)
0923 22:16:40.493180 (+ 485us) server_negotiation.cc:206] Beginning negotiation
0923 22:16:40.493182 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0923 22:16:40.493189 (+ 7us) server_negotiation.cc:407] Connection header received
0923 22:16:40.493207 (+ 18us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 22:16:40.493208 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 22:16:40.493217 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 22:16:40.493605 (+ 388us) server_negotiation.cc:226] Negotiated authn=SASL
0923 22:16:40.493628 (+ 23us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 22:16:40.494295 (+ 667us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 22:16:40.495254 (+ 959us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 22:16:40.495585 (+ 331us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 22:16:40.495823 (+ 238us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 22:16:40.498232 (+ 2409us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0923 22:16:40.498233 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0923 22:16:40.498233 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0923 22:16:40.498240 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0923 22:16:40.502728 (+ 4488us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0923 22:16:40.503080 (+ 352us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0923 22:16:40.503080 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0923 22:16:40.503081 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0923 22:16:40.503128 (+ 47us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0923 22:16:40.503313 (+ 185us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0923 22:16:40.503313 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0923 22:16:40.503313 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0923 22:16:40.503492 (+ 179us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0923 22:16:40.503581 (+ 89us) server_negotiation.cc:1035] Waiting for connection context
0923 22:16:40.503941 (+ 360us) server_negotiation.cc:299] Negotiation successful
0923 22:16:40.504092 (+ 151us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":49,"server-negotiator.queue_time_us":465}
I20260923 22:16:40.506162 23959 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 33bbb8df312f4ee7a608c9edb5621a4b: Bootstrap starting.
I20260923 22:16:40.507187 23959 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 33bbb8df312f4ee7a608c9edb5621a4b: Neither blocks nor log segments found. Creating new log.
I20260923 22:16:40.507524 23959 log.cc:824] T 00000000000000000000000000000000 P 33bbb8df312f4ee7a608c9edb5621a4b: Log is configured to *not* fsync() on all Append() calls
Sep 23 22:16:40 dist-test-slave-s2fj krb5kdc[23728](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790201799, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.1.135.253@KRBTEST.COM
I20260923 22:16:40.508589 23959 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 33bbb8df312f4ee7a608c9edb5621a4b: No bootstrap required, opened a new log
I20260923 22:16:40.511102 23959 raft_consensus.cc:348] T 00000000000000000000000000000000 P 33bbb8df312f4ee7a608c9edb5621a4b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a663d0fc71f24f16811ad260612eced4" member_type: VOTER last_known_addr { host: "127.1.135.254" port: 43661 } } peers { permanent_uuid: "5a3ce1613d3b498d884fe571f308e48a" member_type: VOTER last_known_addr { host: "127.1.135.253" port: 36357 } } peers { permanent_uuid: "33bbb8df312f4ee7a608c9edb5621a4b" member_type: VOTER last_known_addr { host: "127.1.135.252" port: 43047 } }
I20260923 22:16:40.511271 23959 raft_consensus.cc:729] T 00000000000000000000000000000000 P 33bbb8df312f4ee7a608c9edb5621a4b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 33bbb8df312f4ee7a608c9edb5621a4b, State: Initialized, Role: FOLLOWER
I20260923 22:16:40.511364 23959 consensus_queue.cc:260] T 00000000000000000000000000000000 P 33bbb8df312f4ee7a608c9edb5621a4b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a663d0fc71f24f16811ad260612eced4" member_type: VOTER last_known_addr { host: "127.1.135.254" port: 43661 } } peers { permanent_uuid: "5a3ce1613d3b498d884fe571f308e48a" member_type: VOTER last_known_addr { host: "127.1.135.253" port: 36357 } } peers { permanent_uuid: "33bbb8df312f4ee7a608c9edb5621a4b" member_type: VOTER last_known_addr { host: "127.1.135.252" port: 43047 } }
I20260923 22:16:40.511695 23959 sys_catalog.cc:564] T 00000000000000000000000000000000 P 33bbb8df312f4ee7a608c9edb5621a4b [sys.catalog]: configured and running, proceeding with master startup.
I20260923 22:16:40.511991 23888 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 22:16:40.503198 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33370 (local address 127.1.135.253:36357)
0923 22:16:40.503289 (+ 91us) server_negotiation.cc:206] Beginning negotiation
0923 22:16:40.503292 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0923 22:16:40.503303 (+ 11us) server_negotiation.cc:407] Connection header received
0923 22:16:40.503330 (+ 27us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 22:16:40.503332 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 22:16:40.503344 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 22:16:40.503411 (+ 67us) server_negotiation.cc:226] Negotiated authn=SASL
0923 22:16:40.504042 (+ 631us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 22:16:40.504753 (+ 711us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 22:16:40.505711 (+ 958us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 22:16:40.506000 (+ 289us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 22:16:40.506071 (+ 71us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 22:16:40.508844 (+ 2773us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0923 22:16:40.508845 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0923 22:16:40.508846 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0923 22:16:40.508865 (+ 19us) server_negotiation.cc:922] calling sasl_server_start()
0923 22:16:40.510738 (+ 1873us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0923 22:16:40.511100 (+ 362us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0923 22:16:40.511101 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0923 22:16:40.511102 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0923 22:16:40.511173 (+ 71us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0923 22:16:40.511417 (+ 244us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0923 22:16:40.511418 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0923 22:16:40.511418 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0923 22:16:40.511562 (+ 144us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0923 22:16:40.511771 (+ 209us) server_negotiation.cc:1035] Waiting for connection context
0923 22:16:40.511814 (+ 43us) server_negotiation.cc:299] Negotiation successful
0923 22:16:40.511897 (+ 83us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":63}
I20260923 22:16:40.512053 23980 sys_catalog.cc:455] T 00000000000000000000000000000000 P 33bbb8df312f4ee7a608c9edb5621a4b [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a663d0fc71f24f16811ad260612eced4" member_type: VOTER last_known_addr { host: "127.1.135.254" port: 43661 } } peers { permanent_uuid: "5a3ce1613d3b498d884fe571f308e48a" member_type: VOTER last_known_addr { host: "127.1.135.253" port: 36357 } } peers { permanent_uuid: "33bbb8df312f4ee7a608c9edb5621a4b" member_type: VOTER last_known_addr { host: "127.1.135.252" port: 43047 } } }
I20260923 22:16:40.512223 23980 sys_catalog.cc:458] T 00000000000000000000000000000000 P 33bbb8df312f4ee7a608c9edb5621a4b [sys.catalog]: This master's current role is: FOLLOWER
I20260923 22:16:40.513625 1567 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
W20260923 22:16:40.513908 23991 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 33bbb8df312f4ee7a608c9edb5621a4b: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260923 22:16:40.513954 23991 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260923 22:16:40.514544 23783 tablet_service.cc:1939] Received Run Leader Election RPC: tablet_id: "00000000000000000000000000000000"
dest_uuid: "a663d0fc71f24f16811ad260612eced4"
from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:45114
I20260923 22:16:40.514680 23783 raft_consensus.cc:482] T 00000000000000000000000000000000 P a663d0fc71f24f16811ad260612eced4 [term 0 FOLLOWER]: Starting forced leader election (received explicit request)
I20260923 22:16:40.514750 23783 raft_consensus.cc:3037] T 00000000000000000000000000000000 P a663d0fc71f24f16811ad260612eced4 [term 0 FOLLOWER]: Advancing to term 1
I20260923 22:16:40.515671 23783 raft_consensus.cc:504] T 00000000000000000000000000000000 P a663d0fc71f24f16811ad260612eced4 [term 1 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a663d0fc71f24f16811ad260612eced4" member_type: VOTER last_known_addr { host: "127.1.135.254" port: 43661 } } peers { permanent_uuid: "5a3ce1613d3b498d884fe571f308e48a" member_type: VOTER last_known_addr { host: "127.1.135.253" port: 36357 } } peers { permanent_uuid: "33bbb8df312f4ee7a608c9edb5621a4b" member_type: VOTER last_known_addr { host: "127.1.135.252" port: 43047 } }
I20260923 22:16:40.515902 23783 leader_election.cc:288] T 00000000000000000000000000000000 P a663d0fc71f24f16811ad260612eced4 [CANDIDATE]: Term 1 election: Requested vote from peers 5a3ce1613d3b498d884fe571f308e48a (127.1.135.253:36357), 33bbb8df312f4ee7a608c9edb5621a4b (127.1.135.252:43047)
I20260923 22:16:40.516228 23933 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "a663d0fc71f24f16811ad260612eced4" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: true dest_uuid: "33bbb8df312f4ee7a608c9edb5621a4b"
I20260923 22:16:40.516332 23933 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 33bbb8df312f4ee7a608c9edb5621a4b [term 0 FOLLOWER]: Advancing to term 1
I20260923 22:16:40.517162 23933 raft_consensus.cc:2454] T 00000000000000000000000000000000 P 33bbb8df312f4ee7a608c9edb5621a4b [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a663d0fc71f24f16811ad260612eced4 in term 1.
I20260923 22:16:40.515038 23891 master_runner.cc:186] Error getting master registration for 127.1.135.253:36357: OK, instance_id { permanent_uuid: "5a3ce1613d3b498d884fe571f308e48a" instance_seqno: 1790201800306106 } error { code: CATALOG_MANAGER_NOT_INITIALIZED status { code: SERVICE_UNAVAILABLE message: "Catalog manager is not initialized. State: Starting" } }
I20260923 22:16:40.517349 23760 leader_election.cc:302] T 00000000000000000000000000000000 P a663d0fc71f24f16811ad260612eced4 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 33bbb8df312f4ee7a608c9edb5621a4b, a663d0fc71f24f16811ad260612eced4; no voters:
I20260923 22:16:40.517426 23891 master_runner.cc:418] Couldn't verify the masters in the cluster. Trying again...
I20260923 22:16:40.517591 23966 raft_consensus.cc:2781] T 00000000000000000000000000000000 P a663d0fc71f24f16811ad260612eced4 [term 1 FOLLOWER]: Leader election won for term 1
I20260923 22:16:40.517745 23966 raft_consensus.cc:686] T 00000000000000000000000000000000 P a663d0fc71f24f16811ad260612eced4 [term 1 LEADER]: Becoming Leader. State: Replica: a663d0fc71f24f16811ad260612eced4, State: Running, Role: LEADER
I20260923 22:16:40.517833 23966 consensus_queue.cc:237] T 00000000000000000000000000000000 P a663d0fc71f24f16811ad260612eced4 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a663d0fc71f24f16811ad260612eced4" member_type: VOTER last_known_addr { host: "127.1.135.254" port: 43661 } } peers { permanent_uuid: "5a3ce1613d3b498d884fe571f308e48a" member_type: VOTER last_known_addr { host: "127.1.135.253" port: 36357 } } peers { permanent_uuid: "33bbb8df312f4ee7a608c9edb5621a4b" member_type: VOTER last_known_addr { host: "127.1.135.252" port: 43047 } }
I20260923 22:16:40.518286 23993 sys_catalog.cc:455] T 00000000000000000000000000000000 P a663d0fc71f24f16811ad260612eced4 [sys.catalog]: SysCatalogTable state changed. Reason: New leader a663d0fc71f24f16811ad260612eced4. Latest consensus state: current_term: 1 leader_uuid: "a663d0fc71f24f16811ad260612eced4" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a663d0fc71f24f16811ad260612eced4" member_type: VOTER last_known_addr { host: "127.1.135.254" port: 43661 } } peers { permanent_uuid: "5a3ce1613d3b498d884fe571f308e48a" member_type: VOTER last_known_addr { host: "127.1.135.253" port: 36357 } } peers { permanent_uuid: "33bbb8df312f4ee7a608c9edb5621a4b" member_type: VOTER last_known_addr { host: "127.1.135.252" port: 43047 } } }
I20260923 22:16:40.518491 23993 sys_catalog.cc:458] T 00000000000000000000000000000000 P a663d0fc71f24f16811ad260612eced4 [sys.catalog]: This master's current role is: LEADER
I20260923 22:16:40.525183 23999 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260923 22:16:40.525370 23999 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260923 22:16:40.527262 23933 raft_consensus.cc:1261] T 00000000000000000000000000000000 P 33bbb8df312f4ee7a608c9edb5621a4b [term 1 FOLLOWER]: Refusing update from remote peer a663d0fc71f24f16811ad260612eced4: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260923 22:16:40.527701 23993 consensus_queue.cc:1035] T 00000000000000000000000000000000 P a663d0fc71f24f16811ad260612eced4 [LEADER]: Connected to new peer: Peer: permanent_uuid: "33bbb8df312f4ee7a608c9edb5621a4b" member_type: VOTER last_known_addr { host: "127.1.135.252" port: 43047 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260923 22:16:40.529011 23980 sys_catalog.cc:455] T 00000000000000000000000000000000 P 33bbb8df312f4ee7a608c9edb5621a4b [sys.catalog]: SysCatalogTable state changed. Reason: New leader a663d0fc71f24f16811ad260612eced4. Latest consensus state: current_term: 1 leader_uuid: "a663d0fc71f24f16811ad260612eced4" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a663d0fc71f24f16811ad260612eced4" member_type: VOTER last_known_addr { host: "127.1.135.254" port: 43661 } } peers { permanent_uuid: "5a3ce1613d3b498d884fe571f308e48a" member_type: VOTER last_known_addr { host: "127.1.135.253" port: 36357 } } peers { permanent_uuid: "33bbb8df312f4ee7a608c9edb5621a4b" member_type: VOTER last_known_addr { host: "127.1.135.252" port: 43047 } } }
I20260923 22:16:40.529117 23980 sys_catalog.cc:458] T 00000000000000000000000000000000 P 33bbb8df312f4ee7a608c9edb5621a4b [sys.catalog]: This master's current role is: FOLLOWER
I20260923 22:16:40.530305 23993 sys_catalog.cc:455] T 00000000000000000000000000000000 P a663d0fc71f24f16811ad260612eced4 [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "a663d0fc71f24f16811ad260612eced4" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a663d0fc71f24f16811ad260612eced4" member_type: VOTER last_known_addr { host: "127.1.135.254" port: 43661 } } peers { permanent_uuid: "5a3ce1613d3b498d884fe571f308e48a" member_type: VOTER last_known_addr { host: "127.1.135.253" port: 36357 } } peers { permanent_uuid: "33bbb8df312f4ee7a608c9edb5621a4b" member_type: VOTER last_known_addr { host: "127.1.135.252" port: 43047 } } }
I20260923 22:16:40.530387 23993 sys_catalog.cc:458] T 00000000000000000000000000000000 P a663d0fc71f24f16811ad260612eced4 [sys.catalog]: This master's current role is: LEADER
I20260923 22:16:40.530750 23980 sys_catalog.cc:455] T 00000000000000000000000000000000 P 33bbb8df312f4ee7a608c9edb5621a4b [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "a663d0fc71f24f16811ad260612eced4" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a663d0fc71f24f16811ad260612eced4" member_type: VOTER last_known_addr { host: "127.1.135.254" port: 43661 } } peers { permanent_uuid: "5a3ce1613d3b498d884fe571f308e48a" member_type: VOTER last_known_addr { host: "127.1.135.253" port: 36357 } } peers { permanent_uuid: "33bbb8df312f4ee7a608c9edb5621a4b" member_type: VOTER last_known_addr { host: "127.1.135.252" port: 43047 } } }
I20260923 22:16:40.530826 23980 sys_catalog.cc:458] T 00000000000000000000000000000000 P 33bbb8df312f4ee7a608c9edb5621a4b [sys.catalog]: This master's current role is: FOLLOWER
I20260923 22:16:40.531253 24004 mvcc.cc:205] Tried to move back new op lower bound from 7332666574956179456 to 7332666574921691136. Current Snapshot: MvccSnapshot[applied={T|T < 7332666574956179456}]
I20260923 22:16:40.531252 23999 catalog_manager.cc:1347] Generated new cluster ID: 62d11930b5b744f4bc79ab750df14e23
I20260923 22:16:40.531473 23999 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260923 22:16:40.536574 23888 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 22:16:40.518473 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33374 (local address 127.1.135.253:36357)
0923 22:16:40.518510 (+ 37us) server_negotiation.cc:206] Beginning negotiation
0923 22:16:40.518512 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0923 22:16:40.519273 (+ 761us) server_negotiation.cc:407] Connection header received
0923 22:16:40.519509 (+ 236us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 22:16:40.519510 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 22:16:40.519520 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 22:16:40.519549 (+ 29us) server_negotiation.cc:226] Negotiated authn=SASL
0923 22:16:40.521128 (+ 1579us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 22:16:40.521852 (+ 724us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 22:16:40.523800 (+ 1948us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 22:16:40.524560 (+ 760us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 22:16:40.524605 (+ 45us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 22:16:40.531389 (+ 6784us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0923 22:16:40.531391 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0923 22:16:40.531392 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0923 22:16:40.531399 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0923 22:16:40.534478 (+ 3079us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0923 22:16:40.535274 (+ 796us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0923 22:16:40.535275 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0923 22:16:40.535275 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0923 22:16:40.535346 (+ 71us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0923 22:16:40.535686 (+ 340us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0923 22:16:40.535687 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0923 22:16:40.535687 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0923 22:16:40.535817 (+ 130us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0923 22:16:40.535890 (+ 73us) server_negotiation.cc:1035] Waiting for connection context
0923 22:16:40.536367 (+ 477us) server_negotiation.cc:299] Negotiation successful
0923 22:16:40.536457 (+ 90us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":19}
I20260923 22:16:40.537540 23812 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 22:16:40.518651 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45118 (local address 127.1.135.254:43661)
0923 22:16:40.518694 (+ 43us) server_negotiation.cc:206] Beginning negotiation
0923 22:16:40.518696 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0923 22:16:40.519412 (+ 716us) server_negotiation.cc:407] Connection header received
0923 22:16:40.519480 (+ 68us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 22:16:40.519481 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 22:16:40.519503 (+ 22us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 22:16:40.519562 (+ 59us) server_negotiation.cc:226] Negotiated authn=SASL
0923 22:16:40.521105 (+ 1543us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 22:16:40.521822 (+ 717us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 22:16:40.523764 (+ 1942us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 22:16:40.524089 (+ 325us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 22:16:40.524161 (+ 72us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 22:16:40.531829 (+ 7668us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0923 22:16:40.531831 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0923 22:16:40.531831 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0923 22:16:40.531838 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0923 22:16:40.533668 (+ 1830us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0923 22:16:40.536819 (+ 3151us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0923 22:16:40.536820 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0923 22:16:40.536821 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0923 22:16:40.536877 (+ 56us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0923 22:16:40.537049 (+ 172us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0923 22:16:40.537049 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0923 22:16:40.537050 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0923 22:16:40.537162 (+ 112us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0923 22:16:40.537229 (+ 67us) server_negotiation.cc:1035] Waiting for connection context
0923 22:16:40.537388 (+ 159us) server_negotiation.cc:299] Negotiation successful
0923 22:16:40.537458 (+ 70us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":21}
I20260923 22:16:40.539052 23964 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 22:16:40.518960 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45892 (local address 127.1.135.252:43047)
0923 22:16:40.519001 (+ 41us) server_negotiation.cc:206] Beginning negotiation
0923 22:16:40.519003 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0923 22:16:40.520386 (+ 1383us) server_negotiation.cc:407] Connection header received
0923 22:16:40.520452 (+ 66us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 22:16:40.520453 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 22:16:40.520463 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 22:16:40.520518 (+ 55us) server_negotiation.cc:226] Negotiated authn=SASL
0923 22:16:40.524759 (+ 4241us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 22:16:40.525458 (+ 699us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 22:16:40.526410 (+ 952us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 22:16:40.526702 (+ 292us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 22:16:40.526748 (+ 46us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 22:16:40.534996 (+ 8248us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0923 22:16:40.534998 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0923 22:16:40.534998 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0923 22:16:40.535004 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0923 22:16:40.536496 (+ 1492us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0923 22:16:40.538346 (+ 1850us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0923 22:16:40.538347 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0923 22:16:40.538347 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0923 22:16:40.538395 (+ 48us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0923 22:16:40.538551 (+ 156us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0923 22:16:40.538551 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0923 22:16:40.538552 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0923 22:16:40.538670 (+ 118us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0923 22:16:40.538736 (+ 66us) server_negotiation.cc:1035] Waiting for connection context
0923 22:16:40.538898 (+ 162us) server_negotiation.cc:299] Negotiation successful
0923 22:16:40.538969 (+ 71us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":20}
I20260923 22:16:40.540989 23884 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.1.135.252" port: 43047 } attempt: 3
Sep 23 22:16:40 dist-test-slave-s2fj krb5kdc[23728](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790201800, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.135.253@KRBTEST.COM for kudu/127.1.135.252@KRBTEST.COM
I20260923 22:16:40.548344 23886 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 22:16:40.541236 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.135.252:43047 (local address 127.0.0.1:45908)
0923 22:16:40.541276 (+ 40us) negotiation.cc:107] Waiting for socket to connect
0923 22:16:40.541285 (+ 9us) client_negotiation.cc:174] Beginning negotiation
0923 22:16:40.541315 (+ 30us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0923 22:16:40.541695 (+ 380us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0923 22:16:40.541696 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0923 22:16:40.541841 (+ 145us) client_negotiation.cc:189] Negotiated authn=SASL
0923 22:16:40.541878 (+ 37us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 22:16:40.541879 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 22:16:40.542788 (+ 909us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 22:16:40.542789 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 22:16:40.543534 (+ 745us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 22:16:40.543535 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 22:16:40.543977 (+ 442us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 22:16:40.543977 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 22:16:40.544023 (+ 46us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 22:16:40.544421 (+ 398us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0923 22:16:40.544426 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0923 22:16:40.545739 (+ 1313us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0923 22:16:40.547380 (+ 1641us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0923 22:16:40.547381 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0923 22:16:40.547382 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0923 22:16:40.547560 (+ 178us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0923 22:16:40.547738 (+ 178us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0923 22:16:40.547739 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0923 22:16:40.547739 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0923 22:16:40.547784 (+ 45us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0923 22:16:40.547996 (+ 212us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0923 22:16:40.547997 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0923 22:16:40.548034 (+ 37us) client_negotiation.cc:769] Sending connection context
0923 22:16:40.548139 (+ 105us) client_negotiation.cc:240] Negotiation successful
0923 22:16:40.548224 (+ 85us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":25}
I20260923 22:16:40.549579 23964 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 22:16:40.541279 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45908 (local address 127.1.135.252:43047)
0923 22:16:40.541589 (+ 310us) server_negotiation.cc:206] Beginning negotiation
0923 22:16:40.541590 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0923 22:16:40.541597 (+ 7us) server_negotiation.cc:407] Connection header received
0923 22:16:40.541612 (+ 15us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 22:16:40.541613 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 22:16:40.541621 (+ 8us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 22:16:40.541659 (+ 38us) server_negotiation.cc:226] Negotiated authn=SASL
0923 22:16:40.542007 (+ 348us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 22:16:40.542710 (+ 703us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 22:16:40.543618 (+ 908us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 22:16:40.543889 (+ 271us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 22:16:40.543937 (+ 48us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 22:16:40.545821 (+ 1884us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0923 22:16:40.545822 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0923 22:16:40.545822 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0923 22:16:40.545828 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0923 22:16:40.547294 (+ 1466us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0923 22:16:40.547639 (+ 345us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0923 22:16:40.547640 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0923 22:16:40.547640 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0923 22:16:40.547684 (+ 44us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0923 22:16:40.547835 (+ 151us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0923 22:16:40.547835 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0923 22:16:40.547835 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0923 22:16:40.547933 (+ 98us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0923 22:16:40.547978 (+ 45us) server_negotiation.cc:1035] Waiting for connection context
0923 22:16:40.549367 (+ 1389us) server_negotiation.cc:299] Negotiation successful
0923 22:16:40.549447 (+ 80us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":281}
I20260923 22:16:40.552644 23884 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 5a3ce1613d3b498d884fe571f308e48a: Bootstrap starting.
I20260923 22:16:40.553435 23884 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 5a3ce1613d3b498d884fe571f308e48a: Neither blocks nor log segments found. Creating new log.
I20260923 22:16:40.553788 23884 log.cc:824] T 00000000000000000000000000000000 P 5a3ce1613d3b498d884fe571f308e48a: Log is configured to *not* fsync() on all Append() calls
I20260923 22:16:40.554876 23884 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 5a3ce1613d3b498d884fe571f308e48a: No bootstrap required, opened a new log
I20260923 22:16:40.556841 23884 raft_consensus.cc:348] T 00000000000000000000000000000000 P 5a3ce1613d3b498d884fe571f308e48a [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a663d0fc71f24f16811ad260612eced4" member_type: VOTER last_known_addr { host: "127.1.135.254" port: 43661 } } peers { permanent_uuid: "5a3ce1613d3b498d884fe571f308e48a" member_type: VOTER last_known_addr { host: "127.1.135.253" port: 36357 } } peers { permanent_uuid: "33bbb8df312f4ee7a608c9edb5621a4b" member_type: VOTER last_known_addr { host: "127.1.135.252" port: 43047 } }
I20260923 22:16:40.556970 23884 raft_consensus.cc:729] T 00000000000000000000000000000000 P 5a3ce1613d3b498d884fe571f308e48a [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5a3ce1613d3b498d884fe571f308e48a, State: Initialized, Role: FOLLOWER
I20260923 22:16:40.557035 23884 consensus_queue.cc:260] T 00000000000000000000000000000000 P 5a3ce1613d3b498d884fe571f308e48a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a663d0fc71f24f16811ad260612eced4" member_type: VOTER last_known_addr { host: "127.1.135.254" port: 43661 } } peers { permanent_uuid: "5a3ce1613d3b498d884fe571f308e48a" member_type: VOTER last_known_addr { host: "127.1.135.253" port: 36357 } } peers { permanent_uuid: "33bbb8df312f4ee7a608c9edb5621a4b" member_type: VOTER last_known_addr { host: "127.1.135.252" port: 43047 } }
I20260923 22:16:40.557296 23884 sys_catalog.cc:564] T 00000000000000000000000000000000 P 5a3ce1613d3b498d884fe571f308e48a [sys.catalog]: configured and running, proceeding with master startup.
I20260923 22:16:40.557354 23858 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "a663d0fc71f24f16811ad260612eced4" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: true dest_uuid: "5a3ce1613d3b498d884fe571f308e48a"
I20260923 22:16:40.557508 23858 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 5a3ce1613d3b498d884fe571f308e48a [term 0 FOLLOWER]: Advancing to term 1
I20260923 22:16:40.558321 23858 raft_consensus.cc:2454] T 00000000000000000000000000000000 P 5a3ce1613d3b498d884fe571f308e48a [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a663d0fc71f24f16811ad260612eced4 in term 1.
I20260923 22:16:40.558652 23857 raft_consensus.cc:1261] T 00000000000000000000000000000000 P 5a3ce1613d3b498d884fe571f308e48a [term 1 FOLLOWER]: Refusing update from remote peer a663d0fc71f24f16811ad260612eced4: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260923 22:16:40.559046 23966 consensus_queue.cc:1035] T 00000000000000000000000000000000 P a663d0fc71f24f16811ad260612eced4 [LEADER]: Connected to new peer: Peer: permanent_uuid: "5a3ce1613d3b498d884fe571f308e48a" member_type: VOTER last_known_addr { host: "127.1.135.253" port: 36357 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260923 22:16:40.560422 24024 sys_catalog.cc:455] T 00000000000000000000000000000000 P 5a3ce1613d3b498d884fe571f308e48a [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "a663d0fc71f24f16811ad260612eced4" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a663d0fc71f24f16811ad260612eced4" member_type: VOTER last_known_addr { host: "127.1.135.254" port: 43661 } } peers { permanent_uuid: "5a3ce1613d3b498d884fe571f308e48a" member_type: VOTER last_known_addr { host: "127.1.135.253" port: 36357 } } peers { permanent_uuid: "33bbb8df312f4ee7a608c9edb5621a4b" member_type: VOTER last_known_addr { host: "127.1.135.252" port: 43047 } } }
I20260923 22:16:40.560557 24024 sys_catalog.cc:458] T 00000000000000000000000000000000 P 5a3ce1613d3b498d884fe571f308e48a [sys.catalog]: This master's current role is: FOLLOWER
I20260923 22:16:40.560639 24024 sys_catalog.cc:455] T 00000000000000000000000000000000 P 5a3ce1613d3b498d884fe571f308e48a [sys.catalog]: SysCatalogTable state changed. Reason: New leader a663d0fc71f24f16811ad260612eced4. Latest consensus state: current_term: 1 leader_uuid: "a663d0fc71f24f16811ad260612eced4" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a663d0fc71f24f16811ad260612eced4" member_type: VOTER last_known_addr { host: "127.1.135.254" port: 43661 } } peers { permanent_uuid: "5a3ce1613d3b498d884fe571f308e48a" member_type: VOTER last_known_addr { host: "127.1.135.253" port: 36357 } } peers { permanent_uuid: "33bbb8df312f4ee7a608c9edb5621a4b" member_type: VOTER last_known_addr { host: "127.1.135.252" port: 43047 } } }
I20260923 22:16:40.560725 24024 sys_catalog.cc:458] T 00000000000000000000000000000000 P 5a3ce1613d3b498d884fe571f308e48a [sys.catalog]: This master's current role is: FOLLOWER
I20260923 22:16:40.560806 24024 sys_catalog.cc:455] T 00000000000000000000000000000000 P 5a3ce1613d3b498d884fe571f308e48a [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "a663d0fc71f24f16811ad260612eced4" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a663d0fc71f24f16811ad260612eced4" member_type: VOTER last_known_addr { host: "127.1.135.254" port: 43661 } } peers { permanent_uuid: "5a3ce1613d3b498d884fe571f308e48a" member_type: VOTER last_known_addr { host: "127.1.135.253" port: 36357 } } peers { permanent_uuid: "33bbb8df312f4ee7a608c9edb5621a4b" member_type: VOTER last_known_addr { host: "127.1.135.252" port: 43047 } } }
I20260923 22:16:40.560889 24024 sys_catalog.cc:458] T 00000000000000000000000000000000 P 5a3ce1613d3b498d884fe571f308e48a [sys.catalog]: This master's current role is: FOLLOWER
I20260923 22:16:40.562114 24022 mvcc.cc:205] Tried to move back new op lower bound from 7332666574956179456 to 7332666574921691136. Current Snapshot: MvccSnapshot[applied={T|T < 7332666574956179456}]
I20260923 22:16:40.562907 23966 sys_catalog.cc:455] T 00000000000000000000000000000000 P a663d0fc71f24f16811ad260612eced4 [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "a663d0fc71f24f16811ad260612eced4" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a663d0fc71f24f16811ad260612eced4" member_type: VOTER last_known_addr { host: "127.1.135.254" port: 43661 } } peers { permanent_uuid: "5a3ce1613d3b498d884fe571f308e48a" member_type: VOTER last_known_addr { host: "127.1.135.253" port: 36357 } } peers { permanent_uuid: "33bbb8df312f4ee7a608c9edb5621a4b" member_type: VOTER last_known_addr { host: "127.1.135.252" port: 43047 } } }
I20260923 22:16:40.562994 23966 sys_catalog.cc:458] T 00000000000000000000000000000000 P a663d0fc71f24f16811ad260612eced4 [sys.catalog]: This master's current role is: LEADER
I20260923 22:16:40.563822 23999 catalog_manager.cc:1370] Generated new certificate authority record
I20260923 22:16:40.564662 23999 catalog_manager.cc:1504] Loading token signing keys...
I20260923 22:16:40.573551 24020 catalog_manager.cc:1259] Loaded cluster ID: 62d11930b5b744f4bc79ab750df14e23
I20260923 22:16:40.573621 24020 catalog_manager.cc:1552] T 00000000000000000000000000000000 P 5a3ce1613d3b498d884fe571f308e48a: loading cluster ID for follower catalog manager: success
W20260923 22:16:40.573683 24020 catalog_manager.cc:1576] T 00000000000000000000000000000000 P 5a3ce1613d3b498d884fe571f308e48a: acquiring CA information for follower catalog manager: Not found: root CA entry not found
W20260923 22:16:40.573715 24020 catalog_manager.cc:873] Not found: root CA entry not found: failed to prepare follower catalog manager, will retry
I20260923 22:16:40.578688 23999 catalog_manager.cc:5868] T 00000000000000000000000000000000 P a663d0fc71f24f16811ad260612eced4: Generated new TSK 0
I20260923 22:16:40.578897 23999 catalog_manager.cc:1514] Initializing in-progress tserver states...
W20260923 22:16:40.607297 23964 negotiation.cc:337] Failed RPC negotiation. Trace:
0923 22:16:40.605762 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45914 (local address 127.1.135.252:43047)
0923 22:16:40.605805 (+ 43us) server_negotiation.cc:206] Beginning negotiation
0923 22:16:40.605807 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0923 22:16:40.606674 (+ 867us) server_negotiation.cc:407] Connection header received
0923 22:16:40.606772 (+ 98us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 22:16:40.606773 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 22:16:40.606784 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 22:16:40.606813 (+ 29us) server_negotiation.cc:226] Negotiated authn=SASL
0923 22:16:40.607234 (+ 421us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:45914: BlockingRecv error: recv got EOF from 127.0.0.1:45914 (error 108)
Metrics: {"server-negotiator.queue_time_us":21}
W20260923 22:16:40.607996 23812 negotiation.cc:337] Failed RPC negotiation. Trace:
0923 22:16:40.606128 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45124 (local address 127.1.135.254:43661)
0923 22:16:40.606170 (+ 42us) server_negotiation.cc:206] Beginning negotiation
0923 22:16:40.606171 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0923 22:16:40.607533 (+ 1362us) server_negotiation.cc:407] Connection header received
0923 22:16:40.607615 (+ 82us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 22:16:40.607616 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 22:16:40.607625 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 22:16:40.607653 (+ 28us) server_negotiation.cc:226] Negotiated authn=SASL
0923 22:16:40.607929 (+ 276us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:45124: BlockingRecv error: recv got EOF from 127.0.0.1:45124 (error 108)
Metrics: {"server-negotiator.queue_time_us":24}
W20260923 22:16:40.608682 23888 negotiation.cc:337] Failed RPC negotiation. Trace:
0923 22:16:40.606440 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33390 (local address 127.1.135.253:36357)
0923 22:16:40.606490 (+ 50us) server_negotiation.cc:206] Beginning negotiation
0923 22:16:40.606491 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0923 22:16:40.608224 (+ 1733us) server_negotiation.cc:407] Connection header received
0923 22:16:40.608314 (+ 90us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 22:16:40.608315 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 22:16:40.608326 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 22:16:40.608354 (+ 28us) server_negotiation.cc:226] Negotiated authn=SASL
0923 22:16:40.608628 (+ 274us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:33390: BlockingRecv error: recv got EOF from 127.0.0.1:33390 (error 108)
Metrics: {"server-negotiator.queue_time_us":20}
W20260923 22:16:40.610477 23888 negotiation.cc:337] Failed RPC negotiation. Trace:
0923 22:16:40.609801 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33396 (local address 127.1.135.253:36357)
0923 22:16:40.609832 (+ 31us) server_negotiation.cc:206] Beginning negotiation
0923 22:16:40.609834 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0923 22:16:40.610026 (+ 192us) server_negotiation.cc:407] Connection header received
0923 22:16:40.610125 (+ 99us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 22:16:40.610126 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 22:16:40.610141 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 22:16:40.610169 (+ 28us) server_negotiation.cc:226] Negotiated authn=SASL
0923 22:16:40.610438 (+ 269us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:33396: BlockingRecv error: recv got EOF from 127.0.0.1:33396 (error 108)
Metrics: {"server-negotiator.queue_time_us":15}
W20260923 22:16:40.617126 23888 negotiation.cc:337] Failed RPC negotiation. Trace:
0923 22:16:40.616464 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33402 (local address 127.1.135.253:36357)
0923 22:16:40.616495 (+ 31us) server_negotiation.cc:206] Beginning negotiation
0923 22:16:40.616497 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0923 22:16:40.616662 (+ 165us) server_negotiation.cc:407] Connection header received
0923 22:16:40.616750 (+ 88us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 22:16:40.616751 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 22:16:40.616760 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 22:16:40.616787 (+ 27us) server_negotiation.cc:226] Negotiated authn=SASL
0923 22:16:40.617086 (+ 299us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:33402: BlockingRecv error: recv got EOF from 127.0.0.1:33402 (error 108)
Metrics: {"server-negotiator.queue_time_us":15}
W20260923 22:16:40.620843 23888 negotiation.cc:337] Failed RPC negotiation. Trace:
0923 22:16:40.620169 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33414 (local address 127.1.135.253:36357)
0923 22:16:40.620201 (+ 32us) server_negotiation.cc:206] Beginning negotiation
0923 22:16:40.620203 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0923 22:16:40.620408 (+ 205us) server_negotiation.cc:407] Connection header received
0923 22:16:40.620496 (+ 88us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 22:16:40.620497 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 22:16:40.620505 (+ 8us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 22:16:40.620533 (+ 28us) server_negotiation.cc:226] Negotiated authn=SASL
0923 22:16:40.620805 (+ 272us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:33414: BlockingRecv error: recv got EOF from 127.0.0.1:33414 (error 108)
Metrics: {"server-negotiator.queue_time_us":17}
W20260923 22:16:40.626643 23888 negotiation.cc:337] Failed RPC negotiation. Trace:
0923 22:16:40.625865 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33426 (local address 127.1.135.253:36357)
0923 22:16:40.625895 (+ 30us) server_negotiation.cc:206] Beginning negotiation
0923 22:16:40.625897 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0923 22:16:40.626138 (+ 241us) server_negotiation.cc:407] Connection header received
0923 22:16:40.626224 (+ 86us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 22:16:40.626224 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 22:16:40.626233 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 22:16:40.626259 (+ 26us) server_negotiation.cc:226] Negotiated authn=SASL
0923 22:16:40.626602 (+ 343us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:33426: BlockingRecv error: recv got EOF from 127.0.0.1:33426 (error 108)
Metrics: {"server-negotiator.queue_time_us":16}
W20260923 22:16:40.636488 23888 negotiation.cc:337] Failed RPC negotiation. Trace:
0923 22:16:40.635710 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33438 (local address 127.1.135.253:36357)
0923 22:16:40.635741 (+ 31us) server_negotiation.cc:206] Beginning negotiation
0923 22:16:40.635743 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0923 22:16:40.635949 (+ 206us) server_negotiation.cc:407] Connection header received
0923 22:16:40.636054 (+ 105us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 22:16:40.636055 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 22:16:40.636103 (+ 48us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 22:16:40.636132 (+ 29us) server_negotiation.cc:226] Negotiated authn=SASL
0923 22:16:40.636448 (+ 316us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:33438: BlockingRecv error: recv got EOF from 127.0.0.1:33438 (error 108)
Metrics: {"server-negotiator.queue_time_us":16}
W20260923 22:16:40.654548 23888 negotiation.cc:337] Failed RPC negotiation. Trace:
0923 22:16:40.653675 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33444 (local address 127.1.135.253:36357)
0923 22:16:40.653711 (+ 36us) server_negotiation.cc:206] Beginning negotiation
0923 22:16:40.653713 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0923 22:16:40.653939 (+ 226us) server_negotiation.cc:407] Connection header received
0923 22:16:40.654097 (+ 158us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 22:16:40.654098 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 22:16:40.654108 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 22:16:40.654139 (+ 31us) server_negotiation.cc:226] Negotiated authn=SASL
0923 22:16:40.654500 (+ 361us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:33444: BlockingRecv error: recv got EOF from 127.0.0.1:33444 (error 108)
Metrics: {"server-negotiator.queue_time_us":17}
W20260923 22:16:40.688935 23888 negotiation.cc:337] Failed RPC negotiation. Trace:
0923 22:16:40.687926 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33460 (local address 127.1.135.253:36357)
0923 22:16:40.687971 (+ 45us) server_negotiation.cc:206] Beginning negotiation
0923 22:16:40.688015 (+ 44us) server_negotiation.cc:399] Waiting for connection header
0923 22:16:40.688332 (+ 317us) server_negotiation.cc:407] Connection header received
0923 22:16:40.688442 (+ 110us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 22:16:40.688444 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 22:16:40.688455 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 22:16:40.688486 (+ 31us) server_negotiation.cc:226] Negotiated authn=SASL
0923 22:16:40.688878 (+ 392us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:33460: BlockingRecv error: recv got EOF from 127.0.0.1:33460 (error 108)
Metrics: {"server-negotiator.queue_time_us":21}
W20260923 22:16:40.754948 23888 negotiation.cc:337] Failed RPC negotiation. Trace:
0923 22:16:40.754264 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33470 (local address 127.1.135.253:36357)
0923 22:16:40.754319 (+ 55us) server_negotiation.cc:206] Beginning negotiation
0923 22:16:40.754321 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0923 22:16:40.754414 (+ 93us) server_negotiation.cc:407] Connection header received
0923 22:16:40.754488 (+ 74us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 22:16:40.754489 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 22:16:40.754499 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 22:16:40.754539 (+ 40us) server_negotiation.cc:226] Negotiated authn=SASL
0923 22:16:40.754884 (+ 345us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:33470: BlockingRecv error: recv got EOF from 127.0.0.1:33470 (error 108)
Metrics: {"server-negotiator.queue_time_us":27}
W20260923 22:16:40.884891 23888 negotiation.cc:337] Failed RPC negotiation. Trace:
0923 22:16:40.884005 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33486 (local address 127.1.135.253:36357)
0923 22:16:40.884194 (+ 189us) server_negotiation.cc:206] Beginning negotiation
0923 22:16:40.884197 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0923 22:16:40.884304 (+ 107us) server_negotiation.cc:407] Connection header received
0923 22:16:40.884424 (+ 120us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 22:16:40.884425 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 22:16:40.884440 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 22:16:40.884498 (+ 58us) server_negotiation.cc:226] Negotiated authn=SASL
0923 22:16:40.884843 (+ 345us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:33486: BlockingRecv error: recv got EOF from 127.0.0.1:33486 (error 108)
Metrics: {"server-negotiator.queue_time_us":152}
W20260923 22:16:41.142845 23888 negotiation.cc:337] Failed RPC negotiation. Trace:
0923 22:16:41.142186 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33498 (local address 127.1.135.253:36357)
0923 22:16:41.142255 (+ 69us) server_negotiation.cc:206] Beginning negotiation
0923 22:16:41.142257 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0923 22:16:41.142271 (+ 14us) server_negotiation.cc:407] Connection header received
0923 22:16:41.142364 (+ 93us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 22:16:41.142365 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 22:16:41.142377 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 22:16:41.142456 (+ 79us) server_negotiation.cc:226] Negotiated authn=SASL
0923 22:16:41.142784 (+ 328us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:33498: BlockingRecv error: recv got EOF from 127.0.0.1:33498 (error 108)
Metrics: {"server-negotiator.queue_time_us":44}
I20260923 22:16:41.514397 23991 catalog_manager.cc:1259] Loaded cluster ID: 62d11930b5b744f4bc79ab750df14e23
I20260923 22:16:41.514485 23991 catalog_manager.cc:1552] T 00000000000000000000000000000000 P 33bbb8df312f4ee7a608c9edb5621a4b: loading cluster ID for follower catalog manager: success
I20260923 22:16:41.515285 23991 catalog_manager.cc:1574] T 00000000000000000000000000000000 P 33bbb8df312f4ee7a608c9edb5621a4b: acquiring CA information for follower catalog manager: success
I20260923 22:16:41.515599 23991 catalog_manager.cc:1602] T 00000000000000000000000000000000 P 33bbb8df312f4ee7a608c9edb5621a4b: importing token verification keys for follower catalog manager: success; most recent TSK sequence number 0
I20260923 22:16:41.574836 24020 catalog_manager.cc:1574] T 00000000000000000000000000000000 P 5a3ce1613d3b498d884fe571f308e48a: acquiring CA information for follower catalog manager: success
I20260923 22:16:41.575263 24020 catalog_manager.cc:1602] T 00000000000000000000000000000000 P 5a3ce1613d3b498d884fe571f308e48a: importing token verification keys for follower catalog manager: success; most recent TSK sequence number 0
I20260923 22:16:41.658860 24094 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 22:16:41.656174 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33508 (local address 127.1.135.253:36357)
0923 22:16:41.656297 (+ 123us) server_negotiation.cc:206] Beginning negotiation
0923 22:16:41.656298 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0923 22:16:41.656313 (+ 15us) server_negotiation.cc:407] Connection header received
0923 22:16:41.656388 (+ 75us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 22:16:41.656389 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 22:16:41.656437 (+ 48us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 22:16:41.656522 (+ 85us) server_negotiation.cc:226] Negotiated authn=TOKEN
0923 22:16:41.656769 (+ 247us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 22:16:41.657269 (+ 500us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 22:16:41.658112 (+ 843us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 22:16:41.658315 (+ 203us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 22:16:41.658368 (+ 53us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 22:16:41.658493 (+ 125us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0923 22:16:41.658655 (+ 162us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0923 22:16:41.658734 (+ 79us) server_negotiation.cc:1035] Waiting for connection context
0923 22:16:41.658778 (+ 44us) server_negotiation.cc:299] Negotiation successful
0923 22:16:41.658797 (+ 19us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":93,"thread_start_us":72,"threads_started":1}
I20260923 22:16:41.662746 24100 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 22:16:41.660616 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45922 (local address 127.1.135.252:43047)
0923 22:16:41.660723 (+ 107us) server_negotiation.cc:206] Beginning negotiation
0923 22:16:41.660725 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0923 22:16:41.660735 (+ 10us) server_negotiation.cc:407] Connection header received
0923 22:16:41.660805 (+ 70us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 22:16:41.660805 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 22:16:41.660838 (+ 33us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 22:16:41.660950 (+ 112us) server_negotiation.cc:226] Negotiated authn=TOKEN
0923 22:16:41.661010 (+ 60us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 22:16:41.661477 (+ 467us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 22:16:41.662135 (+ 658us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 22:16:41.662319 (+ 184us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 22:16:41.662367 (+ 48us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 22:16:41.662489 (+ 122us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0923 22:16:41.662583 (+ 94us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0923 22:16:41.662635 (+ 52us) server_negotiation.cc:1035] Waiting for connection context
0923 22:16:41.662671 (+ 36us) server_negotiation.cc:299] Negotiation successful
0923 22:16:41.662688 (+ 17us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":88,"thread_start_us":65,"threads_started":1}
I20260923 22:16:41.667173 24094 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 22:16:41.664421 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33522 (local address 127.1.135.253:36357)
0923 22:16:41.664479 (+ 58us) server_negotiation.cc:206] Beginning negotiation
0923 22:16:41.664480 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0923 22:16:41.664529 (+ 49us) server_negotiation.cc:407] Connection header received
0923 22:16:41.664647 (+ 118us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 22:16:41.664648 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 22:16:41.664660 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 22:16:41.664689 (+ 29us) server_negotiation.cc:226] Negotiated authn=TOKEN
0923 22:16:41.664867 (+ 178us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 22:16:41.665379 (+ 512us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 22:16:41.666424 (+ 1045us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 22:16:41.666712 (+ 288us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 22:16:41.666770 (+ 58us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 22:16:41.666901 (+ 131us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0923 22:16:41.666929 (+ 28us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0923 22:16:41.666997 (+ 68us) server_negotiation.cc:1035] Waiting for connection context
0923 22:16:41.667064 (+ 67us) server_negotiation.cc:299] Negotiation successful
0923 22:16:41.667088 (+ 24us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":36}
I20260923 22:16:41.667563 24100 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 22:16:41.664501 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45930 (local address 127.1.135.252:43047)
0923 22:16:41.664566 (+ 65us) server_negotiation.cc:206] Beginning negotiation
0923 22:16:41.664568 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0923 22:16:41.664703 (+ 135us) server_negotiation.cc:407] Connection header received
0923 22:16:41.664770 (+ 67us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 22:16:41.664771 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 22:16:41.664781 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 22:16:41.664831 (+ 50us) server_negotiation.cc:226] Negotiated authn=TOKEN
0923 22:16:41.664917 (+ 86us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 22:16:41.665601 (+ 684us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 22:16:41.666577 (+ 976us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 22:16:41.666830 (+ 253us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 22:16:41.666896 (+ 66us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 22:16:41.667223 (+ 327us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0923 22:16:41.667257 (+ 34us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0923 22:16:41.667302 (+ 45us) server_negotiation.cc:1035] Waiting for connection context
0923 22:16:41.667460 (+ 158us) server_negotiation.cc:299] Negotiation successful
0923 22:16:41.667479 (+ 19us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":44}
I20260923 22:16:41.667791 24106 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 22:16:41.664309 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45138 (local address 127.1.135.254:43661)
0923 22:16:41.664643 (+ 334us) server_negotiation.cc:206] Beginning negotiation
0923 22:16:41.664645 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0923 22:16:41.664798 (+ 153us) server_negotiation.cc:407] Connection header received
0923 22:16:41.664839 (+ 41us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 22:16:41.664840 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 22:16:41.664885 (+ 45us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 22:16:41.664971 (+ 86us) server_negotiation.cc:226] Negotiated authn=TOKEN
0923 22:16:41.665041 (+ 70us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 22:16:41.665769 (+ 728us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 22:16:41.666698 (+ 929us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 22:16:41.667056 (+ 358us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 22:16:41.667119 (+ 63us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 22:16:41.667330 (+ 211us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0923 22:16:41.667424 (+ 94us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0923 22:16:41.667478 (+ 54us) server_negotiation.cc:1035] Waiting for connection context
0923 22:16:41.667680 (+ 202us) server_negotiation.cc:299] Negotiation successful
0923 22:16:41.667705 (+ 25us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":297,"thread_start_us":44,"threads_started":1}
I20260923 22:16:41.669480 1567 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskRiQe_a/build/release/bin/kudu with pid 23741
I20260923 22:16:41.675282 1567 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskRiQe_a/build/release/bin/kudu with pid 23816
I20260923 22:16:41.680102 1567 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskRiQe_a/build/release/bin/kudu with pid 23891
2026-09-23T22:16:41Z chronyd exiting
[ OK ] ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys/0 (2823 ms)
[ RUN ] ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys/1
Loading random data
Initializing database '/tmp/dist-test-taskRiQe_a/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790201539513534-1567-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Sep 23 22:16:41 dist-test-slave-s2fj krb5kdc[24111](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Sep 23 22:16:41 dist-test-slave-s2fj krb5kdc[24111](info): set up 2 sockets
Sep 23 22:16:41 dist-test-slave-s2fj krb5kdc[24111](info): commencing operation
krb5kdc: starting...
W20260923 22:16:42.688396 1567 mini_kdc.cc:121] Time spent starting KDC: real 0.989s user 0.000s sys 0.005s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskRiQe_a/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790201539513534-1567-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskRiQe_a/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790201539513534-1567-0/krb5kdc/test-user.keytab.
Sep 23 22:16:42 dist-test-slave-s2fj krb5kdc[24111](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790201802, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-09-23T22:16:42Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-09-23T22:16:42Z Disabled control of system clock
WARNING: no policy specified for kudu/127.1.135.254@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.1.135.254@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.1.135.254 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskRiQe_a/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790201539513534-1567-0/krb5kdc/kudu_127.1.135.254.keytab.
Entry for principal kudu/127.1.135.254 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskRiQe_a/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790201539513534-1567-0/krb5kdc/kudu_127.1.135.254.keytab.
I20260923 22:16:42.794514 1567 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskRiQe_a/build/release/bin/kudu
/tmp/dist-test-taskRiQe_a/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskRiQe_a/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790201539513534-1567-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskRiQe_a/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790201539513534-1567-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskRiQe_a/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790201539513534-1567-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskRiQe_a/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790201539513534-1567-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.1.135.254:37671
--webserver_interface=127.1.135.254
--webserver_port=0
--keytab_file=/tmp/dist-test-taskRiQe_a/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790201539513534-1567-0/krb5kdc/kudu_127.1.135.254.keytab
--principal=kudu/127.1.135.254
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.1.135.212:38547
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.1.135.254:37671,127.1.135.253:39749,127.1.135.252:33555
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--enable_leader_failure_detection=false
--rpc_authentication=required
--rpc_encryption=required with env {KRB5CCNAME=/tmp/dist-test-taskRiQe_a/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790201539513534-1567-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskRiQe_a/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790201539513534-1567-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskRiQe_a/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790201539513534-1567-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260923 22:16:42.875003 24124 flags.cc:425] Enabled unsafe flag: --enable_leader_failure_detection=false
W20260923 22:16:42.875237 24124 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260923 22:16:42.875255 24124 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260923 22:16:42.875275 24124 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260923 22:16:42.876375 24124 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260923 22:16:42.876411 24124 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260923 22:16:42.876425 24124 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260923 22:16:42.876437 24124 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260923 22:16:42.876449 24124 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260923 22:16:42.877882 24124 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.1.135.212:38547
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--enable_leader_failure_detection=false
--fs_data_dirs=/tmp/dist-test-taskRiQe_a/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790201539513534-1567-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskRiQe_a/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790201539513534-1567-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.1.135.254:37671,127.1.135.253:39749,127.1.135.252:33555
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskRiQe_a/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790201539513534-1567-0/krb5kdc/kudu_127.1.135.254.keytab
--principal=kudu/127.1.135.254
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.1.135.254:37671
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--rpc_encryption=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskRiQe_a/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790201539513534-1567-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.1.135.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.24124
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskRiQe_a/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790201539513534-1567-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 23 Sep 2026 20:53:37 UTC on ccycloud.quasar-rkviyx.root.comops.site
build id 4668083
I20260923 22:16:42.878078 24124 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260923 22:16:42.878258 24124 file_cache.cc:493] Constructed file cache file cache with capacity 419430
I20260923 22:16:42.883154 24124 server_base.cc:1034] running on GCE node
W20260923 22:16:42.883111 24132 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260923 22:16:42.883104 24130 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260923 22:16:42.883132 24129 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260923 22:16:42.883518 24124 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260923 22:16:42.883821 24124 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260923 22:16:42.884963 24124 hybrid_clock.cc:648] HybridClock initialized: now 1790201802884952 us; error 33 us; skew 500 ppm
Sep 23 22:16:42 dist-test-slave-s2fj krb5kdc[24111](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790201802, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.135.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260923 22:16:42.887604 24124 init.cc:377] Logged in from keytab as kudu/127.1.135.254@KRBTEST.COM (short username kudu)
I20260923 22:16:42.888042 24124 webserver.cc:460] Webserver started at http://127.1.135.254:40901/ using document root <none> and password file <none>
I20260923 22:16:42.888338 24124 fs_manager.cc:359] Metadata directory not provided
I20260923 22:16:42.888407 24124 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260923 22:16:42.888571 24124 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260923 22:16:42.889669 24124 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskRiQe_a/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790201539513534-1567-0/minicluster-data/master-0/data/instance:
uuid: "7951ee4f672c48e29ed3b1b74a092b78"
format_stamp: "Formatted at 2026-09-23 22:16:42 on dist-test-slave-s2fj"
server_key: "dec990e0358d60664f20236bbc323a02"
server_key_iv: "070abfbcf23edf0d33ae6b5a0462f60f"
server_key_version: "encryptionkey@0"
I20260923 22:16:42.890028 24124 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskRiQe_a/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790201539513534-1567-0/minicluster-data/master-0/wal/instance:
uuid: "7951ee4f672c48e29ed3b1b74a092b78"
format_stamp: "Formatted at 2026-09-23 22:16:42 on dist-test-slave-s2fj"
server_key: "dec990e0358d60664f20236bbc323a02"
server_key_iv: "070abfbcf23edf0d33ae6b5a0462f60f"
server_key_version: "encryptionkey@0"
I20260923 22:16:42.891338 24124 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.003s
I20260923 22:16:42.892056 24139 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260923 22:16:42.892293 24124 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.001s
I20260923 22:16:42.892397 24124 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskRiQe_a/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790201539513534-1567-0/minicluster-data/master-0/data,/tmp/dist-test-taskRiQe_a/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790201539513534-1567-0/minicluster-data/master-0/wal
uuid: "7951ee4f672c48e29ed3b1b74a092b78"
format_stamp: "Formatted at 2026-09-23 22:16:42 on dist-test-slave-s2fj"
server_key: "dec990e0358d60664f20236bbc323a02"
server_key_iv: "070abfbcf23edf0d33ae6b5a0462f60f"
server_key_version: "encryptionkey@0"
I20260923 22:16:42.892488 24124 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskRiQe_a/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790201539513534-1567-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskRiQe_a/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790201539513534-1567-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskRiQe_a/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790201539513534-1567-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260923 22:16:42.912988 24124 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260923 22:16:42.915525 24124 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260923 22:16:42.915750 24124 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260923 22:16:42.919767 24124 rpc_server.cc:307] RPC server started. Bound to: 127.1.135.254:37671
I20260923 22:16:42.919803 24191 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.1.135.254:37671 every 8 connection(s)
I20260923 22:16:42.920194 24124 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskRiQe_a/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790201539513534-1567-0/minicluster-data/master-0/data/info.pb
I20260923 22:16:42.921238 24192 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260923 22:16:42.922832 24192 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.1.135.254" port: 37671 } has no permanent_uuid. Determining permanent_uuid...
Sep 23 22:16:42 dist-test-slave-s2fj krb5kdc[24111](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790201802, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.135.254@KRBTEST.COM for kudu/127.1.135.254@KRBTEST.COM
I20260923 22:16:42.930188 1567 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskRiQe_a/build/release/bin/kudu as pid 24124
I20260923 22:16:42.930311 1567 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskRiQe_a/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790201539513534-1567-0/minicluster-data/master-0/wal/instance
I20260923 22:16:42.930505 1567 external_mini_cluster.cc:1428] Setting key f4e3baca1fa74a4c650a094196181028
I20260923 22:16:42.932457 24194 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 22:16:42.923499 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.135.254:37671 (local address 127.0.0.1:60000)
0923 22:16:42.923655 (+ 156us) negotiation.cc:107] Waiting for socket to connect
0923 22:16:42.923689 (+ 34us) client_negotiation.cc:174] Beginning negotiation
0923 22:16:42.924204 (+ 515us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0923 22:16:42.924463 (+ 259us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0923 22:16:42.924466 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0923 22:16:42.924731 (+ 265us) client_negotiation.cc:189] Negotiated authn=SASL
0923 22:16:42.924895 (+ 164us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 22:16:42.924899 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 22:16:42.925875 (+ 976us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 22:16:42.925876 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 22:16:42.926546 (+ 670us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 22:16:42.926547 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 22:16:42.927032 (+ 485us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 22:16:42.927033 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 22:16:42.927088 (+ 55us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 22:16:42.927626 (+ 538us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0923 22:16:42.927636 (+ 10us) client_negotiation.cc:656] calling sasl_client_start()
0923 22:16:42.929229 (+ 1593us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0923 22:16:42.931067 (+ 1838us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0923 22:16:42.931074 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0923 22:16:42.931076 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0923 22:16:42.931276 (+ 200us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0923 22:16:42.931519 (+ 243us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0923 22:16:42.931520 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0923 22:16:42.931520 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0923 22:16:42.931614 (+ 94us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0923 22:16:42.931953 (+ 339us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0923 22:16:42.931957 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0923 22:16:42.932035 (+ 78us) client_negotiation.cc:769] Sending connection context
0923 22:16:42.932158 (+ 123us) client_negotiation.cc:240] Negotiation successful
0923 22:16:42.932295 (+ 137us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":119,"thread_start_us":73,"threads_started":1}
I20260923 22:16:42.932565 24195 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 22:16:42.923658 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60000 (local address 127.1.135.254:37671)
0923 22:16:42.923780 (+ 122us) server_negotiation.cc:206] Beginning negotiation
0923 22:16:42.923785 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0923 22:16:42.923822 (+ 37us) server_negotiation.cc:407] Connection header received
0923 22:16:42.924338 (+ 516us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 22:16:42.924340 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 22:16:42.924356 (+ 16us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 22:16:42.924433 (+ 77us) server_negotiation.cc:226] Negotiated authn=SASL
0923 22:16:42.924986 (+ 553us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 22:16:42.925794 (+ 808us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 22:16:42.926626 (+ 832us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 22:16:42.926945 (+ 319us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 22:16:42.927025 (+ 80us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 22:16:42.929301 (+ 2276us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0923 22:16:42.929307 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0923 22:16:42.929307 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0923 22:16:42.929313 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0923 22:16:42.930972 (+ 1659us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0923 22:16:42.931371 (+ 399us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0923 22:16:42.931373 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0923 22:16:42.931374 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0923 22:16:42.931452 (+ 78us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0923 22:16:42.931678 (+ 226us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0923 22:16:42.931680 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0923 22:16:42.931680 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0923 22:16:42.931883 (+ 203us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0923 22:16:42.931991 (+ 108us) server_negotiation.cc:1035] Waiting for connection context
0923 22:16:42.932222 (+ 231us) server_negotiation.cc:299] Negotiation successful
0923 22:16:42.932403 (+ 181us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":63,"server-negotiator.queue_time_us":93,"thread_start_us":60,"threads_started":1}
I20260923 22:16:42.933312 24192 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.1.135.253" port: 39749 } has no permanent_uuid. Determining permanent_uuid...
I20260923 22:16:42.933703 24194 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 22:16:42.933556 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.135.253:39749 (local address 127.0.0.1:34236)
0923 22:16:42.933621 (+ 65us) negotiation.cc:107] Waiting for socket to connect
0923 22:16:42.933671 (+ 50us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.135.253:39749: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":45}
W20260923 22:16:42.933828 24141 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.1.135.253:39749: connect: Connection refused (error 111)
I20260923 22:16:42.934280 24194 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 22:16:42.934164 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.135.253:39749 (local address 127.0.0.1:34238)
0923 22:16:42.934214 (+ 50us) negotiation.cc:107] Waiting for socket to connect
0923 22:16:42.934248 (+ 34us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.135.253:39749: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":33}
W20260923 22:16:42.934404 24192 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.1.135.253:39749: Network error: Client connection negotiation failed: client connection to 127.1.135.253:39749: connect: Connection refused (error 111)
WARNING: no policy specified for kudu/127.1.135.253@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.1.135.253@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.1.135.253 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskRiQe_a/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790201539513534-1567-0/krb5kdc/kudu_127.1.135.253.keytab.
Entry for principal kudu/127.1.135.253 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskRiQe_a/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790201539513534-1567-0/krb5kdc/kudu_127.1.135.253.keytab.
I20260923 22:16:42.952939 1567 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskRiQe_a/build/release/bin/kudu
/tmp/dist-test-taskRiQe_a/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskRiQe_a/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790201539513534-1567-0/minicluster-data/master-1/wal
--fs_data_dirs=/tmp/dist-test-taskRiQe_a/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790201539513534-1567-0/minicluster-data/master-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskRiQe_a/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790201539513534-1567-0/minicluster-data/master-1/logs
--server_dump_info_path=/tmp/dist-test-taskRiQe_a/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790201539513534-1567-0/minicluster-data/master-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.1.135.253:39749
--webserver_interface=127.1.135.253
--webserver_port=0
--keytab_file=/tmp/dist-test-taskRiQe_a/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790201539513534-1567-0/krb5kdc/kudu_127.1.135.253.keytab
--principal=kudu/127.1.135.253
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.1.135.212:38547
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.1.135.254:37671,127.1.135.253:39749,127.1.135.252:33555
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--enable_leader_failure_detection=false
--rpc_authentication=required
--rpc_encryption=required with env {KRB5CCNAME=/tmp/dist-test-taskRiQe_a/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790201539513534-1567-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskRiQe_a/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790201539513534-1567-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskRiQe_a/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790201539513534-1567-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260923 22:16:42.983691 24192 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.1.135.253" port: 39749 } attempt: 1
I20260923 22:16:42.984302 24194 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 22:16:42.984145 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.135.253:39749 (local address 127.0.0.1:34254)
0923 22:16:42.984221 (+ 76us) negotiation.cc:107] Waiting for socket to connect
0923 22:16:42.984267 (+ 46us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.135.253:39749: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":48}
I20260923 22:16:42.984683 24194 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 22:16:42.984587 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.135.253:39749 (local address 127.0.0.1:34262)
0923 22:16:42.984634 (+ 47us) negotiation.cc:107] Waiting for socket to connect
0923 22:16:42.984661 (+ 27us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.135.253:39749: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":33}
W20260923 22:16:42.984817 24192 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.1.135.253:39749: Network error: Client connection negotiation failed: client connection to 127.1.135.253:39749: connect: Connection refused (error 111)
W20260923 22:16:43.033011 24199 flags.cc:425] Enabled unsafe flag: --enable_leader_failure_detection=false
W20260923 22:16:43.033344 24199 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260923 22:16:43.033375 24199 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260923 22:16:43.033412 24199 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260923 22:16:43.034788 24199 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260923 22:16:43.034833 24199 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260923 22:16:43.034847 24199 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260923 22:16:43.034860 24199 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260923 22:16:43.034906 24199 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260923 22:16:43.036459 24199 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.1.135.212:38547
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--enable_leader_failure_detection=false
--fs_data_dirs=/tmp/dist-test-taskRiQe_a/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790201539513534-1567-0/minicluster-data/master-1/data
--fs_wal_dir=/tmp/dist-test-taskRiQe_a/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790201539513534-1567-0/minicluster-data/master-1/wal
--ipki_ca_key_size=768
--master_addresses=127.1.135.254:37671,127.1.135.253:39749,127.1.135.252:33555
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskRiQe_a/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790201539513534-1567-0/krb5kdc/kudu_127.1.135.253.keytab
--principal=kudu/127.1.135.253
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.1.135.253:39749
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--rpc_encryption=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskRiQe_a/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790201539513534-1567-0/minicluster-data/master-1/data/info.pb
--webserver_interface=127.1.135.253
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.24199
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskRiQe_a/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790201539513534-1567-0/minicluster-data/master-1/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 23 Sep 2026 20:53:37 UTC on ccycloud.quasar-rkviyx.root.comops.site
build id 4668083
I20260923 22:16:43.036726 24199 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260923 22:16:43.036954 24199 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260923 22:16:43.041584 24205 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260923 22:16:43.041596 24207 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260923 22:16:43.041621 24204 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260923 22:16:43.041851 24199 server_base.cc:1034] running on GCE node
I20260923 22:16:43.042063 24199 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260923 22:16:43.042311 24199 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260923 22:16:43.043470 24199 hybrid_clock.cc:648] HybridClock initialized: now 1790201803043454 us; error 43 us; skew 500 ppm
Sep 23 22:16:43 dist-test-slave-s2fj krb5kdc[24111](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790201803, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.135.253@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260923 22:16:43.045903 24199 init.cc:377] Logged in from keytab as kudu/127.1.135.253@KRBTEST.COM (short username kudu)
I20260923 22:16:43.046372 24199 webserver.cc:460] Webserver started at http://127.1.135.253:46333/ using document root <none> and password file <none>
I20260923 22:16:43.046600 24199 fs_manager.cc:359] Metadata directory not provided
I20260923 22:16:43.046665 24199 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260923 22:16:43.046808 24199 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260923 22:16:43.047816 24199 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskRiQe_a/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790201539513534-1567-0/minicluster-data/master-1/data/instance:
uuid: "84f2054e82644722812a57bc3cc372e0"
format_stamp: "Formatted at 2026-09-23 22:16:43 on dist-test-slave-s2fj"
server_key: "c7ebc64abc22a8f1db585df761e66a0d"
server_key_iv: "a819d890f8697d5889b15193b3ba47ca"
server_key_version: "encryptionkey@0"
I20260923 22:16:43.048209 24199 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskRiQe_a/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790201539513534-1567-0/minicluster-data/master-1/wal/instance:
uuid: "84f2054e82644722812a57bc3cc372e0"
format_stamp: "Formatted at 2026-09-23 22:16:43 on dist-test-slave-s2fj"
server_key: "c7ebc64abc22a8f1db585df761e66a0d"
server_key_iv: "a819d890f8697d5889b15193b3ba47ca"
server_key_version: "encryptionkey@0"
I20260923 22:16:43.049484 24199 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.001s
I20260923 22:16:43.050161 24214 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260923 22:16:43.050324 24199 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.001s
I20260923 22:16:43.050415 24199 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskRiQe_a/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790201539513534-1567-0/minicluster-data/master-1/data,/tmp/dist-test-taskRiQe_a/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790201539513534-1567-0/minicluster-data/master-1/wal
uuid: "84f2054e82644722812a57bc3cc372e0"
format_stamp: "Formatted at 2026-09-23 22:16:43 on dist-test-slave-s2fj"
server_key: "c7ebc64abc22a8f1db585df761e66a0d"
server_key_iv: "a819d890f8697d5889b15193b3ba47ca"
server_key_version: "encryptionkey@0"
I20260923 22:16:43.050494 24199 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskRiQe_a/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790201539513534-1567-0/minicluster-data/master-1/wal
metadata directory: /tmp/dist-test-taskRiQe_a/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790201539513534-1567-0/minicluster-data/master-1/wal
1 data directories: /tmp/dist-test-taskRiQe_a/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790201539513534-1567-0/minicluster-data/master-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260923 22:16:43.052978 24192 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.1.135.253" port: 39749 } attempt: 2
I20260923 22:16:43.053363 24194 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 22:16:43.053216 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.135.253:39749 (local address 127.0.0.1:34276)
0923 22:16:43.053284 (+ 68us) negotiation.cc:107] Waiting for socket to connect
0923 22:16:43.053328 (+ 44us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.135.253:39749: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":40}
I20260923 22:16:43.053727 24194 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 22:16:43.053622 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.135.253:39749 (local address 127.0.0.1:34290)
0923 22:16:43.053667 (+ 45us) negotiation.cc:107] Waiting for socket to connect
0923 22:16:43.053695 (+ 28us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.135.253:39749: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":29}
W20260923 22:16:43.053833 24192 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.1.135.253:39749: Network error: Client connection negotiation failed: client connection to 127.1.135.253:39749: connect: Connection refused (error 111)
I20260923 22:16:43.062193 24199 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260923 22:16:43.063772 24199 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260923 22:16:43.063930 24199 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260923 22:16:43.067869 24199 rpc_server.cc:307] RPC server started. Bound to: 127.1.135.253:39749
I20260923 22:16:43.067896 24266 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.1.135.253:39749 every 8 connection(s)
I20260923 22:16:43.068254 24199 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskRiQe_a/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790201539513534-1567-0/minicluster-data/master-1/data/info.pb
I20260923 22:16:43.069113 24267 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260923 22:16:43.070595 24267 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.1.135.254" port: 37671 } has no permanent_uuid. Determining permanent_uuid...
Sep 23 22:16:43 dist-test-slave-s2fj krb5kdc[24111](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790201803, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.135.253@KRBTEST.COM for kudu/127.1.135.254@KRBTEST.COM
I20260923 22:16:43.077620 1567 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskRiQe_a/build/release/bin/kudu as pid 24199
I20260923 22:16:43.077725 1567 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskRiQe_a/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790201539513534-1567-0/minicluster-data/master-1/wal/instance
I20260923 22:16:43.077953 1567 external_mini_cluster.cc:1428] Setting key edc1ec60960882dbf17277dd4bcc4027
I20260923 22:16:43.078740 24195 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 22:16:43.071198 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60006 (local address 127.1.135.254:37671)
0923 22:16:43.071232 (+ 34us) server_negotiation.cc:206] Beginning negotiation
0923 22:16:43.071234 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0923 22:16:43.071403 (+ 169us) server_negotiation.cc:407] Connection header received
0923 22:16:43.071828 (+ 425us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 22:16:43.071829 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 22:16:43.071839 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 22:16:43.071888 (+ 49us) server_negotiation.cc:226] Negotiated authn=SASL
0923 22:16:43.072478 (+ 590us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 22:16:43.072951 (+ 473us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 22:16:43.073846 (+ 895us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 22:16:43.074035 (+ 189us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 22:16:43.074089 (+ 54us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 22:16:43.075897 (+ 1808us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0923 22:16:43.075899 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0923 22:16:43.075899 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0923 22:16:43.075906 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0923 22:16:43.077377 (+ 1471us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0923 22:16:43.077841 (+ 464us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0923 22:16:43.077846 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0923 22:16:43.077847 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0923 22:16:43.077895 (+ 48us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0923 22:16:43.078175 (+ 280us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0923 22:16:43.078176 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0923 22:16:43.078176 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0923 22:16:43.078282 (+ 106us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0923 22:16:43.078326 (+ 44us) server_negotiation.cc:1035] Waiting for connection context
0923 22:16:43.078613 (+ 287us) server_negotiation.cc:299] Negotiation successful
0923 22:16:43.078671 (+ 58us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":17}
I20260923 22:16:43.079011 24269 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 22:16:43.071158 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.135.254:37671 (local address 127.0.0.1:60006)
0923 22:16:43.071298 (+ 140us) negotiation.cc:107] Waiting for socket to connect
0923 22:16:43.071335 (+ 37us) client_negotiation.cc:174] Beginning negotiation
0923 22:16:43.071750 (+ 415us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0923 22:16:43.071935 (+ 185us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0923 22:16:43.071938 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0923 22:16:43.072238 (+ 300us) client_negotiation.cc:189] Negotiated authn=SASL
0923 22:16:43.072410 (+ 172us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 22:16:43.072414 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 22:16:43.073026 (+ 612us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 22:16:43.073027 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 22:16:43.073777 (+ 750us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 22:16:43.073779 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 22:16:43.074111 (+ 332us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 22:16:43.074112 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 22:16:43.074204 (+ 92us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 22:16:43.074584 (+ 380us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0923 22:16:43.074591 (+ 7us) client_negotiation.cc:656] calling sasl_client_start()
0923 22:16:43.075815 (+ 1224us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0923 22:16:43.077470 (+ 1655us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0923 22:16:43.077476 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0923 22:16:43.077478 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0923 22:16:43.077761 (+ 283us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0923 22:16:43.077961 (+ 200us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0923 22:16:43.077962 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0923 22:16:43.077962 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0923 22:16:43.078103 (+ 141us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0923 22:16:43.078340 (+ 237us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0923 22:16:43.078346 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0923 22:16:43.078494 (+ 148us) client_negotiation.cc:769] Sending connection context
0923 22:16:43.078598 (+ 104us) client_negotiation.cc:240] Negotiation successful
0923 22:16:43.078763 (+ 165us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":106,"thread_start_us":76,"threads_started":1}
I20260923 22:16:43.079654 24267 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.1.135.253" port: 39749 } has no permanent_uuid. Determining permanent_uuid...
Sep 23 22:16:43 dist-test-slave-s2fj krb5kdc[24111](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790201803, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.135.253@KRBTEST.COM for kudu/127.1.135.253@KRBTEST.COM
WARNING: no policy specified for kudu/127.1.135.252@KRBTEST.COM; defaulting to no policy
I20260923 22:16:43.086776 24269 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 22:16:43.079903 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.135.253:39749 (local address 127.0.0.1:34296)
0923 22:16:43.080122 (+ 219us) negotiation.cc:107] Waiting for socket to connect
0923 22:16:43.080132 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0923 22:16:43.080182 (+ 50us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0923 22:16:43.080312 (+ 130us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0923 22:16:43.080313 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0923 22:16:43.080435 (+ 122us) client_negotiation.cc:189] Negotiated authn=SASL
0923 22:16:43.080472 (+ 37us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 22:16:43.080473 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 22:16:43.081269 (+ 796us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 22:16:43.081270 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 22:16:43.081733 (+ 463us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 22:16:43.081734 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 22:16:43.082215 (+ 481us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 22:16:43.082216 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 22:16:43.082278 (+ 62us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 22:16:43.082675 (+ 397us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0923 22:16:43.082680 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0923 22:16:43.083967 (+ 1287us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0923 22:16:43.085799 (+ 1832us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0923 22:16:43.085800 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0923 22:16:43.085800 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0923 22:16:43.085947 (+ 147us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0923 22:16:43.086229 (+ 282us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0923 22:16:43.086230 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0923 22:16:43.086230 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0923 22:16:43.086279 (+ 49us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0923 22:16:43.086507 (+ 228us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0923 22:16:43.086507 (+ 0us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0923 22:16:43.086537 (+ 30us) client_negotiation.cc:769] Sending connection context
0923 22:16:43.086588 (+ 51us) client_negotiation.cc:240] Negotiation successful
0923 22:16:43.086693 (+ 105us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":205}
I20260923 22:16:43.086915 24271 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 22:16:43.079964 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34296 (local address 127.1.135.253:39749)
0923 22:16:43.080149 (+ 185us) server_negotiation.cc:206] Beginning negotiation
0923 22:16:43.080153 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0923 22:16:43.080182 (+ 29us) server_negotiation.cc:407] Connection header received
0923 22:16:43.080238 (+ 56us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 22:16:43.080240 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 22:16:43.080252 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 22:16:43.080314 (+ 62us) server_negotiation.cc:226] Negotiated authn=SASL
0923 22:16:43.080550 (+ 236us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 22:16:43.081174 (+ 624us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 22:16:43.081811 (+ 637us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 22:16:43.082139 (+ 328us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 22:16:43.082219 (+ 80us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 22:16:43.084048 (+ 1829us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0923 22:16:43.084055 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0923 22:16:43.084056 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0923 22:16:43.084082 (+ 26us) server_negotiation.cc:922] calling sasl_server_start()
0923 22:16:43.085720 (+ 1638us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0923 22:16:43.086081 (+ 361us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0923 22:16:43.086084 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0923 22:16:43.086085 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0923 22:16:43.086144 (+ 59us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0923 22:16:43.086337 (+ 193us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0923 22:16:43.086337 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0923 22:16:43.086337 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0923 22:16:43.086452 (+ 115us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0923 22:16:43.086510 (+ 58us) server_negotiation.cc:1035] Waiting for connection context
0923 22:16:43.086653 (+ 143us) server_negotiation.cc:299] Negotiation successful
0923 22:16:43.086738 (+ 85us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":25,"server-negotiator.queue_time_us":158,"thread_start_us":84,"threads_started":1}
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.1.135.252@KRBTEST.COM" created.
I20260923 22:16:43.087469 24267 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.1.135.252" port: 33555 } has no permanent_uuid. Determining permanent_uuid...
I20260923 22:16:43.087755 24269 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 22:16:43.087664 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.135.252:33555 (local address 127.0.0.1:58354)
0923 22:16:43.087701 (+ 37us) negotiation.cc:107] Waiting for socket to connect
0923 22:16:43.087738 (+ 37us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.135.252:33555: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":24}
W20260923 22:16:43.087905 24217 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.1.135.252:33555: connect: Connection refused (error 111)
I20260923 22:16:43.088299 24269 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 22:16:43.088228 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.135.252:33555 (local address 127.0.0.1:58362)
0923 22:16:43.088262 (+ 34us) negotiation.cc:107] Waiting for socket to connect
0923 22:16:43.088283 (+ 21us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.135.252:33555: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":18}
W20260923 22:16:43.088413 24267 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.1.135.252:33555: Network error: Client connection negotiation failed: client connection to 127.1.135.252:33555: connect: Connection refused (error 111)
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.1.135.252 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskRiQe_a/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790201539513534-1567-0/krb5kdc/kudu_127.1.135.252.keytab.
Entry for principal kudu/127.1.135.252 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskRiQe_a/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790201539513534-1567-0/krb5kdc/kudu_127.1.135.252.keytab.
I20260923 22:16:43.102100 1567 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskRiQe_a/build/release/bin/kudu
/tmp/dist-test-taskRiQe_a/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskRiQe_a/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790201539513534-1567-0/minicluster-data/master-2/wal
--fs_data_dirs=/tmp/dist-test-taskRiQe_a/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790201539513534-1567-0/minicluster-data/master-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskRiQe_a/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790201539513534-1567-0/minicluster-data/master-2/logs
--server_dump_info_path=/tmp/dist-test-taskRiQe_a/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790201539513534-1567-0/minicluster-data/master-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.1.135.252:33555
--webserver_interface=127.1.135.252
--webserver_port=0
--keytab_file=/tmp/dist-test-taskRiQe_a/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790201539513534-1567-0/krb5kdc/kudu_127.1.135.252.keytab
--principal=kudu/127.1.135.252
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.1.135.212:38547
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.1.135.254:37671,127.1.135.253:39749,127.1.135.252:33555
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--enable_leader_failure_detection=false
--rpc_authentication=required
--rpc_encryption=required with env {KRB5CCNAME=/tmp/dist-test-taskRiQe_a/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790201539513534-1567-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskRiQe_a/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790201539513534-1567-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskRiQe_a/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790201539513534-1567-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260923 22:16:43.137617 24267 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.1.135.252" port: 33555 } attempt: 1
I20260923 22:16:43.138114 24269 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 22:16:43.137976 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.135.252:33555 (local address 127.0.0.1:58378)
0923 22:16:43.138042 (+ 66us) negotiation.cc:107] Waiting for socket to connect
0923 22:16:43.138088 (+ 46us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.135.252:33555: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":45}
I20260923 22:16:43.138489 24269 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 22:16:43.138400 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.135.252:33555 (local address 127.0.0.1:58384)
0923 22:16:43.138453 (+ 53us) negotiation.cc:107] Waiting for socket to connect
0923 22:16:43.138474 (+ 21us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.135.252:33555: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":26}
W20260923 22:16:43.138583 24267 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.1.135.252:33555: Network error: Client connection negotiation failed: client connection to 127.1.135.252:33555: connect: Connection refused (error 111)
I20260923 22:16:43.144971 24192 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.1.135.253" port: 39749 } attempt: 3
Sep 23 22:16:43 dist-test-slave-s2fj krb5kdc[24111](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790201802, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.135.254@KRBTEST.COM for kudu/127.1.135.253@KRBTEST.COM
I20260923 22:16:43.151614 24194 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 22:16:43.145242 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.135.253:39749 (local address 127.0.0.1:34302)
0923 22:16:43.145283 (+ 41us) negotiation.cc:107] Waiting for socket to connect
0923 22:16:43.145295 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0923 22:16:43.145359 (+ 64us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0923 22:16:43.145500 (+ 141us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0923 22:16:43.145501 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0923 22:16:43.145661 (+ 160us) client_negotiation.cc:189] Negotiated authn=SASL
0923 22:16:43.145721 (+ 60us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 22:16:43.145723 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 22:16:43.146356 (+ 633us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 22:16:43.146358 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 22:16:43.146872 (+ 514us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 22:16:43.146874 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 22:16:43.147259 (+ 385us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 22:16:43.147260 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 22:16:43.147334 (+ 74us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 22:16:43.147772 (+ 438us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0923 22:16:43.147777 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0923 22:16:43.149077 (+ 1300us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0923 22:16:43.150628 (+ 1551us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0923 22:16:43.150629 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0923 22:16:43.150630 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0923 22:16:43.150796 (+ 166us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0923 22:16:43.151001 (+ 205us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0923 22:16:43.151001 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0923 22:16:43.151002 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0923 22:16:43.151057 (+ 55us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0923 22:16:43.151306 (+ 249us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0923 22:16:43.151307 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0923 22:16:43.151350 (+ 43us) client_negotiation.cc:769] Sending connection context
0923 22:16:43.151432 (+ 82us) client_negotiation.cc:240] Negotiation successful
0923 22:16:43.151523 (+ 91us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":22}
I20260923 22:16:43.151614 24271 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 22:16:43.145312 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34302 (local address 127.1.135.253:39749)
0923 22:16:43.145369 (+ 57us) server_negotiation.cc:206] Beginning negotiation
0923 22:16:43.145371 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0923 22:16:43.145382 (+ 11us) server_negotiation.cc:407] Connection header received
0923 22:16:43.145410 (+ 28us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 22:16:43.145411 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 22:16:43.145423 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 22:16:43.145493 (+ 70us) server_negotiation.cc:226] Negotiated authn=SASL
0923 22:16:43.145810 (+ 317us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 22:16:43.146279 (+ 469us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 22:16:43.146954 (+ 675us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 22:16:43.147170 (+ 216us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 22:16:43.147249 (+ 79us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 22:16:43.149165 (+ 1916us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0923 22:16:43.149167 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0923 22:16:43.149167 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0923 22:16:43.149180 (+ 13us) server_negotiation.cc:922] calling sasl_server_start()
0923 22:16:43.150499 (+ 1319us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0923 22:16:43.150877 (+ 378us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0923 22:16:43.150879 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0923 22:16:43.150879 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0923 22:16:43.150937 (+ 58us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0923 22:16:43.151133 (+ 196us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0923 22:16:43.151134 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0923 22:16:43.151135 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0923 22:16:43.151240 (+ 105us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0923 22:16:43.151315 (+ 75us) server_negotiation.cc:1035] Waiting for connection context
0923 22:16:43.151456 (+ 141us) server_negotiation.cc:299] Negotiation successful
0923 22:16:43.151524 (+ 68us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":38}
I20260923 22:16:43.151979 24192 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.1.135.252" port: 33555 } has no permanent_uuid. Determining permanent_uuid...
I20260923 22:16:43.152335 24194 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 22:16:43.152253 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.135.252:33555 (local address 127.0.0.1:58394)
0923 22:16:43.152292 (+ 39us) negotiation.cc:107] Waiting for socket to connect
0923 22:16:43.152318 (+ 26us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.135.252:33555: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":25}
I20260923 22:16:43.152654 24194 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 22:16:43.152582 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.135.252:33555 (local address 127.0.0.1:58410)
0923 22:16:43.152608 (+ 26us) negotiation.cc:107] Waiting for socket to connect
0923 22:16:43.152640 (+ 32us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.135.252:33555: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":15}
W20260923 22:16:43.152755 24192 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.1.135.252:33555: Network error: Client connection negotiation failed: client connection to 127.1.135.252:33555: connect: Connection refused (error 111)
I20260923 22:16:43.183892 24192 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.1.135.252" port: 33555 } attempt: 1
I20260923 22:16:43.184361 24194 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 22:16:43.184226 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.135.252:33555 (local address 127.0.0.1:58416)
0923 22:16:43.184285 (+ 59us) negotiation.cc:107] Waiting for socket to connect
0923 22:16:43.184324 (+ 39us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.135.252:33555: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":24}
W20260923 22:16:43.184161 24274 flags.cc:425] Enabled unsafe flag: --enable_leader_failure_detection=false
W20260923 22:16:43.184429 24274 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260923 22:16:43.184473 24274 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260923 22:16:43.184499 24274 flags.cc:425] Enabled unsafe flag: --never_fsync=true
I20260923 22:16:43.184763 24194 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 22:16:43.184697 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.135.252:33555 (local address 127.0.0.1:58426)
0923 22:16:43.184722 (+ 25us) negotiation.cc:107] Waiting for socket to connect
0923 22:16:43.184746 (+ 24us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.135.252:33555: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":13}
W20260923 22:16:43.184857 24192 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.1.135.252:33555: Network error: Client connection negotiation failed: client connection to 127.1.135.252:33555: connect: Connection refused (error 111)
W20260923 22:16:43.185804 24274 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260923 22:16:43.185840 24274 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260923 22:16:43.185886 24274 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260923 22:16:43.185905 24274 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260923 22:16:43.185940 24274 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260923 22:16:43.187522 24274 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.1.135.212:38547
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--enable_leader_failure_detection=false
--fs_data_dirs=/tmp/dist-test-taskRiQe_a/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790201539513534-1567-0/minicluster-data/master-2/data
--fs_wal_dir=/tmp/dist-test-taskRiQe_a/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790201539513534-1567-0/minicluster-data/master-2/wal
--ipki_ca_key_size=768
--master_addresses=127.1.135.254:37671,127.1.135.253:39749,127.1.135.252:33555
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskRiQe_a/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790201539513534-1567-0/krb5kdc/kudu_127.1.135.252.keytab
--principal=kudu/127.1.135.252
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.1.135.252:33555
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--rpc_encryption=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskRiQe_a/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790201539513534-1567-0/minicluster-data/master-2/data/info.pb
--webserver_interface=127.1.135.252
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.24274
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskRiQe_a/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790201539513534-1567-0/minicluster-data/master-2/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 23 Sep 2026 20:53:37 UTC on ccycloud.quasar-rkviyx.root.comops.site
build id 4668083
I20260923 22:16:43.187748 24274 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260923 22:16:43.188000 24274 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260923 22:16:43.192392 24282 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260923 22:16:43.192379 24280 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260923 22:16:43.192560 24279 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260923 22:16:43.192638 24274 server_base.cc:1034] running on GCE node
I20260923 22:16:43.192917 24274 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260923 22:16:43.193197 24274 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260923 22:16:43.194358 24274 hybrid_clock.cc:648] HybridClock initialized: now 1790201803194353 us; error 32 us; skew 500 ppm
Sep 23 22:16:43 dist-test-slave-s2fj krb5kdc[24111](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790201803, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.135.252@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260923 22:16:43.196753 24274 init.cc:377] Logged in from keytab as kudu/127.1.135.252@KRBTEST.COM (short username kudu)
I20260923 22:16:43.197204 24274 webserver.cc:460] Webserver started at http://127.1.135.252:42709/ using document root <none> and password file <none>
I20260923 22:16:43.197413 24274 fs_manager.cc:359] Metadata directory not provided
I20260923 22:16:43.197460 24274 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260923 22:16:43.197610 24274 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260923 22:16:43.198588 24274 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskRiQe_a/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790201539513534-1567-0/minicluster-data/master-2/data/instance:
uuid: "da6b6bf5f04f422db1db8315472a528b"
format_stamp: "Formatted at 2026-09-23 22:16:43 on dist-test-slave-s2fj"
server_key: "c7858758f82942b9fdb70e2aa1cd4d21"
server_key_iv: "841912c4b2177f85335ce8f58fcfd1f9"
server_key_version: "encryptionkey@0"
I20260923 22:16:43.198917 24274 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskRiQe_a/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790201539513534-1567-0/minicluster-data/master-2/wal/instance:
uuid: "da6b6bf5f04f422db1db8315472a528b"
format_stamp: "Formatted at 2026-09-23 22:16:43 on dist-test-slave-s2fj"
server_key: "c7858758f82942b9fdb70e2aa1cd4d21"
server_key_iv: "841912c4b2177f85335ce8f58fcfd1f9"
server_key_version: "encryptionkey@0"
I20260923 22:16:43.200219 24274 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.002s
I20260923 22:16:43.200876 24289 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260923 22:16:43.201064 24274 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.000s
I20260923 22:16:43.201149 24274 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskRiQe_a/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790201539513534-1567-0/minicluster-data/master-2/data,/tmp/dist-test-taskRiQe_a/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790201539513534-1567-0/minicluster-data/master-2/wal
uuid: "da6b6bf5f04f422db1db8315472a528b"
format_stamp: "Formatted at 2026-09-23 22:16:43 on dist-test-slave-s2fj"
server_key: "c7858758f82942b9fdb70e2aa1cd4d21"
server_key_iv: "841912c4b2177f85335ce8f58fcfd1f9"
server_key_version: "encryptionkey@0"
I20260923 22:16:43.201218 24274 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskRiQe_a/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790201539513534-1567-0/minicluster-data/master-2/wal
metadata directory: /tmp/dist-test-taskRiQe_a/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790201539513534-1567-0/minicluster-data/master-2/wal
1 data directories: /tmp/dist-test-taskRiQe_a/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790201539513534-1567-0/minicluster-data/master-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260923 22:16:43.206707 24267 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.1.135.252" port: 33555 } attempt: 2
I20260923 22:16:43.207079 24269 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 22:16:43.206912 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.135.252:33555 (local address 127.0.0.1:58436)
0923 22:16:43.207013 (+ 101us) negotiation.cc:107] Waiting for socket to connect
0923 22:16:43.207045 (+ 32us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.135.252:33555: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":81}
I20260923 22:16:43.207366 24269 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 22:16:43.207280 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.135.252:33555 (local address 127.0.0.1:58448)
0923 22:16:43.207323 (+ 43us) negotiation.cc:107] Waiting for socket to connect
0923 22:16:43.207346 (+ 23us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.135.252:33555: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":29}
W20260923 22:16:43.207463 24267 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.1.135.252:33555: Network error: Client connection negotiation failed: client connection to 127.1.135.252:33555: connect: Connection refused (error 111)
I20260923 22:16:43.215692 24274 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260923 22:16:43.217262 24274 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260923 22:16:43.217406 24274 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260923 22:16:43.221144 24274 rpc_server.cc:307] RPC server started. Bound to: 127.1.135.252:33555
I20260923 22:16:43.221174 24341 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.1.135.252:33555 every 8 connection(s)
I20260923 22:16:43.221475 24274 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskRiQe_a/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790201539513534-1567-0/minicluster-data/master-2/data/info.pb
I20260923 22:16:43.222375 24342 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260923 22:16:43.223668 24342 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.1.135.254" port: 37671 } has no permanent_uuid. Determining permanent_uuid...
I20260923 22:16:43.226257 1567 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskRiQe_a/build/release/bin/kudu as pid 24274
I20260923 22:16:43.226359 1567 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskRiQe_a/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790201539513534-1567-0/minicluster-data/master-2/wal/instance
I20260923 22:16:43.226550 1567 external_mini_cluster.cc:1428] Setting key edafad72d2036893d79d24008be7670b
Sep 23 22:16:43 dist-test-slave-s2fj krb5kdc[24111](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790201803, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.135.252@KRBTEST.COM for kudu/127.1.135.254@KRBTEST.COM
Sep 23 22:16:43 dist-test-slave-s2fj krb5kdc[24111](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790201802, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.1.135.252@KRBTEST.COM
I20260923 22:16:43.233131 24195 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 22:16:43.224277 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60012 (local address 127.1.135.254:37671)
0923 22:16:43.224335 (+ 58us) server_negotiation.cc:206] Beginning negotiation
0923 22:16:43.224337 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0923 22:16:43.224513 (+ 176us) server_negotiation.cc:407] Connection header received
0923 22:16:43.224952 (+ 439us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 22:16:43.224953 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 22:16:43.224964 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 22:16:43.225028 (+ 64us) server_negotiation.cc:226] Negotiated authn=SASL
0923 22:16:43.225888 (+ 860us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 22:16:43.226650 (+ 762us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 22:16:43.227832 (+ 1182us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 22:16:43.228031 (+ 199us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 22:16:43.228141 (+ 110us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 22:16:43.230376 (+ 2235us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0923 22:16:43.230377 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0923 22:16:43.230377 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0923 22:16:43.230384 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0923 22:16:43.231732 (+ 1348us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0923 22:16:43.232132 (+ 400us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0923 22:16:43.232133 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0923 22:16:43.232134 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0923 22:16:43.232191 (+ 57us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0923 22:16:43.232416 (+ 225us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0923 22:16:43.232417 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0923 22:16:43.232417 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0923 22:16:43.232548 (+ 131us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0923 22:16:43.232617 (+ 69us) server_negotiation.cc:1035] Waiting for connection context
0923 22:16:43.232926 (+ 309us) server_negotiation.cc:299] Negotiation successful
0923 22:16:43.233006 (+ 80us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":38}
I20260923 22:16:43.233243 24344 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 22:16:43.224262 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.135.254:37671 (local address 127.0.0.1:60012)
0923 22:16:43.224397 (+ 135us) negotiation.cc:107] Waiting for socket to connect
0923 22:16:43.224433 (+ 36us) client_negotiation.cc:174] Beginning negotiation
0923 22:16:43.224834 (+ 401us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0923 22:16:43.225147 (+ 313us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0923 22:16:43.225150 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0923 22:16:43.225494 (+ 344us) client_negotiation.cc:189] Negotiated authn=SASL
0923 22:16:43.225788 (+ 294us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 22:16:43.225794 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 22:16:43.226752 (+ 958us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 22:16:43.226753 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 22:16:43.227743 (+ 990us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 22:16:43.227745 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 22:16:43.228171 (+ 426us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 22:16:43.228172 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 22:16:43.228292 (+ 120us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 22:16:43.228721 (+ 429us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0923 22:16:43.228729 (+ 8us) client_negotiation.cc:656] calling sasl_client_start()
0923 22:16:43.230260 (+ 1531us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0923 22:16:43.231824 (+ 1564us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0923 22:16:43.231830 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0923 22:16:43.231831 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0923 22:16:43.232021 (+ 190us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0923 22:16:43.232255 (+ 234us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0923 22:16:43.232256 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0923 22:16:43.232256 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0923 22:16:43.232348 (+ 92us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0923 22:16:43.232613 (+ 265us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0923 22:16:43.232617 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0923 22:16:43.232783 (+ 166us) client_negotiation.cc:769] Sending connection context
0923 22:16:43.232893 (+ 110us) client_negotiation.cc:240] Negotiation successful
0923 22:16:43.233000 (+ 107us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":100,"thread_start_us":67,"threads_started":1}
I20260923 22:16:43.234032 24342 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.1.135.253" port: 39749 } has no permanent_uuid. Determining permanent_uuid...
I20260923 22:16:43.235970 24346 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 22:16:43.226924 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58462 (local address 127.1.135.252:33555)
0923 22:16:43.227099 (+ 175us) server_negotiation.cc:206] Beginning negotiation
0923 22:16:43.227103 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0923 22:16:43.227121 (+ 18us) server_negotiation.cc:407] Connection header received
0923 22:16:43.227187 (+ 66us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 22:16:43.227190 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 22:16:43.227206 (+ 16us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 22:16:43.227294 (+ 88us) server_negotiation.cc:226] Negotiated authn=SASL
0923 22:16:43.227830 (+ 536us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 22:16:43.228786 (+ 956us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 22:16:43.229610 (+ 824us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 22:16:43.229931 (+ 321us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 22:16:43.229993 (+ 62us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 22:16:43.232595 (+ 2602us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0923 22:16:43.232602 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0923 22:16:43.232603 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0923 22:16:43.232609 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0923 22:16:43.234902 (+ 2293us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0923 22:16:43.235195 (+ 293us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0923 22:16:43.235198 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0923 22:16:43.235198 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0923 22:16:43.235252 (+ 54us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0923 22:16:43.235438 (+ 186us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0923 22:16:43.235439 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0923 22:16:43.235439 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0923 22:16:43.235601 (+ 162us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0923 22:16:43.235742 (+ 141us) server_negotiation.cc:1035] Waiting for connection context
0923 22:16:43.235813 (+ 71us) server_negotiation.cc:299] Negotiation successful
0923 22:16:43.235878 (+ 65us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":129,"thread_start_us":105,"threads_started":1}
Sep 23 22:16:43 dist-test-slave-s2fj krb5kdc[24111](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790201803, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.135.252@KRBTEST.COM for kudu/127.1.135.253@KRBTEST.COM
I20260923 22:16:43.240520 24271 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 22:16:43.234359 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34308 (local address 127.1.135.253:39749)
0923 22:16:43.234405 (+ 46us) server_negotiation.cc:206] Beginning negotiation
0923 22:16:43.234407 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0923 22:16:43.234451 (+ 44us) server_negotiation.cc:407] Connection header received
0923 22:16:43.234519 (+ 68us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 22:16:43.234520 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 22:16:43.234527 (+ 7us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 22:16:43.234570 (+ 43us) server_negotiation.cc:226] Negotiated authn=SASL
0923 22:16:43.234901 (+ 331us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 22:16:43.235285 (+ 384us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 22:16:43.235900 (+ 615us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 22:16:43.236220 (+ 320us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 22:16:43.236298 (+ 78us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 22:16:43.238012 (+ 1714us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0923 22:16:43.238014 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0923 22:16:43.238014 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0923 22:16:43.238021 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0923 22:16:43.239479 (+ 1458us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0923 22:16:43.239819 (+ 340us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0923 22:16:43.239820 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0923 22:16:43.239820 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0923 22:16:43.239863 (+ 43us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0923 22:16:43.240057 (+ 194us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0923 22:16:43.240058 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0923 22:16:43.240058 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0923 22:16:43.240203 (+ 145us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0923 22:16:43.240250 (+ 47us) server_negotiation.cc:1035] Waiting for connection context
0923 22:16:43.240405 (+ 155us) server_negotiation.cc:299] Negotiation successful
0923 22:16:43.240468 (+ 63us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":26}
I20260923 22:16:43.240597 24344 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 22:16:43.234288 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.135.253:39749 (local address 127.0.0.1:34308)
0923 22:16:43.234404 (+ 116us) negotiation.cc:107] Waiting for socket to connect
0923 22:16:43.234414 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0923 22:16:43.234463 (+ 49us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0923 22:16:43.234584 (+ 121us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0923 22:16:43.234585 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0923 22:16:43.234788 (+ 203us) client_negotiation.cc:189] Negotiated authn=SASL
0923 22:16:43.234821 (+ 33us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 22:16:43.234822 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 22:16:43.235359 (+ 537us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 22:16:43.235360 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 22:16:43.235816 (+ 456us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 22:16:43.235817 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 22:16:43.236347 (+ 530us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 22:16:43.236348 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 22:16:43.236402 (+ 54us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 22:16:43.236782 (+ 380us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0923 22:16:43.236786 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0923 22:16:43.237939 (+ 1153us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0923 22:16:43.239535 (+ 1596us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0923 22:16:43.239536 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0923 22:16:43.239536 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0923 22:16:43.239746 (+ 210us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0923 22:16:43.239927 (+ 181us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0923 22:16:43.239928 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0923 22:16:43.239929 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0923 22:16:43.239996 (+ 67us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0923 22:16:43.240269 (+ 273us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0923 22:16:43.240270 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0923 22:16:43.240314 (+ 44us) client_negotiation.cc:769] Sending connection context
0923 22:16:43.240389 (+ 75us) client_negotiation.cc:240] Negotiation successful
0923 22:16:43.240486 (+ 97us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":102}
I20260923 22:16:43.240960 24342 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.1.135.252" port: 33555 } has no permanent_uuid. Determining permanent_uuid...
Sep 23 22:16:43 dist-test-slave-s2fj krb5kdc[24111](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790201802, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.1.135.254@KRBTEST.COM
I20260923 22:16:43.244695 24195 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 22:16:43.236753 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60024 (local address 127.1.135.254:37671)
0923 22:16:43.236789 (+ 36us) server_negotiation.cc:206] Beginning negotiation
0923 22:16:43.236791 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0923 22:16:43.236826 (+ 35us) server_negotiation.cc:407] Connection header received
0923 22:16:43.236892 (+ 66us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 22:16:43.236893 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 22:16:43.236902 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 22:16:43.236942 (+ 40us) server_negotiation.cc:226] Negotiated authn=SASL
0923 22:16:43.237487 (+ 545us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 22:16:43.237999 (+ 512us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 22:16:43.238736 (+ 737us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 22:16:43.239034 (+ 298us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 22:16:43.239111 (+ 77us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 22:16:43.241598 (+ 2487us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0923 22:16:43.241599 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0923 22:16:43.241600 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0923 22:16:43.241606 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0923 22:16:43.243645 (+ 2039us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0923 22:16:43.243957 (+ 312us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0923 22:16:43.243957 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0923 22:16:43.243958 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0923 22:16:43.244014 (+ 56us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0923 22:16:43.244241 (+ 227us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0923 22:16:43.244241 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0923 22:16:43.244241 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0923 22:16:43.244324 (+ 83us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0923 22:16:43.244457 (+ 133us) server_negotiation.cc:1035] Waiting for connection context
0923 22:16:43.244520 (+ 63us) server_negotiation.cc:299] Negotiation successful
0923 22:16:43.244607 (+ 87us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":19}
Sep 23 22:16:43 dist-test-slave-s2fj krb5kdc[24111](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790201803, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.135.252@KRBTEST.COM for kudu/127.1.135.252@KRBTEST.COM
I20260923 22:16:43.248572 24344 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 22:16:43.241159 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.135.252:33555 (local address 127.0.0.1:58474)
0923 22:16:43.241187 (+ 28us) negotiation.cc:107] Waiting for socket to connect
0923 22:16:43.241221 (+ 34us) client_negotiation.cc:174] Beginning negotiation
0923 22:16:43.241252 (+ 31us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0923 22:16:43.241705 (+ 453us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0923 22:16:43.241706 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0923 22:16:43.242247 (+ 541us) client_negotiation.cc:189] Negotiated authn=SASL
0923 22:16:43.242277 (+ 30us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 22:16:43.242279 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 22:16:43.243214 (+ 935us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 22:16:43.243215 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 22:16:43.243900 (+ 685us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 22:16:43.243902 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 22:16:43.244428 (+ 526us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 22:16:43.244429 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 22:16:43.244484 (+ 55us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 22:16:43.245011 (+ 527us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0923 22:16:43.245016 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0923 22:16:43.246141 (+ 1125us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0923 22:16:43.247581 (+ 1440us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0923 22:16:43.247581 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0923 22:16:43.247582 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0923 22:16:43.247728 (+ 146us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0923 22:16:43.247916 (+ 188us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0923 22:16:43.247917 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0923 22:16:43.247918 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0923 22:16:43.247967 (+ 49us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0923 22:16:43.248240 (+ 273us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0923 22:16:43.248241 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0923 22:16:43.248275 (+ 34us) client_negotiation.cc:769] Sending connection context
0923 22:16:43.248357 (+ 82us) client_negotiation.cc:240] Negotiation successful
0923 22:16:43.248447 (+ 90us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":15}
I20260923 22:16:43.248582 24346 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 22:16:43.241506 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58474 (local address 127.1.135.252:33555)
0923 22:16:43.241574 (+ 68us) server_negotiation.cc:206] Beginning negotiation
0923 22:16:43.241576 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0923 22:16:43.241581 (+ 5us) server_negotiation.cc:407] Connection header received
0923 22:16:43.241597 (+ 16us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 22:16:43.241597 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 22:16:43.241605 (+ 8us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 22:16:43.241672 (+ 67us) server_negotiation.cc:226] Negotiated authn=SASL
0923 22:16:43.242378 (+ 706us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 22:16:43.243113 (+ 735us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 22:16:43.244006 (+ 893us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 22:16:43.244338 (+ 332us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 22:16:43.244415 (+ 77us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 22:16:43.246229 (+ 1814us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0923 22:16:43.246230 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0923 22:16:43.246230 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0923 22:16:43.246235 (+ 5us) server_negotiation.cc:922] calling sasl_server_start()
0923 22:16:43.247484 (+ 1249us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0923 22:16:43.247802 (+ 318us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0923 22:16:43.247803 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0923 22:16:43.247803 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0923 22:16:43.247848 (+ 45us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0923 22:16:43.248044 (+ 196us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0923 22:16:43.248044 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0923 22:16:43.248045 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0923 22:16:43.248173 (+ 128us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0923 22:16:43.248245 (+ 72us) server_negotiation.cc:1035] Waiting for connection context
0923 22:16:43.248383 (+ 138us) server_negotiation.cc:299] Negotiation successful
0923 22:16:43.248492 (+ 109us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":50,"server-negotiator.queue_time_us":50}
Sep 23 22:16:43 dist-test-slave-s2fj krb5kdc[24111](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790201802, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.1.135.253@KRBTEST.COM
I20260923 22:16:43.249907 24342 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P da6b6bf5f04f422db1db8315472a528b: Bootstrap starting.
I20260923 22:16:43.250885 24342 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P da6b6bf5f04f422db1db8315472a528b: Neither blocks nor log segments found. Creating new log.
I20260923 22:16:43.251191 24342 log.cc:824] T 00000000000000000000000000000000 P da6b6bf5f04f422db1db8315472a528b: Log is configured to *not* fsync() on all Append() calls
I20260923 22:16:43.251948 24342 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P da6b6bf5f04f422db1db8315472a528b: No bootstrap required, opened a new log
I20260923 22:16:43.252622 24271 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 22:16:43.245276 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34318 (local address 127.1.135.253:39749)
0923 22:16:43.245307 (+ 31us) server_negotiation.cc:206] Beginning negotiation
0923 22:16:43.245309 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0923 22:16:43.245317 (+ 8us) server_negotiation.cc:407] Connection header received
0923 22:16:43.245339 (+ 22us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 22:16:43.245340 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 22:16:43.245349 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 22:16:43.245431 (+ 82us) server_negotiation.cc:226] Negotiated authn=SASL
0923 22:16:43.245884 (+ 453us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 22:16:43.246331 (+ 447us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 22:16:43.247385 (+ 1054us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 22:16:43.247596 (+ 211us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 22:16:43.247663 (+ 67us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 22:16:43.250029 (+ 2366us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0923 22:16:43.250030 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0923 22:16:43.250030 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0923 22:16:43.250037 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0923 22:16:43.251525 (+ 1488us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0923 22:16:43.251846 (+ 321us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0923 22:16:43.251847 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0923 22:16:43.251847 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0923 22:16:43.251903 (+ 56us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0923 22:16:43.252147 (+ 244us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0923 22:16:43.252149 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0923 22:16:43.252149 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0923 22:16:43.252251 (+ 102us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0923 22:16:43.252416 (+ 165us) server_negotiation.cc:1035] Waiting for connection context
0923 22:16:43.252457 (+ 41us) server_negotiation.cc:299] Negotiation successful
0923 22:16:43.252519 (+ 62us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":15}
I20260923 22:16:43.253935 24342 raft_consensus.cc:348] T 00000000000000000000000000000000 P da6b6bf5f04f422db1db8315472a528b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7951ee4f672c48e29ed3b1b74a092b78" member_type: VOTER last_known_addr { host: "127.1.135.254" port: 37671 } } peers { permanent_uuid: "84f2054e82644722812a57bc3cc372e0" member_type: VOTER last_known_addr { host: "127.1.135.253" port: 39749 } } peers { permanent_uuid: "da6b6bf5f04f422db1db8315472a528b" member_type: VOTER last_known_addr { host: "127.1.135.252" port: 33555 } }
I20260923 22:16:43.254077 1567 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260923 22:16:43.254101 24342 raft_consensus.cc:729] T 00000000000000000000000000000000 P da6b6bf5f04f422db1db8315472a528b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: da6b6bf5f04f422db1db8315472a528b, State: Initialized, Role: FOLLOWER
I20260923 22:16:43.254194 24342 consensus_queue.cc:260] T 00000000000000000000000000000000 P da6b6bf5f04f422db1db8315472a528b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7951ee4f672c48e29ed3b1b74a092b78" member_type: VOTER last_known_addr { host: "127.1.135.254" port: 37671 } } peers { permanent_uuid: "84f2054e82644722812a57bc3cc372e0" member_type: VOTER last_known_addr { host: "127.1.135.253" port: 39749 } } peers { permanent_uuid: "da6b6bf5f04f422db1db8315472a528b" member_type: VOTER last_known_addr { host: "127.1.135.252" port: 33555 } }
I20260923 22:16:43.254462 24342 sys_catalog.cc:564] T 00000000000000000000000000000000 P da6b6bf5f04f422db1db8315472a528b [sys.catalog]: configured and running, proceeding with master startup.
I20260923 22:16:43.254875 24348 sys_catalog.cc:455] T 00000000000000000000000000000000 P da6b6bf5f04f422db1db8315472a528b [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7951ee4f672c48e29ed3b1b74a092b78" member_type: VOTER last_known_addr { host: "127.1.135.254" port: 37671 } } peers { permanent_uuid: "84f2054e82644722812a57bc3cc372e0" member_type: VOTER last_known_addr { host: "127.1.135.253" port: 39749 } } peers { permanent_uuid: "da6b6bf5f04f422db1db8315472a528b" member_type: VOTER last_known_addr { host: "127.1.135.252" port: 33555 } } }
I20260923 22:16:43.254977 24348 sys_catalog.cc:458] T 00000000000000000000000000000000 P da6b6bf5f04f422db1db8315472a528b [sys.catalog]: This master's current role is: FOLLOWER
I20260923 22:16:43.254951 24166 tablet_service.cc:1939] Received Run Leader Election RPC: tablet_id: "00000000000000000000000000000000"
dest_uuid: "7951ee4f672c48e29ed3b1b74a092b78"
from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:60024
W20260923 22:16:43.256243 24359 catalog_manager.cc:1558] T 00000000000000000000000000000000 P da6b6bf5f04f422db1db8315472a528b: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260923 22:16:43.256295 24359 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260923 22:16:43.256578 24274 master_runner.cc:186] Error getting master registration for 127.1.135.254:37671: OK, instance_id { permanent_uuid: "7951ee4f672c48e29ed3b1b74a092b78" instance_seqno: 1790201802918661 } error { code: CATALOG_MANAGER_NOT_INITIALIZED status { code: SERVICE_UNAVAILABLE message: "Catalog manager is not initialized. State: Starting" } }
I20260923 22:16:43.259289 24274 master_runner.cc:418] Couldn't verify the masters in the cluster. Trying again...
I20260923 22:16:43.259994 24192 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.1.135.252" port: 33555 } attempt: 2
Sep 23 22:16:43 dist-test-slave-s2fj krb5kdc[24111](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790201802, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.135.254@KRBTEST.COM for kudu/127.1.135.252@KRBTEST.COM
I20260923 22:16:43.267516 24346 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 22:16:43.260343 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58484 (local address 127.1.135.252:33555)
0923 22:16:43.260391 (+ 48us) server_negotiation.cc:206] Beginning negotiation
0923 22:16:43.260393 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0923 22:16:43.260401 (+ 8us) server_negotiation.cc:407] Connection header received
0923 22:16:43.260464 (+ 63us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 22:16:43.260465 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 22:16:43.260474 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 22:16:43.260541 (+ 67us) server_negotiation.cc:226] Negotiated authn=SASL
0923 22:16:43.260817 (+ 276us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 22:16:43.261520 (+ 703us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 22:16:43.262479 (+ 959us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 22:16:43.262792 (+ 313us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 22:16:43.262863 (+ 71us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 22:16:43.264822 (+ 1959us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0923 22:16:43.264823 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0923 22:16:43.264823 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0923 22:16:43.264829 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0923 22:16:43.266389 (+ 1560us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0923 22:16:43.266762 (+ 373us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0923 22:16:43.266763 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0923 22:16:43.266763 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0923 22:16:43.266815 (+ 52us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0923 22:16:43.267020 (+ 205us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0923 22:16:43.267021 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0923 22:16:43.267021 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0923 22:16:43.267163 (+ 142us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0923 22:16:43.267223 (+ 60us) server_negotiation.cc:1035] Waiting for connection context
0923 22:16:43.267369 (+ 146us) server_negotiation.cc:299] Negotiation successful
0923 22:16:43.267435 (+ 66us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":17}
I20260923 22:16:43.267521 24194 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 22:16:43.260295 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.135.252:33555 (local address 127.0.0.1:58484)
0923 22:16:43.260343 (+ 48us) negotiation.cc:107] Waiting for socket to connect
0923 22:16:43.260352 (+ 9us) client_negotiation.cc:174] Beginning negotiation
0923 22:16:43.260389 (+ 37us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0923 22:16:43.260554 (+ 165us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0923 22:16:43.260555 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0923 22:16:43.260707 (+ 152us) client_negotiation.cc:189] Negotiated authn=SASL
0923 22:16:43.260745 (+ 38us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 22:16:43.260747 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 22:16:43.261599 (+ 852us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 22:16:43.261599 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 22:16:43.262394 (+ 795us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 22:16:43.262396 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 22:16:43.262868 (+ 472us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 22:16:43.262869 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 22:16:43.262923 (+ 54us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 22:16:43.263335 (+ 412us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0923 22:16:43.263339 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0923 22:16:43.264737 (+ 1398us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0923 22:16:43.266476 (+ 1739us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0923 22:16:43.266477 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0923 22:16:43.266477 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0923 22:16:43.266687 (+ 210us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0923 22:16:43.266882 (+ 195us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0923 22:16:43.266882 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0923 22:16:43.266883 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0923 22:16:43.266944 (+ 61us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0923 22:16:43.267230 (+ 286us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0923 22:16:43.267231 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0923 22:16:43.267273 (+ 42us) client_negotiation.cc:769] Sending connection context
0923 22:16:43.267348 (+ 75us) client_negotiation.cc:240] Negotiation successful
0923 22:16:43.267430 (+ 82us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":31}
I20260923 22:16:43.268864 24192 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 7951ee4f672c48e29ed3b1b74a092b78: Bootstrap starting.
I20260923 22:16:43.269510 24192 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 7951ee4f672c48e29ed3b1b74a092b78: Neither blocks nor log segments found. Creating new log.
I20260923 22:16:43.269797 24192 log.cc:824] T 00000000000000000000000000000000 P 7951ee4f672c48e29ed3b1b74a092b78: Log is configured to *not* fsync() on all Append() calls
I20260923 22:16:43.270488 24192 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 7951ee4f672c48e29ed3b1b74a092b78: No bootstrap required, opened a new log
I20260923 22:16:43.271751 24192 raft_consensus.cc:348] T 00000000000000000000000000000000 P 7951ee4f672c48e29ed3b1b74a092b78 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7951ee4f672c48e29ed3b1b74a092b78" member_type: VOTER last_known_addr { host: "127.1.135.254" port: 37671 } } peers { permanent_uuid: "84f2054e82644722812a57bc3cc372e0" member_type: VOTER last_known_addr { host: "127.1.135.253" port: 39749 } } peers { permanent_uuid: "da6b6bf5f04f422db1db8315472a528b" member_type: VOTER last_known_addr { host: "127.1.135.252" port: 33555 } }
I20260923 22:16:43.271878 24192 raft_consensus.cc:729] T 00000000000000000000000000000000 P 7951ee4f672c48e29ed3b1b74a092b78 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 7951ee4f672c48e29ed3b1b74a092b78, State: Initialized, Role: FOLLOWER
I20260923 22:16:43.271944 24192 consensus_queue.cc:260] T 00000000000000000000000000000000 P 7951ee4f672c48e29ed3b1b74a092b78 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7951ee4f672c48e29ed3b1b74a092b78" member_type: VOTER last_known_addr { host: "127.1.135.254" port: 37671 } } peers { permanent_uuid: "84f2054e82644722812a57bc3cc372e0" member_type: VOTER last_known_addr { host: "127.1.135.253" port: 39749 } } peers { permanent_uuid: "da6b6bf5f04f422db1db8315472a528b" member_type: VOTER last_known_addr { host: "127.1.135.252" port: 33555 } }
I20260923 22:16:43.272250 24362 sys_catalog.cc:455] T 00000000000000000000000000000000 P 7951ee4f672c48e29ed3b1b74a092b78 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7951ee4f672c48e29ed3b1b74a092b78" member_type: VOTER last_known_addr { host: "127.1.135.254" port: 37671 } } peers { permanent_uuid: "84f2054e82644722812a57bc3cc372e0" member_type: VOTER last_known_addr { host: "127.1.135.253" port: 39749 } } peers { permanent_uuid: "da6b6bf5f04f422db1db8315472a528b" member_type: VOTER last_known_addr { host: "127.1.135.252" port: 33555 } } }
I20260923 22:16:43.272295 24192 sys_catalog.cc:564] T 00000000000000000000000000000000 P 7951ee4f672c48e29ed3b1b74a092b78 [sys.catalog]: configured and running, proceeding with master startup.
I20260923 22:16:43.272336 24362 sys_catalog.cc:458] T 00000000000000000000000000000000 P 7951ee4f672c48e29ed3b1b74a092b78 [sys.catalog]: This master's current role is: FOLLOWER
I20260923 22:16:43.272348 24166 raft_consensus.cc:482] T 00000000000000000000000000000000 P 7951ee4f672c48e29ed3b1b74a092b78 [term 0 FOLLOWER]: Starting forced leader election (received explicit request)
I20260923 22:16:43.272419 24166 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 7951ee4f672c48e29ed3b1b74a092b78 [term 0 FOLLOWER]: Advancing to term 1
I20260923 22:16:43.273206 24166 raft_consensus.cc:504] T 00000000000000000000000000000000 P 7951ee4f672c48e29ed3b1b74a092b78 [term 1 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7951ee4f672c48e29ed3b1b74a092b78" member_type: VOTER last_known_addr { host: "127.1.135.254" port: 37671 } } peers { permanent_uuid: "84f2054e82644722812a57bc3cc372e0" member_type: VOTER last_known_addr { host: "127.1.135.253" port: 39749 } } peers { permanent_uuid: "da6b6bf5f04f422db1db8315472a528b" member_type: VOTER last_known_addr { host: "127.1.135.252" port: 33555 } }
I20260923 22:16:43.273619 24166 leader_election.cc:288] T 00000000000000000000000000000000 P 7951ee4f672c48e29ed3b1b74a092b78 [CANDIDATE]: Term 1 election: Requested vote from peers 84f2054e82644722812a57bc3cc372e0 (127.1.135.253:39749), da6b6bf5f04f422db1db8315472a528b (127.1.135.252:33555)
I20260923 22:16:43.273936 24316 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "7951ee4f672c48e29ed3b1b74a092b78" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: true dest_uuid: "da6b6bf5f04f422db1db8315472a528b"
I20260923 22:16:43.274041 24316 raft_consensus.cc:3037] T 00000000000000000000000000000000 P da6b6bf5f04f422db1db8315472a528b [term 0 FOLLOWER]: Advancing to term 1
I20260923 22:16:43.274825 24316 raft_consensus.cc:2454] T 00000000000000000000000000000000 P da6b6bf5f04f422db1db8315472a528b [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7951ee4f672c48e29ed3b1b74a092b78 in term 1.
I20260923 22:16:43.275144 24142 leader_election.cc:302] T 00000000000000000000000000000000 P 7951ee4f672c48e29ed3b1b74a092b78 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 7951ee4f672c48e29ed3b1b74a092b78, da6b6bf5f04f422db1db8315472a528b; no voters:
I20260923 22:16:43.275273 24362 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 7951ee4f672c48e29ed3b1b74a092b78 [term 1 FOLLOWER]: Leader election won for term 1
I20260923 22:16:43.275422 24362 raft_consensus.cc:686] T 00000000000000000000000000000000 P 7951ee4f672c48e29ed3b1b74a092b78 [term 1 LEADER]: Becoming Leader. State: Replica: 7951ee4f672c48e29ed3b1b74a092b78, State: Running, Role: LEADER
I20260923 22:16:43.275491 24362 consensus_queue.cc:237] T 00000000000000000000000000000000 P 7951ee4f672c48e29ed3b1b74a092b78 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7951ee4f672c48e29ed3b1b74a092b78" member_type: VOTER last_known_addr { host: "127.1.135.254" port: 37671 } } peers { permanent_uuid: "84f2054e82644722812a57bc3cc372e0" member_type: VOTER last_known_addr { host: "127.1.135.253" port: 39749 } } peers { permanent_uuid: "da6b6bf5f04f422db1db8315472a528b" member_type: VOTER last_known_addr { host: "127.1.135.252" port: 33555 } }
I20260923 22:16:43.275825 24362 sys_catalog.cc:455] T 00000000000000000000000000000000 P 7951ee4f672c48e29ed3b1b74a092b78 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 7951ee4f672c48e29ed3b1b74a092b78. Latest consensus state: current_term: 1 leader_uuid: "7951ee4f672c48e29ed3b1b74a092b78" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7951ee4f672c48e29ed3b1b74a092b78" member_type: VOTER last_known_addr { host: "127.1.135.254" port: 37671 } } peers { permanent_uuid: "84f2054e82644722812a57bc3cc372e0" member_type: VOTER last_known_addr { host: "127.1.135.253" port: 39749 } } peers { permanent_uuid: "da6b6bf5f04f422db1db8315472a528b" member_type: VOTER last_known_addr { host: "127.1.135.252" port: 33555 } } }
I20260923 22:16:43.275880 24362 sys_catalog.cc:458] T 00000000000000000000000000000000 P 7951ee4f672c48e29ed3b1b74a092b78 [sys.catalog]: This master's current role is: LEADER
W20260923 22:16:43.276628 24373 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 7951ee4f672c48e29ed3b1b74a092b78: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260923 22:16:43.276678 24373 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260923 22:16:43.275220 24124 master_runner.cc:186] Error getting master registration for 127.1.135.253:39749: OK, instance_id { permanent_uuid: "84f2054e82644722812a57bc3cc372e0" instance_seqno: 1790201803066699 } error { code: CATALOG_MANAGER_NOT_INITIALIZED status { code: SERVICE_UNAVAILABLE message: "Catalog manager is not initialized. State: Starting" } }
I20260923 22:16:43.278265 24124 master_runner.cc:418] Couldn't verify the masters in the cluster. Trying again...
I20260923 22:16:43.283851 24380 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260923 22:16:43.286578 24346 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 22:16:43.275047 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58492 (local address 127.1.135.252:33555)
0923 22:16:43.275097 (+ 50us) server_negotiation.cc:206] Beginning negotiation
0923 22:16:43.275099 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0923 22:16:43.276342 (+ 1243us) server_negotiation.cc:407] Connection header received
0923 22:16:43.276486 (+ 144us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 22:16:43.276487 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 22:16:43.276495 (+ 8us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 22:16:43.276534 (+ 39us) server_negotiation.cc:226] Negotiated authn=SASL
0923 22:16:43.277607 (+ 1073us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 22:16:43.278283 (+ 676us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 22:16:43.279731 (+ 1448us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 22:16:43.280018 (+ 287us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 22:16:43.280085 (+ 67us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 22:16:43.282526 (+ 2441us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0923 22:16:43.282527 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0923 22:16:43.282528 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0923 22:16:43.282545 (+ 17us) server_negotiation.cc:922] calling sasl_server_start()
0923 22:16:43.284898 (+ 2353us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0923 22:16:43.285491 (+ 593us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0923 22:16:43.285492 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0923 22:16:43.285492 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0923 22:16:43.285556 (+ 64us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0923 22:16:43.285853 (+ 297us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0923 22:16:43.285854 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0923 22:16:43.285854 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0923 22:16:43.285976 (+ 122us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0923 22:16:43.286042 (+ 66us) server_negotiation.cc:1035] Waiting for connection context
0923 22:16:43.286413 (+ 371us) server_negotiation.cc:299] Negotiation successful
0923 22:16:43.286495 (+ 82us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":22}
I20260923 22:16:43.286610 24271 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 22:16:43.275555 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34330 (local address 127.1.135.253:39749)
0923 22:16:43.275598 (+ 43us) server_negotiation.cc:206] Beginning negotiation
0923 22:16:43.275600 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0923 22:16:43.278478 (+ 2878us) server_negotiation.cc:407] Connection header received
0923 22:16:43.278576 (+ 98us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 22:16:43.278577 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 22:16:43.278586 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 22:16:43.278614 (+ 28us) server_negotiation.cc:226] Negotiated authn=SASL
0923 22:16:43.279337 (+ 723us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 22:16:43.279989 (+ 652us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 22:16:43.280963 (+ 974us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 22:16:43.281663 (+ 700us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 22:16:43.281708 (+ 45us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 22:16:43.283489 (+ 1781us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0923 22:16:43.283491 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0923 22:16:43.283491 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0923 22:16:43.283497 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0923 22:16:43.285291 (+ 1794us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0923 22:16:43.285685 (+ 394us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0923 22:16:43.285687 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0923 22:16:43.285687 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0923 22:16:43.285753 (+ 66us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0923 22:16:43.285946 (+ 193us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0923 22:16:43.285947 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0923 22:16:43.285947 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0923 22:16:43.286075 (+ 128us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0923 22:16:43.286142 (+ 67us) server_negotiation.cc:1035] Waiting for connection context
0923 22:16:43.286339 (+ 197us) server_negotiation.cc:299] Negotiation successful
0923 22:16:43.286524 (+ 185us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":20}
I20260923 22:16:43.287547 24380 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260923 22:16:43.289722 24316 raft_consensus.cc:1261] T 00000000000000000000000000000000 P da6b6bf5f04f422db1db8315472a528b [term 1 FOLLOWER]: Refusing update from remote peer 7951ee4f672c48e29ed3b1b74a092b78: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260923 22:16:43.290037 24378 consensus_queue.cc:1035] T 00000000000000000000000000000000 P 7951ee4f672c48e29ed3b1b74a092b78 [LEADER]: Connected to new peer: Peer: permanent_uuid: "da6b6bf5f04f422db1db8315472a528b" member_type: VOTER last_known_addr { host: "127.1.135.252" port: 33555 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260923 22:16:43.291666 24348 sys_catalog.cc:455] T 00000000000000000000000000000000 P da6b6bf5f04f422db1db8315472a528b [sys.catalog]: SysCatalogTable state changed. Reason: New leader 7951ee4f672c48e29ed3b1b74a092b78. Latest consensus state: current_term: 1 leader_uuid: "7951ee4f672c48e29ed3b1b74a092b78" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7951ee4f672c48e29ed3b1b74a092b78" member_type: VOTER last_known_addr { host: "127.1.135.254" port: 37671 } } peers { permanent_uuid: "84f2054e82644722812a57bc3cc372e0" member_type: VOTER last_known_addr { host: "127.1.135.253" port: 39749 } } peers { permanent_uuid: "da6b6bf5f04f422db1db8315472a528b" member_type: VOTER last_known_addr { host: "127.1.135.252" port: 33555 } } }
I20260923 22:16:43.291752 24348 sys_catalog.cc:458] T 00000000000000000000000000000000 P da6b6bf5f04f422db1db8315472a528b [sys.catalog]: This master's current role is: FOLLOWER
I20260923 22:16:43.296972 24195 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 22:16:43.276165 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60038 (local address 127.1.135.254:37671)
0923 22:16:43.276199 (+ 34us) server_negotiation.cc:206] Beginning negotiation
0923 22:16:43.276201 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0923 22:16:43.282138 (+ 5937us) server_negotiation.cc:407] Connection header received
0923 22:16:43.284470 (+ 2332us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 22:16:43.284471 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 22:16:43.284480 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 22:16:43.284510 (+ 30us) server_negotiation.cc:226] Negotiated authn=SASL
0923 22:16:43.289023 (+ 4513us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 22:16:43.289666 (+ 643us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 22:16:43.291251 (+ 1585us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 22:16:43.291551 (+ 300us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 22:16:43.291592 (+ 41us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 22:16:43.293796 (+ 2204us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0923 22:16:43.293797 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0923 22:16:43.293798 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0923 22:16:43.293815 (+ 17us) server_negotiation.cc:922] calling sasl_server_start()
0923 22:16:43.295392 (+ 1577us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0923 22:16:43.295940 (+ 548us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0923 22:16:43.295941 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0923 22:16:43.295941 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0923 22:16:43.296005 (+ 64us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0923 22:16:43.296311 (+ 306us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0923 22:16:43.296312 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0923 22:16:43.296313 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0923 22:16:43.296441 (+ 128us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0923 22:16:43.296526 (+ 85us) server_negotiation.cc:1035] Waiting for connection context
0923 22:16:43.296822 (+ 296us) server_negotiation.cc:299] Negotiation successful
0923 22:16:43.296890 (+ 68us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":18}
I20260923 22:16:43.298609 24267 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.1.135.252" port: 33555 } attempt: 3
Sep 23 22:16:43 dist-test-slave-s2fj krb5kdc[24111](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790201803, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.135.253@KRBTEST.COM for kudu/127.1.135.252@KRBTEST.COM
I20260923 22:16:43.305958 24362 sys_catalog.cc:455] T 00000000000000000000000000000000 P 7951ee4f672c48e29ed3b1b74a092b78 [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "7951ee4f672c48e29ed3b1b74a092b78" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7951ee4f672c48e29ed3b1b74a092b78" member_type: VOTER last_known_addr { host: "127.1.135.254" port: 37671 } } peers { permanent_uuid: "84f2054e82644722812a57bc3cc372e0" member_type: VOTER last_known_addr { host: "127.1.135.253" port: 39749 } } peers { permanent_uuid: "da6b6bf5f04f422db1db8315472a528b" member_type: VOTER last_known_addr { host: "127.1.135.252" port: 33555 } } }
I20260923 22:16:43.306041 24362 sys_catalog.cc:458] T 00000000000000000000000000000000 P 7951ee4f672c48e29ed3b1b74a092b78 [sys.catalog]: This master's current role is: LEADER
I20260923 22:16:43.306404 24269 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 22:16:43.298834 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.135.252:33555 (local address 127.0.0.1:58500)
0923 22:16:43.298888 (+ 54us) negotiation.cc:107] Waiting for socket to connect
0923 22:16:43.298898 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0923 22:16:43.298936 (+ 38us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0923 22:16:43.299072 (+ 136us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0923 22:16:43.299073 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0923 22:16:43.299223 (+ 150us) client_negotiation.cc:189] Negotiated authn=SASL
0923 22:16:43.299260 (+ 37us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 22:16:43.299261 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 22:16:43.300169 (+ 908us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 22:16:43.300170 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 22:16:43.300958 (+ 788us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 22:16:43.300960 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 22:16:43.301424 (+ 464us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 22:16:43.301425 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 22:16:43.301480 (+ 55us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 22:16:43.301859 (+ 379us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0923 22:16:43.301864 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0923 22:16:43.303316 (+ 1452us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0923 22:16:43.305146 (+ 1830us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0923 22:16:43.305147 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0923 22:16:43.305148 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0923 22:16:43.305373 (+ 225us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0923 22:16:43.305571 (+ 198us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0923 22:16:43.305572 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0923 22:16:43.305573 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0923 22:16:43.305645 (+ 72us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0923 22:16:43.306106 (+ 461us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0923 22:16:43.306107 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0923 22:16:43.306152 (+ 45us) client_negotiation.cc:769] Sending connection context
0923 22:16:43.306225 (+ 73us) client_negotiation.cc:240] Negotiation successful
0923 22:16:43.306311 (+ 86us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":41}
I20260923 22:16:43.306706 24346 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 22:16:43.298879 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58500 (local address 127.1.135.252:33555)
0923 22:16:43.298915 (+ 36us) server_negotiation.cc:206] Beginning negotiation
0923 22:16:43.298917 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0923 22:16:43.298925 (+ 8us) server_negotiation.cc:407] Connection header received
0923 22:16:43.299000 (+ 75us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 22:16:43.299001 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 22:16:43.299009 (+ 8us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 22:16:43.299060 (+ 51us) server_negotiation.cc:226] Negotiated authn=SASL
0923 22:16:43.299338 (+ 278us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 22:16:43.300045 (+ 707us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 22:16:43.301038 (+ 993us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 22:16:43.301360 (+ 322us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 22:16:43.301424 (+ 64us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 22:16:43.303407 (+ 1983us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0923 22:16:43.303408 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0923 22:16:43.303409 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0923 22:16:43.303415 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0923 22:16:43.305035 (+ 1620us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0923 22:16:43.305435 (+ 400us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0923 22:16:43.305436 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0923 22:16:43.305436 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0923 22:16:43.305490 (+ 54us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0923 22:16:43.305713 (+ 223us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0923 22:16:43.305714 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0923 22:16:43.305715 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0923 22:16:43.305838 (+ 123us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0923 22:16:43.305915 (+ 77us) server_negotiation.cc:1035] Waiting for connection context
0923 22:16:43.306557 (+ 642us) server_negotiation.cc:299] Negotiation successful
0923 22:16:43.306626 (+ 69us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":21}
I20260923 22:16:43.307050 24385 mvcc.cc:205] Tried to move back new op lower bound from 7332666586272296960 to 7332666586216943616. Current Snapshot: MvccSnapshot[applied={T|T < 7332666586272296960}]
I20260923 22:16:43.308108 24267 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 84f2054e82644722812a57bc3cc372e0: Bootstrap starting.
I20260923 22:16:43.308753 24267 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 84f2054e82644722812a57bc3cc372e0: Neither blocks nor log segments found. Creating new log.
I20260923 22:16:43.309056 24267 log.cc:824] T 00000000000000000000000000000000 P 84f2054e82644722812a57bc3cc372e0: Log is configured to *not* fsync() on all Append() calls
I20260923 22:16:43.309633 24348 sys_catalog.cc:455] T 00000000000000000000000000000000 P da6b6bf5f04f422db1db8315472a528b [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "7951ee4f672c48e29ed3b1b74a092b78" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7951ee4f672c48e29ed3b1b74a092b78" member_type: VOTER last_known_addr { host: "127.1.135.254" port: 37671 } } peers { permanent_uuid: "84f2054e82644722812a57bc3cc372e0" member_type: VOTER last_known_addr { host: "127.1.135.253" port: 39749 } } peers { permanent_uuid: "da6b6bf5f04f422db1db8315472a528b" member_type: VOTER last_known_addr { host: "127.1.135.252" port: 33555 } } }
I20260923 22:16:43.309707 24348 sys_catalog.cc:458] T 00000000000000000000000000000000 P da6b6bf5f04f422db1db8315472a528b [sys.catalog]: This master's current role is: FOLLOWER
I20260923 22:16:43.309799 24267 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 84f2054e82644722812a57bc3cc372e0: No bootstrap required, opened a new log
I20260923 22:16:43.310755 24380 catalog_manager.cc:1347] Generated new cluster ID: e348552bc8bb4cab8e67cf4020b1bb8a
I20260923 22:16:43.310892 24380 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260923 22:16:43.311682 24267 raft_consensus.cc:348] T 00000000000000000000000000000000 P 84f2054e82644722812a57bc3cc372e0 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7951ee4f672c48e29ed3b1b74a092b78" member_type: VOTER last_known_addr { host: "127.1.135.254" port: 37671 } } peers { permanent_uuid: "84f2054e82644722812a57bc3cc372e0" member_type: VOTER last_known_addr { host: "127.1.135.253" port: 39749 } } peers { permanent_uuid: "da6b6bf5f04f422db1db8315472a528b" member_type: VOTER last_known_addr { host: "127.1.135.252" port: 33555 } }
I20260923 22:16:43.311815 24267 raft_consensus.cc:729] T 00000000000000000000000000000000 P 84f2054e82644722812a57bc3cc372e0 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 84f2054e82644722812a57bc3cc372e0, State: Initialized, Role: FOLLOWER
I20260923 22:16:43.311890 24267 consensus_queue.cc:260] T 00000000000000000000000000000000 P 84f2054e82644722812a57bc3cc372e0 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7951ee4f672c48e29ed3b1b74a092b78" member_type: VOTER last_known_addr { host: "127.1.135.254" port: 37671 } } peers { permanent_uuid: "84f2054e82644722812a57bc3cc372e0" member_type: VOTER last_known_addr { host: "127.1.135.253" port: 39749 } } peers { permanent_uuid: "da6b6bf5f04f422db1db8315472a528b" member_type: VOTER last_known_addr { host: "127.1.135.252" port: 33555 } }
I20260923 22:16:43.312211 24267 sys_catalog.cc:564] T 00000000000000000000000000000000 P 84f2054e82644722812a57bc3cc372e0 [sys.catalog]: configured and running, proceeding with master startup.
I20260923 22:16:43.312325 24240 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 84f2054e82644722812a57bc3cc372e0 [term 0 FOLLOWER]: Advancing to term 1
I20260923 22:16:43.313174 24240 raft_consensus.cc:1261] T 00000000000000000000000000000000 P 84f2054e82644722812a57bc3cc372e0 [term 1 FOLLOWER]: Refusing update from remote peer 7951ee4f672c48e29ed3b1b74a092b78: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260923 22:16:43.313618 24389 consensus_queue.cc:1035] T 00000000000000000000000000000000 P 7951ee4f672c48e29ed3b1b74a092b78 [LEADER]: Connected to new peer: Peer: permanent_uuid: "84f2054e82644722812a57bc3cc372e0" member_type: VOTER last_known_addr { host: "127.1.135.253" port: 39749 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260923 22:16:43.313506 24241 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "7951ee4f672c48e29ed3b1b74a092b78" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: true dest_uuid: "84f2054e82644722812a57bc3cc372e0"
I20260923 22:16:43.314505 24241 raft_consensus.cc:2454] T 00000000000000000000000000000000 P 84f2054e82644722812a57bc3cc372e0 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7951ee4f672c48e29ed3b1b74a092b78 in term 1.
W20260923 22:16:43.314656 24220 tablet.cc:2369] T 00000000000000000000000000000000 P 84f2054e82644722812a57bc3cc372e0: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260923 22:16:43.317443 24407 sys_catalog.cc:455] T 00000000000000000000000000000000 P 84f2054e82644722812a57bc3cc372e0 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "7951ee4f672c48e29ed3b1b74a092b78" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7951ee4f672c48e29ed3b1b74a092b78" member_type: VOTER last_known_addr { host: "127.1.135.254" port: 37671 } } peers { permanent_uuid: "84f2054e82644722812a57bc3cc372e0" member_type: VOTER last_known_addr { host: "127.1.135.253" port: 39749 } } peers { permanent_uuid: "da6b6bf5f04f422db1db8315472a528b" member_type: VOTER last_known_addr { host: "127.1.135.252" port: 33555 } } }
I20260923 22:16:43.317529 24407 sys_catalog.cc:458] T 00000000000000000000000000000000 P 84f2054e82644722812a57bc3cc372e0 [sys.catalog]: This master's current role is: FOLLOWER
I20260923 22:16:43.317577 24407 sys_catalog.cc:455] T 00000000000000000000000000000000 P 84f2054e82644722812a57bc3cc372e0 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 7951ee4f672c48e29ed3b1b74a092b78. Latest consensus state: current_term: 1 leader_uuid: "7951ee4f672c48e29ed3b1b74a092b78" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7951ee4f672c48e29ed3b1b74a092b78" member_type: VOTER last_known_addr { host: "127.1.135.254" port: 37671 } } peers { permanent_uuid: "84f2054e82644722812a57bc3cc372e0" member_type: VOTER last_known_addr { host: "127.1.135.253" port: 39749 } } peers { permanent_uuid: "da6b6bf5f04f422db1db8315472a528b" member_type: VOTER last_known_addr { host: "127.1.135.252" port: 33555 } } }
I20260923 22:16:43.317625 24407 sys_catalog.cc:458] T 00000000000000000000000000000000 P 84f2054e82644722812a57bc3cc372e0 [sys.catalog]: This master's current role is: FOLLOWER
I20260923 22:16:43.317655 24407 sys_catalog.cc:455] T 00000000000000000000000000000000 P 84f2054e82644722812a57bc3cc372e0 [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "7951ee4f672c48e29ed3b1b74a092b78" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7951ee4f672c48e29ed3b1b74a092b78" member_type: VOTER last_known_addr { host: "127.1.135.254" port: 37671 } } peers { permanent_uuid: "84f2054e82644722812a57bc3cc372e0" member_type: VOTER last_known_addr { host: "127.1.135.253" port: 39749 } } peers { permanent_uuid: "da6b6bf5f04f422db1db8315472a528b" member_type: VOTER last_known_addr { host: "127.1.135.252" port: 33555 } } }
I20260923 22:16:43.317723 24407 sys_catalog.cc:458] T 00000000000000000000000000000000 P 84f2054e82644722812a57bc3cc372e0 [sys.catalog]: This master's current role is: FOLLOWER
I20260923 22:16:43.321139 24403 catalog_manager.cc:1259] Loaded cluster ID: e348552bc8bb4cab8e67cf4020b1bb8a
I20260923 22:16:43.321188 24403 catalog_manager.cc:1552] T 00000000000000000000000000000000 P 84f2054e82644722812a57bc3cc372e0: loading cluster ID for follower catalog manager: success
W20260923 22:16:43.321259 24403 catalog_manager.cc:1576] T 00000000000000000000000000000000 P 84f2054e82644722812a57bc3cc372e0: acquiring CA information for follower catalog manager: Not found: root CA entry not found
W20260923 22:16:43.321295 24403 catalog_manager.cc:873] Not found: root CA entry not found: failed to prepare follower catalog manager, will retry
I20260923 22:16:43.346068 24389 sys_catalog.cc:455] T 00000000000000000000000000000000 P 7951ee4f672c48e29ed3b1b74a092b78 [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "7951ee4f672c48e29ed3b1b74a092b78" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7951ee4f672c48e29ed3b1b74a092b78" member_type: VOTER last_known_addr { host: "127.1.135.254" port: 37671 } } peers { permanent_uuid: "84f2054e82644722812a57bc3cc372e0" member_type: VOTER last_known_addr { host: "127.1.135.253" port: 39749 } } peers { permanent_uuid: "da6b6bf5f04f422db1db8315472a528b" member_type: VOTER last_known_addr { host: "127.1.135.252" port: 33555 } } }
I20260923 22:16:43.346176 24389 sys_catalog.cc:458] T 00000000000000000000000000000000 P 7951ee4f672c48e29ed3b1b74a092b78 [sys.catalog]: This master's current role is: LEADER
I20260923 22:16:43.347585 24380 catalog_manager.cc:1370] Generated new certificate authority record
I20260923 22:16:43.348405 24380 catalog_manager.cc:1504] Loading token signing keys...
I20260923 22:16:43.353474 24380 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 7951ee4f672c48e29ed3b1b74a092b78: Generated new TSK 0
I20260923 22:16:43.353678 24380 catalog_manager.cc:1514] Initializing in-progress tserver states...
W20260923 22:16:43.377256 24195 negotiation.cc:337] Failed RPC negotiation. Trace:
0923 22:16:43.375519 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60048 (local address 127.1.135.254:37671)
0923 22:16:43.375561 (+ 42us) server_negotiation.cc:206] Beginning negotiation
0923 22:16:43.375564 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0923 22:16:43.376595 (+ 1031us) server_negotiation.cc:407] Connection header received
0923 22:16:43.376702 (+ 107us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 22:16:43.376703 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 22:16:43.376713 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 22:16:43.376744 (+ 31us) server_negotiation.cc:226] Negotiated authn=SASL
0923 22:16:43.377191 (+ 447us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:60048: BlockingRecv error: recv got EOF from 127.0.0.1:60048 (error 108)
Metrics: {"server-negotiator.queue_time_us":20}
W20260923 22:16:43.378144 24346 negotiation.cc:337] Failed RPC negotiation. Trace:
0923 22:16:43.375956 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58514 (local address 127.1.135.252:33555)
0923 22:16:43.375998 (+ 42us) server_negotiation.cc:206] Beginning negotiation
0923 22:16:43.376000 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0923 22:16:43.377648 (+ 1648us) server_negotiation.cc:407] Connection header received
0923 22:16:43.377750 (+ 102us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 22:16:43.377751 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 22:16:43.377762 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 22:16:43.377790 (+ 28us) server_negotiation.cc:226] Negotiated authn=SASL
0923 22:16:43.378088 (+ 298us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:58514: BlockingRecv error: recv got EOF from 127.0.0.1:58514 (error 108)
Metrics: {"server-negotiator.queue_time_us":22}
W20260923 22:16:43.378939 24271 negotiation.cc:337] Failed RPC negotiation. Trace:
0923 22:16:43.376331 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34338 (local address 127.1.135.253:39749)
0923 22:16:43.376370 (+ 39us) server_negotiation.cc:206] Beginning negotiation
0923 22:16:43.376372 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0923 22:16:43.378458 (+ 2086us) server_negotiation.cc:407] Connection header received
0923 22:16:43.378575 (+ 117us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 22:16:43.378576 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 22:16:43.378587 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 22:16:43.378614 (+ 27us) server_negotiation.cc:226] Negotiated authn=SASL
0923 22:16:43.378881 (+ 267us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:34338: BlockingRecv error: recv got EOF from 127.0.0.1:34338 (error 108)
Metrics: {"server-negotiator.queue_time_us":19}
W20260923 22:16:43.381189 24271 negotiation.cc:337] Failed RPC negotiation. Trace:
0923 22:16:43.380476 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34350 (local address 127.1.135.253:39749)
0923 22:16:43.380508 (+ 32us) server_negotiation.cc:206] Beginning negotiation
0923 22:16:43.380509 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0923 22:16:43.380710 (+ 201us) server_negotiation.cc:407] Connection header received
0923 22:16:43.380803 (+ 93us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 22:16:43.380804 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 22:16:43.380819 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 22:16:43.380847 (+ 28us) server_negotiation.cc:226] Negotiated authn=SASL
0923 22:16:43.381150 (+ 303us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:34350: BlockingRecv error: recv got EOF from 127.0.0.1:34350 (error 108)
Metrics: {"server-negotiator.queue_time_us":15}
W20260923 22:16:43.384176 24271 negotiation.cc:337] Failed RPC negotiation. Trace:
0923 22:16:43.383408 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34358 (local address 127.1.135.253:39749)
0923 22:16:43.383440 (+ 32us) server_negotiation.cc:206] Beginning negotiation
0923 22:16:43.383442 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0923 22:16:43.383665 (+ 223us) server_negotiation.cc:407] Connection header received
0923 22:16:43.383760 (+ 95us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 22:16:43.383760 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 22:16:43.383769 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 22:16:43.383797 (+ 28us) server_negotiation.cc:226] Negotiated authn=SASL
0923 22:16:43.384140 (+ 343us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:34358: BlockingRecv error: recv got EOF from 127.0.0.1:34358 (error 108)
Metrics: {"server-negotiator.queue_time_us":16}
W20260923 22:16:43.388309 24271 negotiation.cc:337] Failed RPC negotiation. Trace:
0923 22:16:43.387530 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34362 (local address 127.1.135.253:39749)
0923 22:16:43.387569 (+ 39us) server_negotiation.cc:206] Beginning negotiation
0923 22:16:43.387571 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0923 22:16:43.387790 (+ 219us) server_negotiation.cc:407] Connection header received
0923 22:16:43.387880 (+ 90us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 22:16:43.387881 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 22:16:43.387890 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 22:16:43.387917 (+ 27us) server_negotiation.cc:226] Negotiated authn=SASL
0923 22:16:43.388273 (+ 356us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:34362: BlockingRecv error: recv got EOF from 127.0.0.1:34362 (error 108)
Metrics: {"server-negotiator.queue_time_us":15}
W20260923 22:16:43.394179 24271 negotiation.cc:337] Failed RPC negotiation. Trace:
0923 22:16:43.393427 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34366 (local address 127.1.135.253:39749)
0923 22:16:43.393459 (+ 32us) server_negotiation.cc:206] Beginning negotiation
0923 22:16:43.393461 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0923 22:16:43.393685 (+ 224us) server_negotiation.cc:407] Connection header received
0923 22:16:43.393779 (+ 94us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 22:16:43.393780 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 22:16:43.393790 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 22:16:43.393818 (+ 28us) server_negotiation.cc:226] Negotiated authn=SASL
0923 22:16:43.394141 (+ 323us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:34366: BlockingRecv error: recv got EOF from 127.0.0.1:34366 (error 108)
Metrics: {"server-negotiator.queue_time_us":17}
W20260923 22:16:43.404444 24271 negotiation.cc:337] Failed RPC negotiation. Trace:
0923 22:16:43.403522 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34382 (local address 127.1.135.253:39749)
0923 22:16:43.403556 (+ 34us) server_negotiation.cc:206] Beginning negotiation
0923 22:16:43.403558 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0923 22:16:43.403782 (+ 224us) server_negotiation.cc:407] Connection header received
0923 22:16:43.403926 (+ 144us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 22:16:43.403927 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 22:16:43.403937 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 22:16:43.403965 (+ 28us) server_negotiation.cc:226] Negotiated authn=SASL
0923 22:16:43.404403 (+ 438us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:34382: BlockingRecv error: recv got EOF from 127.0.0.1:34382 (error 108)
Metrics: {"server-negotiator.queue_time_us":16}
W20260923 22:16:43.422780 24271 negotiation.cc:337] Failed RPC negotiation. Trace:
0923 22:16:43.421824 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34396 (local address 127.1.135.253:39749)
0923 22:16:43.421862 (+ 38us) server_negotiation.cc:206] Beginning negotiation
0923 22:16:43.421864 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0923 22:16:43.422189 (+ 325us) server_negotiation.cc:407] Connection header received
0923 22:16:43.422297 (+ 108us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 22:16:43.422298 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 22:16:43.422310 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 22:16:43.422341 (+ 31us) server_negotiation.cc:226] Negotiated authn=SASL
0923 22:16:43.422725 (+ 384us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:34396: BlockingRecv error: recv got EOF from 127.0.0.1:34396 (error 108)
Metrics: {"server-negotiator.queue_time_us":18}
W20260923 22:16:43.457106 24271 negotiation.cc:337] Failed RPC negotiation. Trace:
0923 22:16:43.456343 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34410 (local address 127.1.135.253:39749)
0923 22:16:43.456387 (+ 44us) server_negotiation.cc:206] Beginning negotiation
0923 22:16:43.456389 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0923 22:16:43.456530 (+ 141us) server_negotiation.cc:407] Connection header received
0923 22:16:43.456620 (+ 90us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 22:16:43.456621 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 22:16:43.456640 (+ 19us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 22:16:43.456694 (+ 54us) server_negotiation.cc:226] Negotiated authn=SASL
0923 22:16:43.457032 (+ 338us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:34410: BlockingRecv error: recv got EOF from 127.0.0.1:34410 (error 108)
Metrics: {"server-negotiator.queue_time_us":17}
W20260923 22:16:43.523310 24271 negotiation.cc:337] Failed RPC negotiation. Trace:
0923 22:16:43.522462 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34414 (local address 127.1.135.253:39749)
0923 22:16:43.522560 (+ 98us) server_negotiation.cc:206] Beginning negotiation
0923 22:16:43.522562 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0923 22:16:43.522680 (+ 118us) server_negotiation.cc:407] Connection header received
0923 22:16:43.522799 (+ 119us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 22:16:43.522801 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 22:16:43.522813 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 22:16:43.522884 (+ 71us) server_negotiation.cc:226] Negotiated authn=SASL
0923 22:16:43.523261 (+ 377us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:34414: BlockingRecv error: recv got EOF from 127.0.0.1:34414 (error 108)
Metrics: {"server-negotiator.queue_time_us":65}
W20260923 22:16:43.653457 24271 negotiation.cc:337] Failed RPC negotiation. Trace:
0923 22:16:43.652716 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34424 (local address 127.1.135.253:39749)
0923 22:16:43.652785 (+ 69us) server_negotiation.cc:206] Beginning negotiation
0923 22:16:43.652788 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0923 22:16:43.652909 (+ 121us) server_negotiation.cc:407] Connection header received
0923 22:16:43.652993 (+ 84us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 22:16:43.652995 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 22:16:43.653006 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 22:16:43.653065 (+ 59us) server_negotiation.cc:226] Negotiated authn=SASL
0923 22:16:43.653388 (+ 323us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:34424: BlockingRecv error: recv got EOF from 127.0.0.1:34424 (error 108)
Metrics: {"server-negotiator.queue_time_us":37}
W20260923 22:16:43.911422 24271 negotiation.cc:337] Failed RPC negotiation. Trace:
0923 22:16:43.910711 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34438 (local address 127.1.135.253:39749)
0923 22:16:43.910783 (+ 72us) server_negotiation.cc:206] Beginning negotiation
0923 22:16:43.910786 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0923 22:16:43.910796 (+ 10us) server_negotiation.cc:407] Connection header received
0923 22:16:43.910893 (+ 97us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 22:16:43.910895 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 22:16:43.910907 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 22:16:43.910982 (+ 75us) server_negotiation.cc:226] Negotiated authn=SASL
0923 22:16:43.911353 (+ 371us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:34438: BlockingRecv error: recv got EOF from 127.0.0.1:34438 (error 108)
Metrics: {"server-negotiator.queue_time_us":44}
I20260923 22:16:44.256723 24359 catalog_manager.cc:1259] Loaded cluster ID: e348552bc8bb4cab8e67cf4020b1bb8a
I20260923 22:16:44.256804 24359 catalog_manager.cc:1552] T 00000000000000000000000000000000 P da6b6bf5f04f422db1db8315472a528b: loading cluster ID for follower catalog manager: success
I20260923 22:16:44.257627 24359 catalog_manager.cc:1574] T 00000000000000000000000000000000 P da6b6bf5f04f422db1db8315472a528b: acquiring CA information for follower catalog manager: success
I20260923 22:16:44.257947 24359 catalog_manager.cc:1602] T 00000000000000000000000000000000 P da6b6bf5f04f422db1db8315472a528b: importing token verification keys for follower catalog manager: success; most recent TSK sequence number 0
I20260923 22:16:44.322450 24403 catalog_manager.cc:1574] T 00000000000000000000000000000000 P 84f2054e82644722812a57bc3cc372e0: acquiring CA information for follower catalog manager: success
I20260923 22:16:44.322887 24403 catalog_manager.cc:1602] T 00000000000000000000000000000000 P 84f2054e82644722812a57bc3cc372e0: importing token verification keys for follower catalog manager: success; most recent TSK sequence number 0
I20260923 22:16:44.427107 24474 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 22:16:44.424765 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44776 (local address 127.1.135.253:39749)
0923 22:16:44.424863 (+ 98us) server_negotiation.cc:206] Beginning negotiation
0923 22:16:44.424864 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0923 22:16:44.424879 (+ 15us) server_negotiation.cc:407] Connection header received
0923 22:16:44.424953 (+ 74us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 22:16:44.424953 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 22:16:44.425002 (+ 49us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 22:16:44.425088 (+ 86us) server_negotiation.cc:226] Negotiated authn=TOKEN
0923 22:16:44.425221 (+ 133us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 22:16:44.425695 (+ 474us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 22:16:44.426411 (+ 716us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 22:16:44.426619 (+ 208us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 22:16:44.426683 (+ 64us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 22:16:44.426817 (+ 134us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0923 22:16:44.426926 (+ 109us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0923 22:16:44.426984 (+ 58us) server_negotiation.cc:1035] Waiting for connection context
0923 22:16:44.427024 (+ 40us) server_negotiation.cc:299] Negotiation successful
0923 22:16:44.427042 (+ 18us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":79,"thread_start_us":51,"threads_started":1}
I20260923 22:16:44.431366 24480 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 22:16:44.428921 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34486 (local address 127.1.135.252:33555)
0923 22:16:44.429038 (+ 117us) server_negotiation.cc:206] Beginning negotiation
0923 22:16:44.429040 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0923 22:16:44.429048 (+ 8us) server_negotiation.cc:407] Connection header received
0923 22:16:44.429090 (+ 42us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 22:16:44.429091 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 22:16:44.429137 (+ 46us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 22:16:44.429221 (+ 84us) server_negotiation.cc:226] Negotiated authn=TOKEN
0923 22:16:44.429349 (+ 128us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 22:16:44.429852 (+ 503us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 22:16:44.430563 (+ 711us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 22:16:44.430778 (+ 215us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 22:16:44.430853 (+ 75us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 22:16:44.431022 (+ 169us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0923 22:16:44.431135 (+ 113us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0923 22:16:44.431210 (+ 75us) server_negotiation.cc:1035] Waiting for connection context
0923 22:16:44.431258 (+ 48us) server_negotiation.cc:299] Negotiation successful
0923 22:16:44.431280 (+ 22us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":96,"thread_start_us":54,"threads_started":1}
I20260923 22:16:44.435849 24474 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 22:16:44.433027 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44778 (local address 127.1.135.253:39749)
0923 22:16:44.433074 (+ 47us) server_negotiation.cc:206] Beginning negotiation
0923 22:16:44.433076 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0923 22:16:44.433247 (+ 171us) server_negotiation.cc:407] Connection header received
0923 22:16:44.433328 (+ 81us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 22:16:44.433329 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 22:16:44.433339 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 22:16:44.433393 (+ 54us) server_negotiation.cc:226] Negotiated authn=TOKEN
0923 22:16:44.433510 (+ 117us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 22:16:44.434054 (+ 544us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 22:16:44.435022 (+ 968us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 22:16:44.435333 (+ 311us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 22:16:44.435405 (+ 72us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 22:16:44.435539 (+ 134us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0923 22:16:44.435573 (+ 34us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0923 22:16:44.435624 (+ 51us) server_negotiation.cc:1035] Waiting for connection context
0923 22:16:44.435730 (+ 106us) server_negotiation.cc:299] Negotiation successful
0923 22:16:44.435754 (+ 24us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":23}
I20260923 22:16:44.436088 24480 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 22:16:44.433097 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34502 (local address 127.1.135.252:33555)
0923 22:16:44.433234 (+ 137us) server_negotiation.cc:206] Beginning negotiation
0923 22:16:44.433236 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0923 22:16:44.433426 (+ 190us) server_negotiation.cc:407] Connection header received
0923 22:16:44.433450 (+ 24us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 22:16:44.433450 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 22:16:44.433459 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 22:16:44.433507 (+ 48us) server_negotiation.cc:226] Negotiated authn=TOKEN
0923 22:16:44.433651 (+ 144us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 22:16:44.434353 (+ 702us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 22:16:44.435301 (+ 948us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 22:16:44.435620 (+ 319us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 22:16:44.435683 (+ 63us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 22:16:44.435810 (+ 127us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0923 22:16:44.435840 (+ 30us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0923 22:16:44.435891 (+ 51us) server_negotiation.cc:1035] Waiting for connection context
0923 22:16:44.435963 (+ 72us) server_negotiation.cc:299] Negotiation successful
0923 22:16:44.435985 (+ 22us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":118}
I20260923 22:16:44.436443 24486 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 22:16:44.433028 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37644 (local address 127.1.135.254:37671)
0923 22:16:44.433313 (+ 285us) server_negotiation.cc:206] Beginning negotiation
0923 22:16:44.433315 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0923 22:16:44.433324 (+ 9us) server_negotiation.cc:407] Connection header received
0923 22:16:44.433370 (+ 46us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 22:16:44.433371 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 22:16:44.433421 (+ 50us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 22:16:44.433514 (+ 93us) server_negotiation.cc:226] Negotiated authn=TOKEN
0923 22:16:44.433698 (+ 184us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 22:16:44.434446 (+ 748us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 22:16:44.435293 (+ 847us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 22:16:44.435645 (+ 352us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 22:16:44.435701 (+ 56us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 22:16:44.435900 (+ 199us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0923 22:16:44.436012 (+ 112us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0923 22:16:44.436102 (+ 90us) server_negotiation.cc:1035] Waiting for connection context
0923 22:16:44.436325 (+ 223us) server_negotiation.cc:299] Negotiation successful
0923 22:16:44.436344 (+ 19us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":260,"thread_start_us":62,"threads_started":1}
I20260923 22:16:44.441800 1567 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskRiQe_a/build/release/bin/kudu with pid 24124
I20260923 22:16:44.447006 1567 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskRiQe_a/build/release/bin/kudu with pid 24199
I20260923 22:16:44.451972 1567 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskRiQe_a/build/release/bin/kudu with pid 24274
2026-09-23T22:16:44Z chronyd exiting
[ OK ] ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys/1 (2772 ms)
[----------] 2 tests from ConnectToFollowerMasterTest (5595 ms total)
[----------] Global test environment tear-down
[==========] 56 tests from 4 test suites ran. (264945 ms total)
[ PASSED ] 55 tests.
[ FAILED ] 1 test, listed below:
[ FAILED ] SecurityITest.TestJwtMiniCluster
1 FAILED TEST